# Package com.codename1.io

package com.codename1.io

Networking, Storage, Filesystem & related API's

The IO package includes all of the main features related to storage and networking with the exception of `SQL` & XML parsing.

Storage

`com.codename1.io.Storage` is accessed via the `Storage` class. It is a flat filesystem like interface and contains the ability to list/delete and write to named storage entries.

The `com.codename1.io.Storage` API also provides convenient methods to write objects to `com.codename1.io.Storage` and read them from `com.codename1.io.Storage` specifically `readObject` & `writeObject`.

Notice that objects in `com.codename1.io.Storage` are deleted when an app is uninstalled but are retained between application updates.

The sample code below demonstrates listing the content of the storage, adding/viewing and deleting entries within the storage:

    public void pictureUpload(final Callback resultURL) {
        String picture = Capture.capturePhoto(1024, -1);
        if(picture!=null){
            String filestack = "https://www.filestackapi.com/api/store/S3?key=MY_KEY&filename=myPicture.jpg";
            MultipartRequest request = new MultipartRequest() {
               protected void readResponse(InputStream input) throws IOException  {
                  JSONParser jp = new JSONParser();
                  Map result = jp.parseJSON(new InputStreamReader(input, "UTF-8"));
                  String url = (String)result.get("url");
                  if(url == null) {
                     resultURL.onError(null, null, 1, result.toString());
                     return;
                  }
                  resultURL.onSucess(url);
               }
            };
            request.setUrl(filestack);
            try {
                request.addData("fileUpload", picture, "image/jpeg");
                request.setFilename("fileUpload", "myPicture.jpg");
                NetworkManager.getInstance().addToQueue(request);
            } catch(IOException err) {
                err.printStackTrace();
            }
        }
    }

The Preferences API

`com.codename1.io.Storage` also offers a very simple API in the form of the `com.codename1.io.Preferences` class. The `com.codename1.io.Preferences` class allows developers to store simple variables, strings, numbers, booleans etc. in storage without writing any storage code. This is a common use case within applications e.g. you have a server token that you need to store you can store & read it like this:

    // save a token to storage
    Preferences.set("token", myToken);

    // get the token from storage or null if it isn't there
    String token = Preferences.get("token", null);

This gets somewhat confusing with primitive numbers e.g. if you use `Preferences.set("primitiveLongValue", myLongNumber)` then invoke `Preferences.get("primitiveLongValue", 0)` you might get an exception!

This would happen because the value is physically a `Long` object but you are trying to get an `Integer`. The workaround is to remain consistent and use code like this `Preferences.get("primitiveLongValue", (long)0)`.

File System

`com.codename1.io.FileSystemStorage` provides file system access. It maps to the underlying OS's file system API providing most of the common operations expected from a file API somewhat in the vain of `java.io.File` & `java.io.FileInputStream` e.g. opening, renaming, deleting etc.

Notice that the file system API is somewhat platform specific in its behavior. All paths used the API should be absolute otherwise they are not guaranteed to work.

The main reason `java.io.File` & `java.io.FileInputStream` weren't supported directly has a lot to do with the richness of those two API's. They effectively allow saving a file anywhere, however mobile devices are far more restrictive and don't allow apps to see/modify files that are owned by other apps.

File Paths & App Home

All paths in `com.codename1.io.FileSystemStorage` are absolute, this simplifies the issue of portability significantly since the concept of relativity and current working directory aren't very portable.

All URL's use the `/` as their path separator we try to enforce this behavior even in Windows.

Directories end with the `/` character and thus can be easily distinguished by their name.

The `com.codename1.io.FileSystemStorage` API provides a `getRoots()` call to list the root directories of the file system (you can then "dig in" via the `listFiles` API). However, this is confusing and unintuitive for developers.

To simplify the process of creating/reading files we added the `getAppHomePath()` method. This method allows us to obtain the path to a directory where files can be stored/read.

We can use this directory to place an image to share as we did in the share sample.

Warning: A common Android hack is to write files to the SDCard storage to share them among apps. Android 4.x disabled the ability to write to arbitrary directories on the SDCard even when the appropriate permission was requested.

A more advanced usage of the `com.codename1.io.FileSystemStorage` API can be a `com.codename1.io.FileSystemStorage` `Tree`:

    Form hi = new Form("FileSystemTree", new BorderLayout());
    TreeModel tm = new TreeModel() {
    @Override
        public Vector getChildren(Object parent) {
            String[] files;
            if(parent == null) {
                files = FileSystemStorage.getInstance().getRoots();
                return new Vector(Arrays.asList(files));
            } else {
                try {
                    files = FileSystemStorage.getInstance().listFiles((String)parent);
                } catch(IOException err) {
                    Log.e(err);
                    files = new String[0];
                }
            }
            String p = (String)parent;
            Vector result = new Vector();
            for(String s : files) {
                result.add(p + s);
            }
            return result;
        }
    @Override
        public boolean isLeaf(Object node) {
            return !FileSystemStorage.getInstance().isDirectory((String)node);
        }
    };
    Tree t = new Tree(tm) {
    @Override
        protected String childToDisplayLabel(Object child) {
            String n = (String)child;
            int pos = n.lastIndexOf("/");
            if(pos

    Storage vs. File System

        The question of storage vs. file system is often confusing for novice mobile developers. This embeds
        two separate questions:

    -

    Why are there 2 API's where one would have worked?

    -

    Which one should I pick?

        The main reasons for the 2 API's are technical. Many OS's provide 2 ways of accessing data
        specific to the app and this is reflected within the API. E.g. on Android the
        `com.codename1.io.FileSystemStorage` maps to API's such as `java.io.FileInputStream`
        whereas the `com.codename1.io.Storage` maps to `Context.openFileInput()`.

        The secondary reason for the two API's is conceptual. `com.codename1.io.FileSystemStorage` is
        more powerful and in a sense provides more ways to fail, this is compounded by the complex
        on-device behavior of the API. `com.codename1.io.Storage` is designed to be friendlier to the
        uninitiated and more portable.

        You should pick `com.codename1.io.Storage` unless you have a specific requirement that prevents it.
        Some API's such as `Capture` expect a `com.codename1.io.FileSystemStorage` URI
        so in those cases this would also be a requirement.

        Another case where `com.codename1.io.FileSystemStorage` is beneficial is the case of hierarchy or
        native API usage. If you need a a directory structure or need to communicate with a native
        API the `com.codename1.io.FileSystemStorage` approach is usually easier.

        **Warning: **In some OS's the `com.codename1.io.FileSystemStorage` API can find the
        content of the `com.codename1.io.Storage` API. As one is implemented on top of the other. This is
        undocumented behavior that can change at any moment!

    Network Manager & Connection Request

        One of the more common problems in Network programming is spawning a new thread to handle
        the network operations. In Codename One this is done seamlessly and becomes unessential
        thanks to the `com.codename1.io.NetworkManager`.

        `com.codename1.io.NetworkManager` effectively alleviates the need for managing network threads by
        managing the complexity of network threading. The connection request class can be used to
        facilitate web service requests when coupled with the JSON/XML parsing capabilities.

        To open a connection one needs to use a `com.codename1.io.ConnectionRequest`
        object, which has some similarities to the networking mechanism in JavaScript but is obviously somewhat more
        elaborate.

    You can send a get request to a URL using something like:

    ```java
    ConnectionRequest request = new ConnectionRequest(url, false);
    request.addResponseListener((e) -> {
    // process the response
    });

    // request will be handled asynchronously
    NetworkManager.getInstance().addToQueue(request);

    Notice that you can also implement the same thing and much more by avoiding the response
    listener code and instead overriding the methods of the `com.codename1.io.ConnectionRequest` class
    which offers multiple points to override e.g.

    ConnectionRequest request = new ConnectionRequest(url, false) {
    protected void readResponse(InputStream input) {
    // just read from the response input stream
    }

    protected void postResponse() {
    // invoked on the EDT after processing is complete to allow the networking code
    // to update the UI
    }

    protected void buildRequestBody(OutputStream os) {
    // writes post data, by default this "just works" but if you want to write this
    // manually then override this
    }
    };
    NetworkManager.getInstance().addToQueue(request);

    Notice that overriding `buildRequestBody(OutputStream)` will only work for
    `POST` requests and will replace writing the arguments.

    **Important:** You don't need to close the output/input streams passed to the
    request methods. They are implicitly cleaned up.

    `com.codename1.io.NetworkManager` also supports synchronous requests which work in a similar
    way to `Dialog` via the `invokeAndBlock` call and thus don't block
    the EDT illegally. E.g. you can do something like this:

    ConnectionRequest request = new ConnectionRequest(url, false);
    // request will be handled synchronously
    NetworkManager.getInstance().addToQueueAndWait(request);
    byte[] resultOfRequest = request.getData();

    Notice that in this case the `addToQueueAndWait` method returned after the
    connection completed. Also notice that this was totally legal to do on the EDT!

Threading

    By default the `com.codename1.io.NetworkManager` launches with a single network thread. This is
    sufficient for very simple applications that don't do too much networking but if you need to
    fetch many images concurrently and perform web services in parallel this might be an issue.

    **Warning:** Once you increase the thread count there is no guarantee of order for your requests.
    Requests
    might not execute in the order with which you added them to the queue!

To update the number of threads use:

    NetworkManager.getInstance().updateThreadCount(4);

    All the callbacks in the `ConnectionRequest` occur on the network thread and
    **not on the EDT**!

    There is one exception to this rule which is the `postResponse()` method designed
    to update the UI after the networking code completes.

    **Important:** Never change the UI from a `com.codename1.io.ConnectionRequest`
    callback. You can either use a listener on the `com.codename1.io.ConnectionRequest`, use
    `postResponse()` (which is the only exception to this rule) or wrap your UI code with
    `com.codename1.ui.Display#callSerially(java.lang.Runnable)`.

Arguments, Headers & Methods

    HTTP/S is a complex protocol that expects complex encoded data for its requests. Codename
    One tries to simplify and abstract most of these complexities behind common sense API's while
    still providing the full low level access you would expect from such an API.

Arguments

    HTTP supports several "request methods", most commonly `GET` &
    `POST` but also a few others such as `HEAD`, `PUT`,
    `DELETE` etc.

Arguments in HTTP are passed differently between `GET` and `POST` methods. That is what the `setPost` method in Codename One determines, whether arguments added to the request should be placed using the `GET` semantics or the `POST` semantics.

So if we continue our example from above we can do something like this:

    ConnectionRequest request = new ConnectionRequest(url, false);
    request.addArgument("MyArgName", value);

    This will implicitly add a get argument with the content of `value`. Notice that we
    don't really care what value is. It's implicitly HTTP encoded based on the get/post semantics.
    In this case it will use the get encoding since we passed `false` to the constructor.

A simpler implementation could do something like this:

    ConnectionRequest request = new ConnectionRequest(url +
    "MyArgName=" + Util.encodeUrl(value), false);

    This would be almost identical but doesn't provide the convenience for switching back and
    forth between `GET`/`POST` and it isn't as fluent.

We can skip the encoding in complex cases where server code expects illegal HTTP characters (this happens) using the `addArgumentNoEncoding` method. We can also add multiple arguments with the same key using `addArgumentArray`.

Methods

    As we explained above, the `setPost()` method allows us to manipulate the
    get/post semantics of a request. This implicitly changes the `POST`
    or `GET` method submitted to the server.

    However, if you wish to have finer grained control over the submission process e.g. for making a
    `HEAD` request you can do this with code like:

    ConnectionRequest request = new ConnectionRequest(url, false);
    request.setHttpMethod("HEAD");

Headers

    When communicating with HTTP servers we often pass data within headers mostly for
    authentication/authorization but also to convey various properties.

    Some headers are builtin as direct API's e.g. content type is directly exposed within the API
    since it's a pretty common use case. We can set the content type of a post request using:

    ConnectionRequest request = new ConnectionRequest(url, true);
    request.setContentType("text/xml");

    We can also add any arbitrary header type we want, e.g. a very common use case is basic
    authorization where the authorization header includes the Base64 encoded user/password
    combination as such:

    String authCode = user + ":" + password;
    String authHeader = "Basic " + Base64.encode(authCode.getBytes());
    request.addRequestHeader("Authorization", authHeader);

This can be quite tedious to do if you want all requests from your app to use this header. For this use case you can just use:

    String authCode = user + ":" + password;
    String authHeader = "Basic " + Base64.encode(authCode.getBytes());
    NetworkManager.getInstance().addDefaultHeader("Authorization", authHeader);

Server Headers

    Server returned headers are a bit trickier to read. We need to subclass the connection request
    and override the `readHeaders` method e.g.:

    ConnectionRequest request = new ConnectionRequest(url, false) {
    protected void readHeaders(Object connection) throws IOException {
    String[] headerNames = getHeaderFieldNames(connection);
    for(String headerName : headerNames) {
    String headerValue = getHeader(headerName);
    //....
    }
    }
    protected void readResponse(InputStream input) {
    // just read from the response input stream
    }
    };
    NetworkManager.getInstance().addToQueue(request);

    Here we can extract the headers one by one to handle complex headers such as cookies,
    authentication etc.

Error Handling

    As you noticed above practically all of the methods in the `ConectionRequest`
    throw `IOException`. This allows you to avoid the `try`/`catch`
    semantics and just let the error propagate up the chain so it can be handled uniformly by
    the application.

There are two distinct placed where you can handle a networking error:

The `com.codename1.io.ConnectionRequest` - by overriding callback methods

The `com.codename1.io.NetworkManager` error handler

    Notice that the `com.codename1.io.NetworkManager` error handler takes precedence thus allowing
    you to define a global policy for network error handling by consuming errors.

    E.g. if I would like to block all network errors from showing anything to the user I could do
    something like this:

    NetworkManager.getInstance().addToQueue(request);
    NetworkManager.getInstance().addErrorListener((e) -> e.consume());

    The error listener is invoked first with the `com.codename1.io.NetworkEvent` matching the
    error. Consuming the event prevents it from propagating further down the chain into the
    `com.codename1.io.ConnectionRequest` callbacks.

    We can also override the error callbacks of the various types in the request e.g. in the case of a
    server error code we can do:

    ConnectionRequest request = new ConnectionRequest(url, false) {
    protected void handleErrorResponseCode(int code, String message) {
    if(code == 444) {
    // do something
    }
    }
    protected void readResponse(InputStream input) {
    // just read from the response input stream
    }
    };
    NetworkManager.getInstance().addToQueue(request);

    **Important:** The error callback callback is triggered in the network thread!

    As a result it can't access the UI to show a `Dialog` or anything like that.

    Another approach is to use the `setFailSilently(true)` method on the
    `com.codename1.io.ConnectionRequest`. This will prevent the
    `com.codename1.io.ConnectionRequest` from displaying any errors to the user. It's a very
    powerful strategy if you use the synchronous version of the API's e.g.:

    ConnectionRequest request = new ConnectionRequest(url, false);
    request.setFailSilently(true);
    NetworkManager.getInstance().addToQueueAndWait(request);
    if(request.getResponseCode() != 200) {
    // probably an error...
    }

    This code will only work with the synchronous "AndWait" version of the method since the response
    code will take a while to return for the non-wait version.

Error Stream

    When we get an error code that isn't 200/300 we ignore the result. This is problematic as the
    result might contain information we need. E.g. many webservices provide further XML/JSON
    based details describing the reason for the error code.

    Calling `setReadResponseForErrors(true)` will trigger a mode where even errors
    will receive the `readResponse` callback with the error stream. This also means
    that API's like `getData` and the listener API's will also work correctly in
    case of error.

GZIP

    Gzip is a very common compression format based on the lz algorithm, it's used by web servers
    around the world to compress data.

    Codename One supports `com.codename1.io.gzip.GZIPInputStream` and
    `com.codename1.io.gzip.GZIPOutputStream`, which allow you to compress data
    seamlessly into a stream and extract compressed data from a stream. This is very useful and
    can be applied to every arbitrary stream.

    Codename One also features a `com.codename1.io.gzip.GZConnectionRequest`, which
    will automatically unzip an HTTP response if it is indeed gzipped. Notice that some devices (iOS)
    always request gzip'ed data and always decompress it for us, however in the case of iOS it
    doesn't remove the gziped header. The `GZConnectionRequest` is aware of such
    behaviors so its better to use that when connecting to the network (if applicable).

    By default `GZConnectionRequest` doesn't request gzipped data (only unzips it
    when its received) but its pretty easy to do so just add the HTTP header
    `Accept-Encoding: gzip` e.g.:

    GZConnectionRequest con = new GZConnectionRequest();
    con.addRequestHeader("Accept-Encoding", "gzip");

Do the rest as usual and you should have smaller responses from the servers.

File Upload

    `com.codename1.io.MultipartRequest` tries to simplify the process of uploading a file from
    the local device to a remote server.

    You can always submit data in the `buildRequestBody` but this is flaky and has
    some limitations in terms of devices/size allowed. HTTP standardized file upload capabilities
    thru the multipart request protocol, this is implemented by countless servers and is well
    documented. Codename One supports this out of the box.

    Since we assume most developers reading this will be familiar with Java here is the way to
    implement the multipart upload in the servlet API.

    // File: MultipartClientSample.java
    MultipartRequest request = new MultipartRequest();
    request.setUrl(url);
    request.addData("myFileName", fullPathToFile, "text/plain")
    NetworkManager.getInstance().addToQueue(request);

    // File: UploadServlet.java

- Related Packages
  Package
  Description
  com.codename1.io.gzip
  gzip support based on https://github.com/ymnk/jzlib
  com.codename1.io.rest
  Simplified REST API
  com.codename1.io.services
  Default WebService implementations
  com.codename1.io.tar
  Tar support allowing us to package multiple separate files into a single package which is useful in supporting hierarchies in a portable way.
- Class
  Description
  AccessToken
  This class represent an access token.
  BufferedInputStream
  Based on the buffered input stream from the JDK with some minor tweaks to allow external classes to monitor stream status and progress.
  BufferedOutputStream
  Based on the buffered output stream from the JDK with some minor tweaks to allow external classes to monitor stream status and progress.
  CacheMap
  A cache map is essentially a hashtable that indexes entries based on age and is limited to a fixed size.
  CharArrayReader
  A specialized `Reader` for reading the contents of a char array.
  ConnectionRequest
  This class represents a connection object in the form of a request response typically common for HTTP/HTTPS connections.
  ConnectionRequest.CachingMode
  There are 5 caching modes:
  ConnectionRequest.SSLCertificate
  Encapsulates an SSL certificate fingerprint.
  Cookie
  A cookie for an HTTP request
  CSVParser
  The `CSVParser` allows importing data into applications quickly from a CSV source.
  Data
  An interface that can be implemented by any object to allow it to be treated as data.
  Data.ByteData
  Wraps a byte[] array as a Data object.
  Data.FileData
  Wraps a File as a Data object.
  Data.StorageData
  Wraps a Storage object as a Data object.
  Data.StringData
  Externalizable
  `Externalizable` is similar to the Java SE `Externalizable` interface this interface.
  File
  This class provides a similar API to `java.io.File` making it almost into a "drop in" replacement.
  File.FileFilter
  Interface for filtering files.
  File.FilenameFilter
  Interface to filter filenames.
  FileSystemStorage
  Unlike networking, the file system storage mostly tries to emulate java.io.File with some simplifications for mobile devices.
  IOProgressListener
  Callback for IO updates from a buffered input/output stream
  JSONParseCallback
  The event based parser allows parsing without creating an object tree by receiving callbacks to this class.
  JSONParser
  Fast and dirty parser for JSON content on the web, it essentially returns a `java.util.Map` object containing the object fields mapped to their values.
  Log
  Pluggable logging framework that allows a developer to log into storage using the file connector API.
  MalformedURLException
  Thrown when attempting to create a URL that doesn't conform to specification.
  MultipartRequest
  A multipart post request allows a developer to submit large binary data files to the server in a multipart mime post request.
  NetworkEvent
  Event containing more meta data for network events which may be error events or an update for progress indication code.
  NetworkManager
  Main entry point for managing the connection requests, this is essentially a threaded queue that makes sure to route all connections via the network thread while sending the callbacks through the Codename One EDT.
  Oauth2
  This is a utility class that allows Oauth2 authentication This utility uses the Codename One XHTML Component to display the authentication pages.
  Oauth2.RefreshTokenRequest
  PreferenceListener
  Respond to changes to the Preferences
  Preferences
  Simple map like class to store application and Codename One preference settings in the `com.codename1.io.Storage`.
  Properties
  A `Properties` object is a `Hashtable` where the keys and values must be `String`s.
  Socket
  Class implementing the socket API
  Socket.StopListening
  This interface can be invoked to stop listening on a server socket
  SocketConnection
  Callback for establishment of a socket connection.
  Storage
  Abstracts the underlying application specific storage system, unlike the `com.codename1.io.FileSystemStorage` this class is a higher level abstraction.
  URL
  This class provides a similar API to `URL` making it almost into a "drop in" replacement.
  Util
  Various utility methods used for HTTP/IO operations
  WebServiceProxyCall
  Utility class used by the webservice proxy code to invoke server code
  WebServiceProxyCall.WSDefinition
  Webservice definition type, allows defining the argument values for a specific WS call