post - Android HttpUrlConnection EOFException




example json (7)

I would like to know if there are known issues on Android with HttpUrlConnection and POST requests. We are experiencing intermittent EOFExceptions when making POST requests from an Android client. Retrying the same request will eventually work. Here is a sample stack trace:

java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:579)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:827)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)

There are many similar bug reports and posts to but I cannot understand if there really is an issue and if so, what versions of Android are affected and what the proposed fix/work around is.

Here are some of the similar reports I am referring to:

Here is a potential Android framework fix

I do know there was an issue with poisoned connections in the connection pool in pre-Froyo but these issues are occurring on new ICS+ devices exclusively. If there were a problem on later devices I would expect some kind of official Android documentation of the issue.


Answers

I suspect it might be the server that is at fault here, and the HttpURLConnection is not as forgiving as other implementations. That was the cause of my EOFException. I suspect in my case this would not be intermittent (fixed it before testing the N retry workaround), so the answers above relate to other issues and be a correct solution in those cases.

My server was using python SimpleHTTPServer and I was wrongly assuming all I needed to do to indicate success was the following:

self.send_response(200)

That sends the initial response header line, a server and a date header, but leaves the stream in the state where you are able to send additional headers too. HTTP requires an additional new line after headers to indicate they are finished. It appears if this new line isn't present when you attempt to get the result body InputStream or response code etc with HttpURLConnection then it throws the EOFException (which is actually reasonable, thinking about it). Some HTTP clients did accept the short response and reported the success result code which lead to me perhaps unfairly pointing the finger at HttpURLConnection.

I changed my server to do this instead:

self.send_response(200)
self.send_header("Content-Length", "0")
self.end_headers()

No more EOFException with that code.


HttpURLConnection library internally maintains a pool of Connections. So, whenever a request is send, it first checks if there is an existing connection already present in the pool, based on which it decides to create a new one.

These connections are nothing but sockets, and this library by default does not closes these sockets. It may sometimes happen that a connection (socket) which is not currently being used and is present in the pool is no longer usable as the Server may choose to terminate the connection after some time. Now, since the connection even though is closed by the server, the library does not knows about it and assumes the connection/socket to be still connected. Thus it sends the new request using this stale connection and hence we get EOFException.

The best way to handle this is to check the Response Headers after each request you send. The server always sends a "Connection: Close" before terminating a connection (HTTP 1.1). So, you can use getHeaderField() and check for "Connection" field. Another thing to note is that server ONLY sends this connection field when it is about to terminate the connection. So, you need to code around this with the possibility of getting a "null" in the normal case (when server is not closing the connection)


Our conclusion is that there is an issue in the Android platform. Our workaround was to catch the EOFException and retry the request N number of times. Below is the pseudo code:

private static final int MAX_RETRIES = 3;

private ResponseType fetchResult(RequestType request) {
    return fetchResult(request, 0);
}

private ResponseType fetchResult(RequestType request, int reentryCount) {
    try {
        // attempt to execute request
    } catch (EOFException e) {
        if (reentryCount < MAX_RETRIES) {
            fetchResult(request, reentryCount + 1);
        }
    }
    // continue processing response
}

Yes. There is a problem in the Android platform, specifically, in Android libcore with version 4.1-4.3.

The problem is introduced in this commit: https://android.googlesource.com/platform/libcore/+/b2b02ac6cd42a69463fd172531aa1f9b9bb887a8

Android 4.4 switched http lib to "okhttp" which doesn't have this problem.

Problem explained as follow:

On android 4.1-4.3, when you are using URLConnection/HttpURLConnection to POST with "ChunkedStreamingMode" or "FixedLengthStreamingMode" set, URLConnection/HttpURLConnection will not do silent retry if the reused connection is stale. You should retry POST at most "http.maxConnections+1" times in your code, just as previous answers suggest.


connection.addRequestProperty("Accept-Encoding", "gzip");

is the answer


This worked for me.

public ResponseObject sendPOST(String urlPrefix, JSONObject payload) throws JSONException {
    String line;
    StringBuffer jsonString = new StringBuffer();
    ResponseObject response = new ResponseObject();
    try {

        URL url = new URL(POST_URL);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setDoInput(true);
        connection.setDoOutput(true);
        connection.setReadTimeout(10000);
        connection.setConnectTimeout(15000);
        connection.setRequestMethod("POST");
        connection.setRequestProperty("Accept", "application/json");
        connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");

        OutputStream os = connection.getOutputStream();
        os.write(payload.toString().getBytes("UTF-8"));
        os.close();
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        while ((line = br.readLine()) != null) {
            jsonString.append(line);
        }
        response.setResponseMessage(connection.getResponseMessage());
        response.setResponseReturnCode(connection.getResponseCode());
        br.close();
        connection.disconnect();
    } catch (Exception e) {
        Log.w("Exception ",e);
        return response;
    }
    String json = jsonString.toString();
    response.setResponseJsonString(json);
    return response;
}

Complementing the @djechlin answer (good answer by the way!), this function call could be also used as dummy code to hold a breakpoint in an IDE when you want to stop in some specific iteration or a particular recursive call, for example:

isUserAGoat() could be used instead of a dummy variable declaration that will be shown in the IDE as a warning and, in Eclipse particular case, will clog the breakpoint mark, making it difficult to enable/disable it. If the method is used as a convention, all the invocations could be later filtered by some script (during commit phase maybe?).

Google guys are heavy Eclipse users (they provide several of their projects as Eclipse plugins: Android SDK, GAE, etc), so the @djechlin answer and this complementary answer make a lot of sense (at least for me).





android httpurlconnection