SSH Server Identification never received - Handshake Deadlock


Answers

Question

We're having some trouble trying to implement a Pool of SftpConnections for our application.

We're currently using SSHJ (Schmizz) as the transport library, and facing an issue we simply cannot simulate in our development environment (but the error keeps showing randomly in production, sometimes after three days, sometimes after just 10 minutes).

The problem is, when trying to send a file via SFTP, the thread gets locked in the init method from schmizz' TransportImpl class:

   @Override
    public void init(String remoteHost, int remotePort, InputStream in, OutputStream out)
            throws TransportException {
        connInfo = new ConnInfo(remoteHost, remotePort, in, out);

    try {

        if (config.isWaitForServerIdentBeforeSendingClientIdent()) {
            receiveServerIdent();
            sendClientIdent();
        } else {
            sendClientIdent();
            receiveServerIdent();
        }


        log.info("Server identity string: {}", serverID);

    } catch (IOException e) {
        throw new TransportException(e);
    }

    reader.start();
}

isWaitForServerIdentBeforeSendingClientIdent is FALSE for us, so first of all the client (we) send our identification, as appears in logs:

"Client identity String: blabla"

Then it's turn for the receiveServerIdent:

    private void receiveServerIdent() throws IOException 
{
        final Buffer.PlainBuffer buf = new Buffer.PlainBuffer();
        while ((serverID = readIdentification(buf)).isEmpty()) {
            int b = connInfo.in.read();
            if (b == -1)
                throw new TransportException("Server closed connection during identification exchange");
            buf.putByte((byte) b);
        }
    }

The thread never gets the control back, as the server never replies with its identity. Seems like the code is stuck in this While loop. No timeouts, or SSH exceptions are thrown, my client just keeps waiting forever, and the thread gets deadlocked.

This is the readIdentification method's impl:

private String readIdentification(Buffer.PlainBuffer buffer)
        throws IOException {
    String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
    if (ident.isEmpty()) {
        return ident;
    }

    if (!ident.startsWith("SSH-2.0-") && !ident.startsWith("SSH-1.99-"))
        throw new TransportException(DisconnectReason.PROTOCOL_VERSION_NOT_SUPPORTED,
                                     "Server does not support SSHv2, identified as: " + ident);

    return ident;
}

Seems like ConnectionInfo's inputstream never gets data to read, as if the server closed the connection (even if, as said earlier, no exception is thrown).

I've tried to simulate this error by saturating the negotiation, closing sockets while connecting, using conntrack to kill established connections while the handshake is being made, but with no luck at all, so any help would be HIGHLY appreciated.

: )




Tags