Research on realizing keep-alive principle by http client

Browsers always add the header information of Connection: Keep-Alive when requesting. We all know that on the server side (nginx), the connection retention time can be controlled by setting keepalive_timeout.
Does the maintenance of HTTP connections require browser (client) support? You will certainly know that you need client support. So today let's take a look at how the client maintains these http connections.
debug with java.net.HttpURLConnection.

Test code

package net.mengkang.demo;

import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;

public class Demo {
    public static void main(String[] args) throws IOException {
        test();
        test();
    }

    private static void test() throws IOException {
        URL url = new URL("http://static.mengkang.net/upload/image/2019/0921/1569075837628814.jpeg");

        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Charset", "UTF-8");
        connection.setRequestProperty("Connection", "Keep-Alive");
        connection.setRequestMethod("GET");
        connection.connect();

        BufferedInputStream bufferedInputStream = new BufferedInputStream(connection.getInputStream());

        File file = new File("./xxx.jpeg");
        OutputStream out = new FileOutputStream(file);
        int size;
        byte[] buf = new byte[1024];
        while ((size = bufferedInputStream.read(buf)) != -1) {
            out.write(buf, 0, size);
        }

        connection.disconnect();
    }
}

Parsing returned header information

When the client retrieves the returned byte stream from the server

connection.getInputStream()

HttpClientBy parsing the returned header information, I simplified the extraction of the most important logical code.

private boolean parseHTTPHeader(MessageHeader var1, ProgressSource var2, HttpURLConnection var3) throws IOException {
    String var15 = var1.findValue("Connection");
    ...
    if (var15 != null && var15.toLowerCase(Locale.US).equals("keep-alive")) {
        HeaderParser var11 = new HeaderParser(var1.findValue("Keep-Alive"));
        this.keepAliveConnections = var11.findInt("max", this.usingProxy ? 50 : 5);
        this.keepAliveTimeout = var11.findInt("timeout", this.usingProxy ? 60 : 5);
    }
    ...
}

Whether it is necessary to maintain a long connection is decided by the client, so the header information returned by the server should prevail. For example, the request sent by the client is Connection: Keep-Alive, and the return of the server is Connection: Close, which is also subject to the server.

Client Request Completion

When bufferedInputStream.read(buf) is first executed, HttpClient executes the finished() method

public void finished() {
    if (!this.reuse) {
        --this.keepAliveConnections;
        this.poster = null;
        if (this.keepAliveConnections > 0 && this.isKeepingAlive() && !this.serverOutput.checkError()) {
            this.putInKeepAliveCache();
        } else {
            this.closeServer();
        }

    }
}

Add to http long connection cache

protected static KeepAliveCache kac = new KeepAliveCache();

protected synchronized void putInKeepAliveCache() {
    if (this.inCache) {
        assert false : "Duplicate put to keep alive cache";

    } else {
        this.inCache = true;
        kac.put(this.url, (Object)null, this);
    }
}
public class KeepAliveCache extends HashMap<KeepAliveKey, ClientVector> implements Runnable {
    ...
    public synchronized void put(URL var1, Object var2, HttpClient var3) {
        KeepAliveKey var5 = new KeepAliveKey(var1, var2); // var2 null
        ClientVector var6 = (ClientVector)super.get(var5);
        if (var6 == null) {
            int var7 = var3.getKeepAliveTimeout();
            var6 = new ClientVector(var7 > 0 ? var7 * 1000 : 5000);
            var6.put(var3);
            super.put(var5, var6);
        } else {
            var6.put(var3);
        }
    }
    ...
}

This involves KeepAliveKey and Client Vector

class KeepAliveKey {
    private String protocol = null;
    private String host = null;
    private int port = 0;
    private Object obj = null;
}

This object is designed because only protocol+host+port can determine the same connection. So KeepAliveKey is used as the key of KeepAliveCache.
ClientVector is a stack, where requests from the same domain are stacked each time.

class ClientVector extends Stack<KeepAliveEntry> {
    private static final long serialVersionUID = -8680532108106489459L;
    int nap;

    ClientVector(int var1) {
        this.nap = var1;
    }

    synchronized void put(HttpClient var1) {
        if (this.size() >= KeepAliveCache.getMaxConnections()) {
            var1.closeServer();
        } else {
            this.push(new KeepAliveEntry(var1, System.currentTimeMillis()));
        }
    }
    ...
}

"Disconnect" Connection

connection.disconnect();

If a long connection is maintained, some streams are actually shut down, and the socket is not shut down.

public void disconnect() {
...
      boolean var2 = var1.isKeepingAlive();
      if (var2) {
          var1.closeIdleConnection();
      }
...
}
public void closeIdleConnection() {
    HttpClient var1 = kac.get(this.url, (Object)null);
    if (var1 != null) {
        var1.closeServer();
    }
}

Reuse of Connections

public static HttpClient New(URL var0, Proxy var1, int var2, boolean var3, HttpURLConnection var4) throws IOException {
    ...
    HttpClient var5 = null;
    if (var3) {
        var5 = kac.get(var0, (Object)null);
        ...
    }

    if (var5 == null) {
        var5 = new HttpClient(var0, var1, var2);
    } else {
        ...
        var5.url = var0;
    }

    return var5;
}
public class KeepAliveCache extends HashMap<KeepAliveKey, ClientVector> implements Runnable {
    ...
    public synchronized HttpClient get(URL var1, Object var2) {
        KeepAliveKey var3 = new KeepAliveKey(var1, var2);
        ClientVector var4 = (ClientVector)super.get(var3);
        return var4 == null ? null : var4.get();
    }
    ...
}

When ClientVector fetches, it exits the stack. If the connection has timed out, it closes the connection with the server and continues to execute the operation of exiting the stack.

class ClientVector extends Stack<KeepAliveEntry> {
    private static final long serialVersionUID = -8680532108106489459L;
    int nap;

    ClientVector(int var1) {
        this.nap = var1;
    }

    synchronized HttpClient get() {
        if (this.empty()) {
            return null;
        } else {
            HttpClient var1 = null;
            long var2 = System.currentTimeMillis();

            do {
                KeepAliveEntry var4 = (KeepAliveEntry)this.pop();
                if (var2 - var4.idleStartTime > (long)this.nap) {
                    var4.hc.closeServer();
                } else {
                    var1 = var4.hc;
                }
            } while(var1 == null && !this.empty());

            return var1;
        }
    }
    ...
}

In this way, the reuse of client http connection is realized.

Sketch summary


KeepAliveCache also scans every five seconds to detect expired httpClient s.

Tags: Java Nginx socket

Posted on Mon, 07 Oct 2019 10:55:35 -0700 by kellz