golang tcp handshake / wave experiment, close ﹣ wait optimization

Three handshakes, four waves, as is often said in books
The following is an excerpt from other blogs

Gorang's tcp programming is relatively simple, but if we grab packets, we will find that in many cases, especially in the explicit complex network environment, it is difficult to repeat four waves

Experiment one repeated three handshakes and four waves

tcp server

package main

import (
	"fmt"
	"io"
	"net"
)

func main() {
	addr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:8092")
	if err != nil {
		panic(err)
	}
	listener, err := net.ListenTCP("tcp", addr)
	if err != nil {
		panic(err)
	}
	defer listener.Close()
	fmt.Println("start listening")
	for {
		tcpConn, err := listener.AcceptTCP()
		if err != nil {
			fmt.Println(err)
			continue
		}
		fmt.Println("new connection:" + tcpConn.RemoteAddr().String())
		go handle(tcpConn)
	}
}

func handle(conn *net.TCPConn) {
	conn.Close()
	return
}

tcp client

package main

import (
	"fmt"
	"net"
	"time"
)

func main() {
	tcpAddr, _ := net.ResolveTCPAddr("tcp","127.0.0.1:8092")
	conn,err := net.DialTCP("tcp",nil,tcpAddr)
	if err!=nil {
		panic(err)
	}

	defer conn.Close()

	fmt.Println(conn.LocalAddr().String() + " connection establish")

	onMessageReceived(conn)
}

func onMessageReceived(conn *net.TCPConn) {
	conn.Close()
	return
}

server listens to the connection, disconnect immediately
When the client successfully connects, disconnect it immediately

Using wireshark to grab the lo (127.0.0.1) network card, you can see

Port used by client: 53236
Port used by server: 8092

Look at the screenshot. It's basically the same as the three handshakes and four waves mentioned above

In fact, there may not be four complete waves

The above example is clean without any business code. In fact, when there is business code, it may appear that one party closes the connection first.

Experiment 2: server listens all the time, client disconnects first

Change the code slightly. The server listen s to the connection all the time. The client sends data ("hello") once, and then actively disconnects the connection

server

package main

import (
	"fmt"
	"io"
	"net"
)

func main() {
	addr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:8092")
	if err != nil {
		panic(err)
	}
	listener, err := net.ListenTCP("tcp", addr)
	if err != nil {
		panic(err)
	}
	defer listener.Close()
	fmt.Println("start listening")
	for {
		tcpConn, err := listener.AcceptTCP()
		if err != nil {
			fmt.Println(err)
			continue
		}
		fmt.Println("new connection:" + tcpConn.RemoteAddr().String())
		go handle(tcpConn)
	}
}

func handle(conn *net.TCPConn) {
	for {
		buf := make([]byte, 10*1024)
		_, err := conn.Read(buf)
		if err != nil {
			if err == io.EOF {
				continue
			} else {
				fmt.Printf("Read err:%v close\n", err)
				conn.Close()
				break
			}
		}
		fmt.Printf("Read data:%v", string(buf))
	}
}

client

package main

import (
	"fmt"
	"net"
)

func main() {
	tcpAddr, _ := net.ResolveTCPAddr("tcp", "127.0.0.1:8092")
	conn, err := net.DialTCP("tcp", nil, tcpAddr)
	if err != nil {
		panic(err)
	}

	defer conn.Close()

	fmt.Println(conn.LocalAddr().String() + " connection establish")

	onMessageReceived(conn)
}

func onMessageReceived(conn *net.TCPConn) {
	_, err := conn.Write([]byte("hello\n"))
	if err != nil {
		panic(err)
	}
	conn.Close()
}

Capture screenshots

If the client explicitly disconnects, the server will still send keep alive packet to the client. It can be seen from the time stamp that the duration from the first keep alive package to the last keep alive package is about 60s, which is generally 2MSL
In linux system, this parameter is located in

$ cat /proc/sys/net/ipv4/tcp_fin_timeout
60

This parameter just corresponds to the duration of keep alive package

If you look at port 8092 through netstat when sending a keep alive packet, you can see that the connection status changes to

$ netstat -ano|grep 8092
tcp        0      0 127.0.0.1:8092          0.0.0.0:*               LISTEN      off (0.00/0/0)
tcp        0      0 127.0.0.1:8092          127.0.0.1:60204         CLOSE_WAIT  keepalive (9.12/0/0)

This situation is quite common. For tcp/http clients, the client is usually disconnected first, then the connection will remain closed and wait for a period of time, and finally disconnected.
Therefore, in this experiment, three handshakes, three waves, and the last wave is RST response, indicating that the connection has been reset.

Experiment 3: the server disconnects first, and the client sends data all the time

Client: just use the client of Experiment 2,
Server: just use the server of Experiment 1

In this case, there are three handshakes and three waves. Compared with experiment 2, there is less 60 seconds of keep alive package

HTTP client close? Wait optimization

One of the optimization directions is to reuse (keep alive) connections. From Experiment 2, it can be seen that keep alive is a heartbeat mechanism. If the client disconnects and re uses the connection within a period of time, the number of close and wait in the system can be reduced

http client uses the default transport when no transport is specified. You can use the following code

// go/src/net/http/client.go
func (c *Client) transport() RoundTripper {
	if c.Transport != nil {
		return c.Transport
	}
	return DefaultTransport
}

// DefaultTransport is the default implementation of Transport and is
// used by DefaultClient. It establishes network connections as needed
// and caches them for reuse by subsequent calls. It uses HTTP proxies
// as directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and
// $no_proxy) environment variables.
var DefaultTransport RoundTripper = &Transport{
	Proxy: ProxyFromEnvironment,
	DialContext: (&net.Dialer{
		Timeout:   30 * time.Second,
		KeepAlive: 30 * time.Second,
		DualStack: true,
	}).DialContext,
	ForceAttemptHTTP2:     true,
	MaxIdleConns:          100,
	IdleConnTimeout:       90 * time.Second,
	TLSHandshakeTimeout:   10 * time.Second,
	ExpectContinueTimeout: 1 * time.Second,
}

transport also has a default parameter DefaultMaxIdleConnsPerHost=2

const DefaultMaxIdleConnsPerHost = 2
...
type Transport struct {
	// MaxIdleConns controls the maximum number of idle (keep-alive)
	// connections across all hosts. Zero means no limit.
	MaxIdleConns int

	// MaxIdleConnsPerHost, if non-zero, controls the maximum idle
	// (keep-alive) connections to keep per-host. If zero,
	// DefaultMaxIdleConnsPerHost is used.
	MaxIdleConnsPerHost int
}

Here is a custom connection state, idle, that is, keep alive connection
MaxIdleConnsPerHost: a target address. By default, there are only two idel connections. Generally, for internal forwarding services, you can consider enlarging this value to improve the performance of http client

There are several other Timeout parameters for Transport. You can try them as needed

41 original articles published, praised 12, visited 120000+
Private letter follow

Tags: network Programming Linux less

Posted on Sun, 15 Mar 2020 01:36:00 -0700 by cursed