Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money

When I was working recently, I needed to test the performance of an access service. Unexpectedly, I detected a problem with full Linux handles

What exactly is the problem, let’s take a look

Normal operation

In the project, some HTTP requests are written like this:

  • requesthttpsIn order to bypasstls, plus theTLSClientConfig: &tls.Config{InsecureSkipVerify: true}configuration
  • Normal access to the requested address we need
  • Get the data we want normally, parse normally
func main(a) {
	client := http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
		},
	}
	resp, err := client.Get("https://www.xxxxxx.com")
	iferr ! =nil {
		fmt.Println("Get err : ", err)
		return
	}
	defer resp.Body.Close()
	body, err := ioutil.ReadAll(resp.Body)
	fmt.Println(string(body))
}
Copy the code

For example, the following is the result of visiting Baidu, what is wrong with it

t# go run main.go<html> <head> <script> location.replace(location.href.replace("https://","http://")); </script> </head> <body> <noscript><meta http-equiv="refresh" content="0; url=http://www.baidu.com/"></noscript> </body> </html>Copy the code

Found the problem

However, when the request code like this is used for performance testing, the actual problem we encounter is that the ** Linux handle ** is full

The number of handles is full. Simple thinking has the following two preliminary possibilities:

  • The number of Linux handles is too small. Procedure
  • The HTTP code does not release the connection

I know of three ways to change the number of handles on Linux:

1. Modify /etc/profile

Modify /etc/profile directly by adding the following statement at the end of the file

ulimit -n 65535
Copy the code

For example, set the number of handles to 65535

Execution after Modification

 source /etc/profile
Copy the code

See the effect

ulimit -a
Copy the code

2, modify,limits.conffile

Modify limits. Conf directly to make it effective

vim /etc/security/limits.conf
Copy the code

3. Modify the login file

We can add the bottom line to the /etc/pam.d/login file

 session   required   pam_limits.so
Copy the code

For example, add as above

For methods 2 and 3 to take effect, you need to re-ssh the server or restart the server

Although I increased the number of Linux handles, I found that in the performance test, it was only a little longer, but still the connection number was full, why?

Taking a closer look at the code, the code also has the HTTP connection closed

So the question is where?

Find a problem and solve it

A closer look at the code leaves only one doubt, and that is the following sentence

client := http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true}},}Copy the code

This sentence was originally used to bypass TLS without much thought, but now take a closer look at the functionality inside the HTTP.transport structure

type Transport struct {
	idleMu       sync.Mutex
	closeIdle    bool                                // user has requested to close all idle conns
	idleConn     map[connectMethodKey][]*persistConn // most recently used at end
	idleConnWait map[connectMethodKey]wantConnQueue  // waiting getConns
	idleLRU      connLRU

	reqMu       sync.Mutex
	reqCanceler map[cancelKey]func(error)

	altMu    sync.Mutex   // guards changing altProto only
	altProto atomic.Value // of nil or map[string]RoundTripper, key is URI scheme

	connsPerHostMu   sync.Mutex
	connsPerHost     map[connectMethodKey]int
	connsPerHostWait map[connectMethodKey]wantConnQueue // waiting getConns. Omit multiple lines...........// TLSClientConfig specifies the TLS configuration to use with
	// tls.Client.
	// If nil, the default configuration is used.
	// If non-nil, HTTP/2 support may not be enabled by default.
	TLSClientConfig *tls.Config

	// TLSHandshakeTimeout specifies the maximum amount of time waiting to
	// wait for a TLS handshake. Zero means no timeout.
	TLSHandshakeTimeout time.Duration

	// DisableKeepAlives, if true, disables HTTP keep-alives and
	// will only use the connection to the server for a single
	// HTTP request.
	//
	// This is unrelated to the similarly named TCP keep-alives.
	DisableKeepAlives bool. Omit multiple lines........... }Copy the code

DisableKeepAlives disables long connections. Each request creates a connection and closes the connection immediately after the request is completed

The problem is solved when Transport is set up correctly

client := http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
			DisableKeepAlives: true,}}Copy the code

The problem appears to be that HTTP.Transport is not set up properly

In fact, go HTTP package for connection management we have not been familiar with him, for his specific implementation of the connection and details of the code, the future opportunity to share

After the code is modified and the performance test is normally passed, we must be in awe of the technology and the code

Welcome to like, follow and favorites

Friends, your support and encouragement, I insist on sharing, improve the quality of the power

All right, that’s it for this time

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am Nezha, welcome to like, see you next time ~