“This is the 10th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

primers

As the back-end architecture design is gradually shifting to micro-service architecture, the back-end development can concentrate on developing business logic (CRUD), which also leads to some developers do not know much about the processing mode of the client network request, because they only need to care about the interface entry and exit parameters and connect with the client developer.

I noticed this problem because during the troubleshooting on the Intranet, I found that there were dozens of abnormal time consuming records of interfaces and authentication failure (timeout) of some interfaces. Although these abnormal logs were few, THEY still aroused my alarm. After an analysis, it was found that the business logic of the interface was not complicated, so it located the device and found that the test sister was weakening the network test.

The reason for the authentication failure is that the time consumed by the server to receive the request has consumed the time of the entire interface, and the authentication service does not have much time, resulting in the authentication failure (timeout).

The authentication service in the project is independently deployed

What is weak networking

Most back-end developers don’t care what the user’s network looks like, or whether the client programmer queues requests, defends them, or throttles them, just how they are handled. In fact, with the popularity of mobile devices today, the network stability of most users is basically in a state of constant change.

For example, if a person walks into a toilet, their mobile phone network will likely fluctuate (some places even install signal blockers in the toilet).

For more details on weak networking, please refer to mobile IM Developers.

To understand the consequences of online volatility, we need to look at a classic interview question:

What stages does an HTTP request go through from sending to receiving to receiving a response?

In the absence of HTTP connection pooling:

  • Domain name resolution phase, which resolves the IP address of the server
  • Establishes a TCP connection with the server and sends HTTP packets
  • Wait for the server to respond. After reading the HTTP response packet, render the page or perform other operations

All of this discussion is based on HTTP1.1 for more details, read the twists and turns of an HTTP request

Taking the registered interface as an example, the time it takes for a request to be sent to receive the response data is broken down into the following sections

Obviously, a problem is that the user in the weak networking environment will lead to the unstable situation of sending request data and receiving response data, and then the client thinks that the HTTP request failed.

The server does not need to do anything about the request failure caused by a weak network because the request has not yet reached the business (it may be rejected by Nginx or the gateway as a Read timeout). The client needs to do retry logic.

We focus on the receiving response phase, where the client request is sent and processed by the back end, but the receiving response data phase fails.

  • For read-only interfaces, you do not need to perform any operation on the back end. You can retry the network request directly after the client fails.
  • For the data change interface, since the server has committed the transaction before the response message is written out, the server needs to handle request idempotency if the client retries. (idempotent processing is very practical but also very eight knowledge points belong to is)

SLOW POST

SLOW POST is one of the DDOS attacks. SLOW POST reads and sends data slowly, forcing the server to maintain a meaningless TCP connection for a long time, thus squeezing the server’s resources to the last drop.

They are associated with weak networking because they are both unstable in the way data is sent and received. The difference is the initiative.

Its implementation principle is very simple, only need to construct a valid HTTP request message, and then send them one by one byte or one at a time of receiving the response message, doing so is server operating system needs to maintain a malicious user long hours in the kernel of a TCP connection to squeeze the normal user resources, eventually lead to service disruptions.

For details, see slowhttpTest

defense

The defense works simply by setting a Read/Writetimeout for each HTTP request.

  • Returns if the request has not been read within the specified timeRequest timeouterror
  • If the response data is not written within the specified time, the connection is closed to release resources

This is decent because the authors of open source software have taken these issues into account and provided parameters to control them, but most of the default values of open source software are either set to a high value or not set at all, so we need to adjust to the actual situation.

Niginx

  • throughclient_header_timeoutandclient_body_timeoutTwo parameter assignments control reading HTTP request headers (HttpHeader) and request content (Body)
  • throughsend_timeoutTo control the timeout for sending response data

Please refer to this article for details

Tomcat

Adjusting the connectionTimeout number of Connector in server.xml to control the total request timeout is not as fine as Nginx

    <Connector port="8080" protocol="HTTP / 1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
Copy the code

Golang

Golang’s project can set the read and write timeout when initializing http.Server and can set the keep-alive timeout (how long the connection is closed without activity).

	server := http.Server{
		ReadTimeout:       5 * time.Second, / / read timeout
		ReadHeaderTimeout: 5 * time.Second, // The read header timed out
		WriteTimeout:      5 * time.Second, / / write timeout
		IdleTimeout:       5 * time.Second, / / Keep alive - timed out
	}
Copy the code

conclusion

  • Weak networking should be used to ensure interface idempotency
  • To deal withSLOW POSTAttack, set read and write timeout
  • Can be usedNginxYou don’t want to expose your services