preface

When using Tomcat, you often encounter configuration problems such as connection number and thread number. To truly understand these concepts, you must first understand The Connector of Tomcat.

The main function of Connector is to receive connection requests and create Request and Response objects for exchanging data with requesters. Threads are then allocated to Engine (that is, the Servlet container) to handle the Request and pass the resulting Request and Response objects to Engine. When the Engine completes processing the request, the response is also sent back to the client through the Connector.

Servlet container processing requests requires Connector scheduling and control. Connector is the backbone of Tomcat request processing. Therefore, the configuration and use of Connector has an important impact on Tomcat performance. This article will start with connectors and discuss some important issues related to connectors, including NIO/BIO mode, thread pools, connection counts, and more.

According to different protocols, the Connector can be divided into HTTP Connector, AJP Connector, etc. This article only discusses HTTP Connector.

Nio, Bio, APR

1. Connector protocol

Connector uses a different protocol when handling HTTP requests. Different versions of Tomcat support different protocols. The most typical protocols are BIO, NIO, and APR (which are supported in Tomcat7. Removed BIO support).

BIO is a Blocking IO. NIO is a non-blocking I/O. And APR is Apache Portable Runtime, is Apache Portable Runtime library, using the local library can achieve high scalability, high performance; Apr is the preferred mode for running highly concurrent applications on Tomcat, but packages such as Apr, APr-utils, tomcat-native, etc.

2, how to specify protocol

The protocol used by Connector can be specified through the protocol attribute in the < Connector > element, or the default value can be used.

The value of protocol and its protocol are as follows:

• HTTP/1.1: the default value, the protocol used depends on the Version of Tomcat
• Org. Apache. Coyote. Http11. Http11Protocol: BIO
• Org. Apache. Coyote. Http11. Http11NioProtocol: NIO
• Org. Apache. Coyote. Http11. Http11Nio2Protocol: NIO2
• Org. Apache. Coyote. Http11. Http11AprProtocol: APR

If protocol is not specified, the default HTTP/1.1 is used, which means the following: in Tomcat7, either BIO or APR is automatically selected (APR is used if a local library for APR is found, BIO is used otherwise); In Tomcat8, NIO or APR is automatically selected (APR if you find a local library for APR, NIO otherwise).

3. How is BIO/NIO different

Whether the BIO or NIO Connector processes a request, the general process is the same:

Receives the connection in the Accept queue (when the client sends a request to the server, if the client establishes a connection with the OS after completing a three-way handshake, the OS puts the connection in the Accept queue); Get the requested data in the connection and generate the request; Call the servlet container to process the request; Returns the response. The connection is at the TCP level (transport layer), corresponding to sockets. The request is HTTP layer (application layer), must rely on TCP connection implementation; Multiple HTTP requests can be sent over a TCP connection.

In the Connector implemented by BIO, the primary entity handling requests is the JIoEndpoint object. JIoEndpoint maintains acceptors and workers: Acceptors receive sockets and process the sockets from the Worker thread pool for free threads. If there are no free threads in the Worker thread pool, the Acceptor blocks. Worker is Tomcat’s own thread pool. If other thread pools are configured through <Executor>, the working principle is similar to Worker.

In the Connector of the NIO implementation, the primary entity handling requests is the NIoEndpoint object. In addition to Acceptor and Worker acceptors, the NIoEndpoint also uses Poller, which is processed as shown in the following figure:

After receiving the socket, an Acceptor sends the request to a Poller, which is key to implementing NIO, rather than processing the request directly using the Worker thread. Acceptors send requests to pollers via queues, using the typical producer-consumer pattern. In Poller, we maintain a Selector object; When the Poller removes the socket from the queue, it registers it with that Selector. It then iterates through the Selector to find the socket that is readable and uses the thread in the Worker to process the request. Like BIO, workers can be replaced by custom thread pools.

According to the above process, NIoEndpoint still uses blocking mode to process requests, whether acceptors accept sockets or threads process requests. However, in the process of “reading the socket and handing it to the thread in the Worker”, a non-blocking NIO implementation is used, which is the main difference between NIO mode and BIO mode (the other differences have less impact on performance and will be left out for now). This difference can significantly improve Tomcat efficiency in the case of large concurrency:

Most HTTP requests currently use long connections (HTTP/1.1 default keep-alive is true). Long connections mean that a TCP socket does not release any new requests immediately after the current request ends. Instead, the socket waits for a timeout before releasing them. If the BIO is used, the process of “reading the socket and handing it to the Worker thread” is blocked, which means that the Worker thread processing the socket is occupied and cannot be released while the socket is waiting for the next request or release. Therefore, the number of sockets that Tomcat can process at the same time cannot exceed the maximum number of threads, which greatly limits performance. However, with NIO, the process of “reading the socket and handing it to the Worker thread” is non-blocking. When the socket is waiting for the next request or release, it does not occupy the Worker thread. Therefore, the number of sockets that Tomcat can process at the same time is much larger than the maximum number of threads, and the concurrency performance is greatly improved.

AcceptCount maxConnections maxThreads

To recap how Tomcat handles requests, it receives the connection in the Accept queue (when a client sends a request to the server, the OS places the connection in the Accept queue if the client completes a three-way handshake with the OS). Get the requested data in the connection and generate the request; Call the servlet container to process the request; Returns the response.

Corresponding parameters in Connector function as follows:

1, the acceptCount

Length of the Accept queue; When the number of connections in the Accept queue reaches the acceptCount, the queue is full and all incoming requests are rejected. The default value is 100.

2, maxConnections

The maximum number of connections that Tomcat can receive and process at any time. When Tomcat receives maxConnections, the Acceptor thread does not read the connections in the Accept queue. The threads in the Accept queue will block until Tomcat receives fewer connections than maxConnections. If the value is set to -1, the number of connections is unlimited.

The defaults are dependent on the protocol used by the connector: NIO defaults to 10000, APR/native defaults to 8192, and BIO defaults to maxThreads (Executor’s maxThreads if Executor is configured).

3, the maxThreads

Maximum number of request processing threads. The default is 200 (both Tomcat7 and 8 are correct). If the Connector is bound to executors, this value is ignored because the Connector will use the bound Executor instead of the built-in thread pool to perform tasks.

MaxThreads specifies the maximum number of threads, not the actual number of running cpus. In fact, maxThreads is much larger in size than the number of CPU cores. This is because the thread processing the request may spend very little time actually computing, and most of the time may be blocked, waiting for the database to return data, waiting for data to be read or written to the disk, etc. Thus, at any given moment, only a few threads are actually using the physical CPU, and most threads are waiting; So it makes sense that the number of threads is much greater than the number of physical cores.

In other words, Tomcat can keep the CPU busy by using many more threads than the number of CPU cores, greatly increasing CPU utilization.

4. Parameter Settings

(1) The setting of maxThreads depends on both the characteristics of the application and the number of CPU cores on the server. As you can see from the previous section, the number of maxThreads should be much larger than the number of CPU cores; The larger the number of CPU cores, the larger maxThreads should be; The less CPU-intensive the application is (the more IO intensive), the larger the maxThreads should be in order to make full use of the CPU. Of course, a larger value for maxThreads is not always better. If maxThreads are too large, the CPU will spend a lot of time switching threads and the overall efficiency will be reduced.

(2) The setting of maxConnections is related to the running mode of Tomcat. If Tomcat is using BIO, maxConnections should have the same value as maxThreads; If Tomcat uses NIO, the maxConnections value should be much larger than the maxThreads value, similar to the default value for Tomcat.

(3) As you can see from the previous introduction, although Tomcat can process maxConnections at the same time, the server can receive maxConnections+acceptCount. The setting of acceptCount depends on how the application expects to react in case of high connections. If the setting is too large, incoming requests will take a long time. If the value is too small, connection refused will be returned immediately.

Thread pool Executor

The Executor element represents the thread pool in Tomcat and can be shared by other components; To use this thread pool, the component needs to specify it through the Executor property.

Executor is an embedded element of the Service element. Typically, Connector components use thread pools; To enable Connector to use thread pools, the Executor element should be placed in front of Connector. The following is an example for configuring Executor and Connector:

?

The main attributes of an Executor include:

• Name: flag for the thread pool
• MaxThreads: Maximum number of active threads in the thread pool. Default: 200 (Tomcat7 and 8 are both)
• MinSpareThreads: Minimum number of threads to hold in the thread pool. The minimum value is 25
• MaxIdleTime: The maximum amount of time that a thread is idle, and the thread is closed when idle exceeds this value (unless the number of threads is smaller than minSpareThreads). The unit is ms. The default value is 60000 (1 minute).
• Daemon: whether background threads exist. Default value: true
• ThreadPriority: indicates the priority of a thread. The default value is 5
• NamePrefix: specifies the prefix of the thread name. The thread name in the thread pool is namePrefix+ thread number

4. Check the current status

The concepts of Tomcat connection count and thread count and how to set them were introduced above. Here is how to view the connection count and thread count in the server.

There are two ways to check the server status :(1) using off-the-shelf tools, (2) directly using Linux commands to check the server status.

Ready-made tools, such as JDK’s JConsole tool, can easily check thread information (in addition to viewing CPU, memory, classes, JVM basic information, etc.), Tomcat’s own Manager, charging tool New Relic, etc. Here is the interface for jConsole to view thread information:

Here’s how to check the number of connections and threads in the server from the Linux command line.

1. Connection number

If Tomcat receives HTTP requests on port 8083, you can use the following statement to check the connection:

Netstat - NAT | grep, 8083Copy the code

The result is as follows:

As you can see, one connection is in the Listen state, listening for requests; In addition, there are four ESTABLISHED connections and two CLOSE_WAIT connections.

2, thread

Run the ps command to view the process status. For example, run the following command:

Ps - e | grep JavaCopy the code

The results are shown below:

As you can see, only one process is printed; 27989 is the thread ID. Java refers to the Java command executed. This is because when tomcat is started, all internal work is done in this process, including the main thread, garbage collection thread, Acceptor thread, request processing thread, and so on.

You can see how many threads are in the process by using the following command. Where, NLWP indicates number of light-weight process.

Ps - o NLWP 27989Copy the code

As you can see, there are 73 threads inside the process; However, 73 does not exclude threads in idle state. To get the actual number of running threads, do the following:

ps -eLo pid ,stat | grep 27989 | grep running | wc -lCopy the code

Ps-elo pid,stat can find all threads, and print the process number and the current status of the thread. The two grep commands filter the process number and thread state respectively. Wc Statistics number. Among them, the ps – eLo pid, stat | grep 27989 output of the results are as follows:

Only partial results are captured; Sl indicates that most threads are idle.

The original address: https://www.cnblogs.com/kismetv/p/7806063.html