This is the 16th day of my participation in the August More Text Challenge
The BIO and NIO modes and workflow of Connector components of Tomcat are introduced in detail.
In this article, we briefly introduced several modes of Connector components. Now we will introduce the BIO and NIO modes and workflow of Connector components in detail.
1 the BIO connector
The BIO connector implementation architecture is as follows:
In the BIO connector implementation, mainly adopts org. Apache. Coyote. Http11. Http11Protocol processing request, The Http11Protocol contains the JIoEndpoint object and the Http11ConnectionHandler object.
JIoEndpoint maintains two thread pools, Acceptor and Worker. An Acceptor accepts a socket connection and processes the socket from the Worker thread pool. If there are no free threads in the Worker thread pool, the Acceptor will block. Once the Worker thread gets the socket, it fetches the Http11Processor object from the Http11Processor object pool for further processing. The Worker thread pool is the default thread pool, and we’ve already shown that you can configure your own Executor tag and refer to it as a Worker thread pool!
- The Http11ConnectionHandler object maintains a pool of Http11Processor objects, Http11Processor object parses the socket of the HTTP request and encapsulated into the request object (this request is for the temporary use of a class, it is the full name of the class of org. Apache. Coyote. Request), And create the org. Apache. Coyote. Request.
- Then call org. Apache. Catalina. Connector. CoyoteAdapter continue to complete parsing, construct org. Apache. Catalina. The Request and Response objects, GetService ().getContainer().getPipeline().getFirst().invoke(request, response) forwards the request to the corresponding Container(Engine, The Servlet container).
- Then there is the Engine->Host->Context->Wrapper process, where the Engine finds the corresponding Web application and calls the corresponding Servlet method, and sends the response back to the client through the socket. Note that these actions are also done in the Worker thread.
2 the NIO connector
NIO’s Connector implementation has the following architecture, which is more complex than the BIO implementation:
In the NIO connector implementation, mainly adopts org. Apache. Coyote. Http11. Http11NioProtocol processing request, The Http11NioProtocol contains the NioEndpoint object and the Http11ConnectionHandler object.
NioEndpoint mainly includes Acceptor, Poller, and Worker components. As shown in the figure, the main flow of NioEndpoint is as follows:
The Acceptor thread in NioEndpoint is used to Accept socket connections from the Accept queue. The Acceptor thread is used to Accept sockets in the traditional serversocket.accept () mode, which is blocked. This is consistent with BIO. Will get a SocketChannel object, and then encapsulated in a tomcat implementation class org.apache.tomcat.util.net.NioChannel object. You then wrap the NioChannel object in a PollerEvent object and push the PollerEvent object into the Events Queue.
This is a typical producer-consumer pattern. Acceptors communicate with Poller threads through the Events Queue. Acceptors are producers of the Events queue and Pollers are consumers of the Events queue.
Poller threads are the main thread of a NIO implementation and can have multiple Poller threads. The Poller thread maintains a Selector object on which Tomcat’s “NIO” logic is based. Poller fetles the PollerEvent object from the Events Queue, registers the channel in this object with the primary Selector as an OP_READ event, and performs the select operation. Iterate over the sockets that can read data (triggering the OP_READ event), retrieve the available Worker threads from the Worker thread pool, and pass the socket to the Worker. The whole process is a typical NIO implementation.
After the Worker obtains the socket from the Poller, it encapsulates the socket in the SocketProcessor object. Then fetch the Http11NioProcessor object from the Http11ConnectionHandler, call the logic of the CoyoteAdapter from the Http11NioProcessor, Subsequent implementations are as blocking as the BIO implementation and are done by Worker threads.
In the architecture, a NioSelectPool object is maintained in the NioSelectorPool, which in turn maintains a BlockPoller thread. This thread performs NIO logic based on the secondary Selector, mainly processing blocking read and write operations. Such as reading data from the socket request body and writing data to the socket through response.
When the data (request body) becomes unreadable (client data has not been sent), the encapsulated OPEN_READ event is registered in the BlockPoller thread, which then blocks the current thread (usually the Tomcat IO thread). If the OPEN_READ event is triggered, The blocked thread will wake up. The write of response data (response headers and response bodies) is performed by the Tomcat IO thread. When the data is not writable (the original socket send buffer is full), the enclosed OPEN_WRITE event object is registered in the BlockPoller thread, and the current thread is blocked. If the OPEN_WRITE event is fired, the blocked thread will be woken up. The current thread here is generally the Worker thread. BlockPoller is mainly used to reduce Poller stress!
Tomcat’s NIO connector is not completely non-blocking. Some acceptors, such as acceptors, accept sockets and read/write data from sockets in blocking mode. The socket received by Acceptor is sent to the Worker thread due to the multiplexing model and is non-blocking. It is sent to the Worker thread only when a new request is received and released immediately after the request is processed. Every time an Acceptor receives a socket connection, it must immediately request a Worker thread to process it. Otherwise, the Acceptor thread will block.
3 summary
For BIO connector, i.e. synchronous blocking, the Tomcat server implements the mode of connecting to one Worker thread, that is, when the client has a connection, the server needs to start a thread for processing. If the connection does not do anything (for example, the subsequent request does not come, Or the connection has not been released after the request is completed) causes unnecessary thread overhead and is very resource-intensive. BIO connector support has been removed in the latest tomcat8.5 and later versions.
For connector of NIO, that is, synchronous non-blocking, the server implementation mode is one request one thread, that is, all connection requests sent by the client will be registered with the Selector of the multiplexer, and the multiplexer will start a Worker thread to process only when the multiplexer polls the connection to have I/O request events. The Worker thread is released immediately after the request is processed.
Why can NIO handle 10000 socket connections? Most HTTP requests currently use long connections (HTTP/1.1 default keep-alive is true). Long connections mean that a TCP socket does not release any new requests immediately after the current request ends. Instead, the socket waits for a timeout before releasing them. If the BIO is used, the process of “reading the socket and handing it to the Worker thread” is blocked, which means that the Worker thread processing the socket is occupied and cannot be released while the socket is waiting for the next request or release. Therefore, the number of sockets that Tomcat can handle at the same time can actually not exceed the maximum number of threads, which greatly limits performance. With NIO, the process of “reading the socket and handing it to the Worker thread” is non-blocking and does not occupy the Worker thread while the socket is waiting for the request to arrive or release. Therefore, the Secoter-based I/O multiplexing mechanism of Tomcat can make the number of sockets processed at the same time much larger than the maximum number of threads, and the concurrency performance is greatly improved.
Tomcat’s official description and comparison of Connector are as follows:
Here’s BIO versus NIO versus APR (tomcat.apache.org/tomcat-7.0-…
Here’s NIO versus NIO2 versus APR (tomcat.apache.org/tomcat-8.5-…
As you can see, BIO and NIO block when they actually Read and respond to data (Read Request Body and Write Response Headers and Body), while NIO uses the multiplexing model, NIO is also non-blocking when it reads Request Headers while waiting for the next Request. Why?
The BIO Read Request Headers is blocked because it does not have Poller for I/O multiplexing. Instead, it directly fetkes the Request Headers from the Worker thread of the thread pool. The BIO Read Request Headers is blocked because the Request data is still being sent from the network adapter to the OS kernel buffer. It didn’t really come.
NIO’s Read Request Headers are non-blocking because the socketchannel.read () method is used to Read the Request Headers and the Request row, and the poll thread continues to monitor the next data arrival. NIO blocks the reading of the request body. If the request body data is found to be unreadable, the encapsulated OP_READ event is first registered in the event queue of the BlockPoller object instance, and then the Worker thread is blocked.
References:
- Apache Tomcat 8 Configuration Reference
- In-depth understanding -Tomcat-(II)- Macro understanding -Tomcat- Components and architecture
- An in-depth look at the NIO model in Tomcat
- Explain the connection count and thread pool of Tomcat
If you need to communicate, or the article is wrong, please leave a message directly. In addition, I hope to like, collect, pay attention to, I will continue to update a variety of Java learning blog!