“This is the 17th day of my participation in the First Challenge 2022.

1. The background

In exploring the implementation of long polling, there are asynchronous requests using Servlet3. Let’s take a look at Servlet3’s asynchronous requests

The Servlet is now version 5

2. Servlet synchronization request

Take the Tomcat server as an example.

  • The Http request arrives at Tomcat
  • Tomcat pulls threads from the thread pool to process requests arriving at Tomcat
  • Parse the request Http to HttpServletRequest
  • Distribute to specific servlets to handle corresponding services
  • Return processed data via HttpServletResponse

Normally the request model is the same as the above model, all requests are sent to the Tomcat server’s thread pool for processing, and the entire action is released back to the thread pool after processing.

There is a problem if the late business process takes a long time. The thread that handles the request is always occupied. As more and more requests are made, more and more threads are occupied. Until all threads in the thread pool are exhausted. Subsequent entries are blocked waiting for the thread to process them.

When the user does not care about the return of the submission, the business processing thread pool can be defined. After the front-end request is submitted, the Tomcat thread will submit the processing to the business thread pool for immediate return. Asynchronous tasks (@async) in Spring do just that.

3. Servlet asynchronous request

Take the Tomcat service as an example:

  • Parse the request Http to HttpServletRequest
  • Distribute to a specific Servlet for processing, submit the business to a custom business thread pool, and the Tomcat thread is immediately released.
  • When the business thread finishes executing the task, the result is forwarded to the Tomcat thread pool.
  • Return processed data via HttpServletResponse

The overall process of introducing asynchronous Servlet3:

With asynchronous servlets, the Tomcat thread only handles request resolution, and all long business operations are handed over to the business thread pool, so the Tomcat thread can handle more requests than synchronous requests. The business is handed over to the business process, but the front end is still waiting for the results to return (synchronous waiting to return).

Asynchronously, the front end waits for the result to return synchronously. Many people feel that an asynchronous request will return faster. This is not true because asynchrony involves thread switching. All return times will be slower than synchronous.

While not reducing the corresponding time, there are other obvious advantages:

  • It can handle a higher number of concurrent connections and improve overall system throughput
  • Request resolution is completely separated from business processing and has a single responsibility
  • Customizing the business thread pool makes it easier to monitor, degrade, and so on
  • You can customize different thread pools based on different services, which are isolated from each other and do not affect each other

4. Use of asynchronous servlets

Using asynchronous servlets requires only three steps:

  1. HttpServletRequest#startAsync()To obtainAsyncContextAsynchronous context
  2. Use custom business thread pools for business processing
  3. AsyncContext#getResponse()Returns the processing result to the front end and then callsAsyncContext#complete()

5. Implementation examples in Spring

The code is shown below:

  1. Enabling asynchronous Servlets
  2. Simulating business execution
  3. Returns the result to the front end

We said that the front end is always waiting synchronously and let’s verify that by running the code. The results are shown below:

Code address: github.com/mxsm/spring…