When ThreadPoolExecutor is initialized, it takes the following parameters:

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler) {
Copy the code

ThreadPoolExecutor (ThreadPoolExecutor, ThreadPoolExecutor, ThreadPoolExecutor, ThreadPoolExecutor, ThreadPoolExecutor, ThreadPoolExecutor)

1. Four implementations

1.1 coreSize = = maxSize

Many of you have seen or written code like this:

ThreadPoolExecutor executor = new ThreadPoolExecutor(10.10.600000L, TimeUnit.DAYS,
                                                     new LinkedBlockingQueue());
Copy the code

This line of code mainly shows that coreSize and maxSize are equal when initializing ThreadPoolExecutor, so that as the number of requests increases, it will look like this:

  1. When the number of requests < coreSize, the new thread;
  2. If the number of requests >= coreSize && is insufficient, add the task to the queue.
  3. When the queue is full, the task is rejected because coreSize and maxSize are equal.

The main purpose of this writing is to make the thread increase to maxSize, and do not recycle the thread, to prevent thread recycling, to avoid increasing the loss of recycling. Generally, business traffic has peaks and valleys, in the trough of the flow, the thread will not be recycled; MaxSize threads can handle traffic peaks without the need to slowly initialize to maxSize.

There are two prerequisites for this setup:

  1. AllowCoreThreadTimeOut We take the default false and will not actively set it to true. AllowCoreThreadTimeOut is false so that the core thread will not be reclaimed when the thread is idle
  2. KeepAliveTime and TimeUnit are both large so that threads are idle for a long time and cannot be easily reclaimed

We now have sufficient machine resources, we do not have to worry about idle threads will waste machine resources, so this writing is very common at present.

1.2 maxSize unbounded + SynchronousQueue

When a thread pool selects a queue, we’ll also see SynchronousQueue, which has a stack and a queue. The default SynchronousQueue is a stack, and there is no container for storing elements. There is a one to one mapping between putting and holding elements. If there is no corresponding take, the PUT operation will block, and the put operation will return only after the wired program executes the take operation.

The advantages and disadvantages of the maxSize unbounded + SynchronousQueue combination are obvious:

Advantages: The blocking queue has no storage space. As soon as the request comes, an idle thread must be found to process the request. If not, a new thread must be created in the thread pool to execute the request. With any other queue, we only know that the task has been submitted, but we have no way of knowing whether the current task is being consumed or piling up in the queue.

Disadvantages:

  1. It consumes resources. When a lot of requests come in, we create a lot of new threads to handle them
  2. Because SynchronousQueue has no storage space, the maxSize+1 task is rejected if the number of threads in the pool has reached maxSize and there are no idle threads. So if the volume of requests is difficult to predict, the size of maxSize is also difficult to set

1.3 maxSize bounded + Queue unbounded

The combination of maxSize bounded + Queue unbounded can be used in some scenarios where traffic is not required to be real-time but fluctuates.

For example, if we set maxSize to 20 and Queue to the default constructor’s LinkedBlockingQueue, the pros and cons of doing this are as follows:

Advantages:

  1. Under the condition of fixed COMPUTER CPU, the number of threads that can work at the same time per second is limited. At this time, it is also a waste to open a lot of threads. It is better to put these requests into a queue to wait, which can reduce the CPU competition between threads.
  2. The LinkedBlockingQueue default constructor constructs a list with the maximum size of Integer, which is ideal for ebb and flow scenarios where a large number of requests are blocked in the queue during peak traffic, allowing a limited number of threads to consume slowly.

Disadvantages: During peak traffic, a large number of requests are blocked in the queue, and it is difficult to ensure the real-time performance of requests. Therefore, this combination cannot be used in scenarios with high requirements on real-time performance.

1.4 maxSize bounded + Queue bounded

This combination is a complement to the shortcoming of 3. We change the queue from unbounded to bounded, as long as the queued task can complete the task within the required time.

This combination requires a combination of thread and queue sizes to ensure that most requests can be returned within the required time.

2. Three questions

2.1 How do I Prevent Idle Threads from being Reclaimed?

In some cases, we do not want idle threads to be collected, so we set keepAliveTime to 0. This is actually wrong. When we set keepAliveTime to 0, the poll method will return null immediately after timeout blocking on the queue. That is, idle threads are immediately recycled.

So if we want idle threads not to be recycled, we can set keepAliveTime to infinity and TimeUnit to large units of time. For example, we can set keepAliveTime to 365 and TimeUnit to timeunit.days. This means that threads that are idle will not be collected for a year.

In practice, machines usually have enough memory. If you set maxSize properly, you don’t want threads to be reclaimed even if they are idle, so you can also set keepAliveTime to infinity.

2.2 When should a common thread pool be used?

In real development, we do not share a thread pool for all scenarios under a particular business, generally following the following principles:

  1. Query and write not common thread pool, Internet applications in general, the query volume is more than the amount of writing, if the query and write to go thread pool, we must not public thread pool, that is to say, the query query thread pool, to go write a thread pool, if public, when large amounts of query, write requests may be to the queue to queue, Can’t be dealt with in time;
  2. Multiple written business scenario depends whether need common thread pool, in principle, every business scenarios, use your own thread pool alone, never Shared, so in the aspect of business management, current limiting fuse is easy, once the multiple business scenarios public thread pool, may cause influence each other between the business scenario, now the machine memory is very big, It also makes sense to use its own thread pool for each write business scenario;
  3. Multiple query business scenarios can share thread pools, and query requests generally have several characteristics: There are many query scenarios, short RT time, and large amount of query. If each query scenario has a separate thread pool, the first one consumes resources, and the second one is difficult to define the size of threads and queues in the thread pool, which is complicated. Therefore, multiple similar query business scenarios can share the thread pool.

2.3 How to calculate the thread size and queue size?

When we use thread pools in real development, we need to carefully consider thread sizes (coreSize, maxSize) and task Queue sizes (Queue size) from several aspects:

  1. In terms of business, we need to consider the concurrency of all businesses when initializing the thread pool
    1. If all the current services have a lot of traffic at the same time, then when setting up the thread pool for the current business, we try to set the thread size and queue size as small as possible
    2. If almost all services do not have traffic at the same time, you can set it slightly larger
  2. According to the real-time requirements of business
    1. If the real time requirement is high, we set the queue size to be smaller, coreSize == maxSize, and maxSize to be larger
    2. If real time requirements are low, you can make the queue larger

Suppose that only one kind of service is running on the machine at a certain period of time, and the real-time requirements of the service are high. The average RT (ResponseTime) of each request is 200ms, the request timeout time is 2000ms, the machine has a 4-core CPU, 16G memory, and the QPS of a machine is 100. At this point we can simulate how to set up:

  1. 4 core CPU, assuming the CPU can run full, the RT of each request is 200ms, that is, 200ms can execute 4 requests, 2000ms can execute 200/200 * 4 = 40 requests;
  2. In fact, the performance of a 4-core CPU is much higher than that. We can add 10 requests to our head, which means we can expect to execute 50 requests in 2000ms.
  3. The QPS of a machine is 100, at this point we calculate that a machine can handle at most 50 requests in 2 seconds, so at this point we need to add at least one machine without RT optimization.

Thread pools can be set up like this:

ThreadPoolExecutor executor = new ThreadPoolExecutor(15.15.365L, TimeUnit.DAYS,
                                                     new LinkedBlockingQueue(35));
Copy the code

The maximum number of threads is 15, and the maximum number of queues is 35, so that the machine can handle the maximum 50 requests in 2000ms. Of course, according to the performance and real-time requirements of your machine, you can adjust the ratio of the number of threads and queue size, as long as the total is less than 50.

The above is only a very rough setting, in the actual work, but also need to constantly observe and adjust according to the actual situation.

3.Executors

Excutors have wrapped four implementations of thread pools, which are set to different parameters when new ThreadPoolExeCutor() is used

3.1 FixedThreadPool: a fixed size thread pool

new ThreadPoolExeCutor(nThreads,nThreads,0L,TimeUnit.SECONDS,new LinkedBlockingQueue<Runnable>())
Copy the code
  • Features: All threads are core threads. When threads are idle, they will not be recycled and can quickly respond to external requests
  • Purpose: It can be used to control the number of threads under known concurrency pressure

3.2 CachedThreadPool: cache thread pool

new ThreadPoolExeCutor(0,Integer.MAX_VALUE,60L,TimeUnit.SECONDS,new SynchronousQueue<Runnable>());
Copy the code
  • Features: indeterminate number of threads, only non-core threads, timeout recycle, create a new thread for each task
  • Purpose: can be infinitely expanded, suitable for performing a large number of less time-consuming tasks

3.3 ScheduleThreadPool: indicates a scheduled thread pool

new ThreadPoolExecutor(corePoolSize,Integer.MAX_VALUE, ~,~, new DelayedQueue<Runnable>())
Copy the code
  • Features: fixed core number, non-core is not fixed, can carry out periodic tasks and scheduled tasks
  • Purpose: Can be used for delay start, timing start, suitable for multiple background threads to perform periodic tasks

3.4 SingleThreadPool: single-threaded thread pool

new ThreadPoolExecutor(1.1.0L,TimeUnit.SECONDS,new LinkedBlockingQueue<Runnable>())
Copy the code
  • Features: Ensures that all tasks are executed sequentially in the same thread without dealing with synchronization issues
  • Purpose: Can be used in scenarios where execution order needs to be guaranteed and only one thread is executing