1. Introduction to thread pools

1.1 What is a Thread Pool?

Thread pooling is a tool for managing threads based on pooling techniques and is often found in multi-threaded servers

Having too many threads creates additional overhead, including the cost of creating and destroying threads, scheduling threads, and so on, and reduces the overall performance of the computer. A thread pool maintains multiple threads, waiting for the supervisor to assign tasks that can be executed concurrently. This approach, on the one hand, avoids the cost of creating and destroying threads when processing tasks, on the other hand, avoids excessive scheduling problems caused by the expansion of the number of threads, and ensures the full utilization of the kernel.

Using thread pools has the following benefits:

Reduce resource consumption: Reuse existing threads through pooling technology to reduce the loss of thread creation and destruction.

Speed up: When a task arrives, it can be executed without waiting for the created thread.

Improve the manageability of threads: Threads are limited resources (JVM startup can specify ThreadStackSize), such as unrestricted creation, not only will consume system resources, but also will cause the system resources are not reasonable allocation, affecting application performance. Thread pools allow for uniform allocation, tuning, and monitoring;

Provide more powerful extensions: The thread pool is extensible, allowing developers to add more functionality to it. Such as delay timer thread pool ScheduledThreadPoolExecutor, allows a stay of execution or regular task execution.

1.2 What problems do thread pools solve?

The core problem solved by thread pools is resource management. In a concurrent environment, the system cannot determine how many tasks need to be executed and how many resources need to be invested at any given time. This uncertainty raises a number of questions:

A. Frequent application/destruction of resources and scheduling of resources will cause additional consumption;

B. Lack of control over applications for unlimited resources, which may deplete resources;

C. The system cannot properly control resources, which reduces system stability.

1.3 Core design and implementation of thread pool

1.3.1 Overall design

The core thread pool in Java is a ThreadPoolExecutor, inherited from right to left.

Executor <- ExecutorService <- AbstractExecutorService <- ThreadPoolExecutor

Executor is designed to decouple task submission from task execution. Instead of focusing on how to create threads, users only need to provide a Runnable object to submit the running logic of a task to an Executor. The Executor completes resource allocation and task execution.

ExecutorService extends some of the capabilities: the ability to execute tasks, the ability to add a future generation method for one or a batch of asynchronous tasks, the ability to control threads, such as Submit/Invoke /shutdown;

AbstractExecutorService: the top of the abstract class, will perform the task of process series up, ensure the realization of the lower can focus on a mission method;

ThreadPoolExecutor: It maintains its own lifecycle while simultaneously managing threads and tasks, making a good combination of the two to execute parallel tasks. The operating mechanism is shown in the following figure

The thread pool internally builds a producer-consumer model that decouples threads and tasks to create a well-buffered task and reuse threads. The operation of thread pool is divided into two parts: task management and thread management. The task management part acts as the producer, and when the task is submitted, the thread pool determines the subsequent flow of the task:

(1) Directly request the thread to execute the task;

(2) Buffer to the queue waiting for the thread to execute;

(3) Reject the task. The thread management part is the consumer, which is maintained uniformly in the thread pool, and the thread is allocated according to the task request. When the thread finishes executing the task, it will continue to obtain new tasks to execute. Finally, when the thread fails to obtain the task, the thread will be reclaimed.

1.3.2 Thread pool Lifecycle

The running state of the thread pool is not explicitly set by the user, but is maintained internally as the thread pool runs. A variable is used internally to maintain two values: the running state (runState) and the number of threads (workerCount). In its implementation, thread pools combine maintenance of two key parameters, runState and workerCount, as shown in the following code:

CTRL is an AtomicInteger field that controls the running status of the pool and the number of valid threads in the pool. The runState of the thread pool and the number of valid threads in the pool (workerCount) are stored. The higher 3 bits hold the runState and the lower 29 bits hold the workerCount. Using one variable to store two values can avoid inconsistency when making decisions, and it is not necessary to occupy lock resources to maintain consistency between the two. As you can see from reading the thread pool source code, it is often the case that you have to determine both the thread pool’s health and the number of threads. The thread pool also provides several methods for users to obtain the current status of the thread pool and the number of threads. It’s all bit operations, and it’s much faster than the basic operations.

With respect to the internal encapsulation of the get lifecycle state, the number of get thread pool threads is calculated as follows:

There are currently five life cycle states for thread pools, as shown in the figure below.

1.3.3 Task Execution Mechanism

Task scheduling

First, all tasks are scheduled by the execute method. This part checks the status of the current thread pool, the number of running threads, and the running policy, and determines whether the next process to execute is to directly request the thread to execute, buffer it to the queue, or reject the task directly. The execution process is as follows:

A. Check the RUNNING status of the thread pool. If the thread pool is not RUNNING, the thread pool is rejected. If workerCount < corePoolSize, a thread is created and started to execute the newly submitted task.

B. If workerCount >= corePoolSize and the blocking queue in the thread pool is not full, add the task to the blocking queue.

C. If workerCount >= corePoolSize && workerCount < maximumPoolSize and the blocking queue in the thread pool is full, create and start a thread to execute the newly submitted task.

D. If workerCount >= maximumPoolSize and the blocking queue in the thread pool is full, the task is processed according to the reject policy. By default, an exception is thrown directly.

Task buffer

Task buffer module is the core of the thread pool can management tasks, the thread pool is the management of tasks and threads, the key is to decoupling, threads and tasks not directly related, the thread pool by consumers, producers model by blocking buffer queue cache task, the worker thread from blocking queue for tasks to perform.

BlockingQueue is a double buffered queue. BlockingQueue uses two queues internally, allowing two threads to store and fetch to the queue at the same time. At the same time, the efficiency of queue access is improved. BlockingQueue:

  • ArrayBlockingQueue: BlockingQueue of specified size whose construction must specify size. The objects it contains are sorted in FIFO order.

  • LinkedBlockingQueue: An unfixed size BlockingQueue. If the size is specified when it is constructed, the generated BlockingQueue has a size limit. If the size is not specified, the size is determined by integer.max_value. The objects it contains are sorted in FIFO order.

  • PriorityBlockingQueue: Similar to LinkedBlockingQueue, but instead of a FIFO, the objects in it are sorted according to their natural order or the constructor’s Comparator.

  • SynchronizedQueue: A special BlockingQueue that must be operated alternately by putting and fetching.

Task refuse

The task rejection module is the protection part of the thread pool. The thread pool has a maximum capacity. When the task cache queue of the thread pool is full and the number of threads in the thread pool reaches maximumPoolSize, the task needs to be rejected and the task rejection policy is adopted to protect the thread pool. A denial policy is an interface that can be implemented to a custom denial policy executor

2. Thread pool practices

2.1 Service Background

In today’s Internet industry, in order to maximize the multi-core performance of the CPU, parallel computing capability is indispensable. Managing thread acquisition concurrency through a thread pool is a very basic operation, so let’s look at two typical scenarios where thread pool acquisition concurrency is used.

Scenario 1: Quickly process user requirements

Description: In our course schedule business, when the user requests the course schedule information, a large number of data needs to be captured and assembled and returned to the user

Analysis: in this scenario, we hope is the faster response to user needs, the better, but the schedule information is very complex, accompanied by calls and calls between the cascade, slopes couplet, etc., more development of children’s shoes often use the simple way, the thread pool will be encapsulated into the task parallel execution, shorten the overall response time. The most important thing in this scenario is to get the maximum response speed to satisfy the user, so do not set queues to buffer concurrent tasks, and raise the corePoolSize and maxPoolSize to create as many threads as possible to execute the task quickly.

Scenario 2:

Description: A large number of offline computing tasks that need to be performed quickly. In our clock-in business, we need to generate statistical data of all dimensions of the school, such as grade, class, teacher, parents, etc., such as the number of clock-in/non-clock-in class xx.

Batch task

Analysis: In this case, should also use multithreading strategy, parallel computing. However, the difference between this scenario and the response speed first scenario is that this scenario has a large amount of tasks and does not require instantaneous completion. Instead, it focuses on how to use limited resources to process as many tasks as possible in a unit of time, which is the throughput first problem. Therefore, you should set the queue to buffer concurrent tasks and adjust the appropriate corePoolSize to set the number of threads to process tasks. In this case, setting too many threads can also lead to frequent thread context switches, which can slow down the processing of tasks and reduce throughput.

2.2 Practical problems and thinking schemes

2.2.1 Thinking of practical problems and schemes

The core problem with thread pool usage is that the parameters of the thread pool are not well configured. On the one hand, the mechanism of thread pool operation is not well understood, and proper configuration depends heavily on the personal experience and knowledge of developers. On the other hand, thread pool execution is highly dependent on task type, and IO – and CPU-intensive tasks run very differently, leading to the fact that there is no well-established empirical strategy for developers to refer to.

Despite careful evaluation, there is no guarantee that the proper parameters will be calculated at one time, so can we reduce the cost of modifying the thread pool parameters so that they can at least be adjusted quickly in case of failure to shorten the recovery time? Based on this thinking, can we migrate the parameters of the thread pool from the code to the distributed configuration center, so that the parameters of the thread pool can be dynamically configured and take effect immediately? The comparison of the parameters modification process before and after the dynamic configuration of the thread pool parameters is as follows:

Ladies and gentlemen, above is all the content of this article, can see here, are great god 👍. Each chapter content can not guarantee the complete original, but also after careful consideration and practice of thought crystallization, technical articles mutual reference is often, if there is a copyright problem, first to apologize to the author, has not been used for commercial purposes, if there will be active contact, find problems please contact immediately leave a message.

Thank you very much for reading here, if this article is good, feel something

Please like 👍 follow ❤️ share 👥