Introduction to Thread Pools

Thread Pool is a tool to manage threads based on pooling idea, which often appears in multi-threaded server, such as MySQL.

Too many lines will bring extra costs, including the cost of creating and destroying threads, the cost of scheduling threads, etc., and also reduce the overall performance of the computer. A thread pool maintains multiple threads waiting for a supervisor to assign tasks that can be executed concurrently. This approach, on the one hand, avoids the cost of creating and destroying threads while processing tasks, on the other hand, avoids the excessive scheduling problem caused by the expansion of the number of threads, and ensures the full utilization of the kernel.

The thread pool described in this article is the ThreadPoolExecutor class provided in the JDK.

Of course, there are a number of benefits to using thread pools:

  • Reduced resource consumption: Reuse of created threads through pooling techniques to reduce wastage from thread creation and destruction.
  • Improved response time: Tasks can be executed immediately when they arrive without waiting for threads to be created.
  • Improve manageability of threads: Threads are scarce resources. If they are created without limit, they will not only consume system resources, but also cause resource scheduling imbalance due to unreasonable distribution of threads, which reduces system stability. Thread pools allow for uniform allocation, tuning, and monitoring.
  • More and more power: Thread pools are extensible, allowing developers to add more functionality to them. Such as delay timer thread pool ScheduledThreadPoolExecutor, allows a stay of execution or regular task execution.

The core problem solved by thread pools is resource management. In a concurrent environment, the system cannot determine how many tasks need to be executed or how many resources need to be invested at any given time. This uncertainty raises several questions:

  1. The additional cost of applying/destroying resources and scheduling resources frequently can be significant.
  2. The lack of means to suppress unlimited resource applications may lead to the risk of system resource exhaustion.
  3. The system cannot properly manage internal resource distribution, which reduces system stability.

To solve the problem of resource allocation, thread Pooling adopts the idea of Pooling. Pooling, as the name suggests, is the idea of managing resources together in order to maximize returns and minimize risks.

Use of thread pools

public class Test {
    static class Task implements Runnable{
        private String name;

        public Task(String name) {
            this.name = name;
        }

        @Override
        public void run(a) {
            System.out.println("start task: " + name);
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("end task: "+ name); }}public static void main(String[] args) {
        ExecutorService es = Executors.newFixedThreadPool(2);
        for (int i = 0; i < 6; i++){
            es.execute(new Task("task"+ i)); } es.shutdown(); }}/* Execution result Start task: task0 start task: task1 end task: task0 start task: task2 end task: task1 start task: task3 end task: task2 start task: task4 end task: task3 start task: task5 end task: task4 end task: task5 */
Copy the code

The core parameters of the thread pool

  • Number of core threads: The core threads are always alive. When the task arrival core thread will be created and initialized, of course also can call prestartCoreThread/prestartAllCoreThreads method to let the thread pool to create core thread from the start.

  • Maximum number of threads that a thread pool can hold: When the number of active threads reaches this value, subsequent tasks will block.

  • Idle timeout for non-core threads: If this timeout is exceeded, non-core threads will be reclaimed. Combine with the time units below to get the specific duration.

  • Time unit: millisecond, second, minute.

  • Task queue: Runnable objects submitted through the execute() method of the thread pool, which will be stored in this parameter.

  • Thread factory: Create a new thread for the thread pool.

  • Rejection policy: a policy on how to process new tasks if the thread pool cannot accommodate them.

private volatile int corePoolSize;								// Number of core threads
private int largestPoolSize;											// Maximum number of threads
private volatile long keepAliveTime; 							// The timeout duration of non-core threads
private final BlockingQueue<Runnable> workQueue;	// Task queue
private volatile ThreadFactory threadFactory;			// Thread factory
private volatile RejectedExecutionHandler handler;// Reject the policy

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler) {
    if (corePoolSize < 0 ||
        maximumPoolSize <= 0 ||
        maximumPoolSize < corePoolSize ||
        keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}
Copy the code

Several common thread pools

  • Fixed length thread pool: only core threads & will not be collected, the number of threads is fixed, and the task queue has no size limit (the exceeded thread task will wait in the queue); Application scenario: Controls the maximum number of concurrent threads.

  • Timed thread pool: fixed number of core threads, unlimited number of non-core threads (immediately recycled when idle); Application scenario: Perform scheduled or periodic tasks.

  • Cacheable thread pool: only non-core threads, the number of threads is not fixed (can be infinite), flexible recovery of idle threads (with timeout mechanism, almost no system resources when all recovered), new threads (when no threads available), any thread tasks will be executed immediately, without waiting; Application scenario: Execute a large number of thread tasks with low time consumption.

  • Single threaded thread pool: Only one core thread (ensures that all tasks are executed in a single thread in the specified order without dealing with thread synchronization). Application scenario: Operations that are not suitable for concurrent operations but may cause I/O block and affect UI thread response, such as database operations and file operations.

Task handling

When a new task is submitted to the thread pool, a series of judgments are made based on the state of the thread pool to determine how to execute the task. All tasks are scheduled by the Execute method, which checks the running status of the current thread pool, the number of running threads, and the running policy, and determines the next process to be executed. Whether to directly apply for thread execution, buffer the task to the queue, or reject the task directly. Its execution process is as follows:

  1. First, check the running status of the thread pool. If it is not running, reject it directly. The thread pool must ensure that the task is executed in the running state.
  2. When the number of worker threads is less than the number of core threads, a thread is created and started to execute the newly submitted task.
  3. When the number of worker threads is greater than or equal to the number of core threads, and the blocking queue in the thread pool is not full, the task is added to the blocking queue.
  4. When the number of core threads is less than or equal to the number of worker threads < the maximum number of threads, and the blocking queue in the thread pool is full, a thread is created and started to execute the newly submitted task.
  5. If the number of worker threads is greater than or equal to the maximum and the blocking queue in the thread pool is full, the task is processed according to the reject policy. The default processing method is to throw exceptions directly.

Task buffering

The task buffer module is the core part of the thread pool that can manage tasks. The essence of thread pool is the management of tasks and threads, and the key idea to achieve this is to decouple the tasks and threads from the direct correlation, so that the subsequent allocation work can be done. Thread pools are implemented in producer-consumer mode through a blocking queue. The blocking queue caches tasks from which the worker thread retrieves them.

A BlockingQueue is a queue that supports two additional operations. The two additional operations are: when the queue is empty, the thread that fetched the element waits for the queue to become non-empty. When the queue is full, the thread that stores the element waits for the queue to become available. Blocking queues are often used in producer and consumer scenarios, where the producer is the thread that adds elements to the queue and the consumer is the thread that takes elements from the queue. A blocking queue is a container in which producers hold elements, and consumers only take elements from the container.

The following figure shows thread 1 adding elements to the blocking queue and thread 2 removing elements from the blocking queue:

Different queues can implement different task access strategies. Here, we can revisit the blocking queue members:

Rejection of a task

The task rejection module is the protected part of the thread pool. The thread pool has a maximum capacity. When the task cache queue of the thread pool is full and the number of threads in the thread pool reaches maximumPoolSize, the task must be rejected and the task rejection policy is adopted to protect the thread pool.

A rejection policy is an interface designed as follows:

public interface RejectedExecutionHandler {
    void rejectedExecution(Runnable r, ThreadPoolExecutor executor);
}
Copy the code

Users can implement this interface to customize rejection policies or choose from the four existing rejection policies provided by the JDK, which have the following features:

References:

  1. Android multithreading: ThreadPool full parsing
  2. Understanding Java thread pools in depth: ThreadPoolExecutor
  3. Implementation principle of Java Thread Pool and its practice in Meituan business – Meituan Technical team