This is the fifth article in the concurrent programming series. When it comes to concurrent programming, how little thread pool is there? There are many scenarios in ali thread pool, and good use of thread pool is also a daily development must master.

Interviewer: I see that you have optimized the system performance in your resume. Can you give me a brief introduction? What are the optimizations, and how do you measure their effectiveness?

Me: Blah blah blah… For example, our system used to query users’ personal identity information, contact information, order status information and integral information. Previously, the system was single-thread serial processing, and I used the thread pool to process the four tasks in parallel, and then combined the processing results.

Interviewer: You said you used thread pools. Can you tell me why? Can I create four threads for processing?

Me: Yes, of course.

Me: But thread pools are more appropriate. There is a clause in alibaba’s development regulations:

3. [Mandatory] Thread resources must be provided through the thread pool. Explicit creation of threads in the application is not allowed. Note: The benefit of using a thread pool is to reduce the time spent creating and destroying threads and the overhead of system resources, solving the problem of insufficient resources. If you don’t use thread pools, you can run out of memory or “overswitch” by creating a large number of similar threads.

Me: It’s like when you go to a restaurant and the waiter always washes the dishes in advance. They don’t wash the dishes until you come to eat. The dishes are like threads in the thread pool.

Interviewer: Do you know the class structure of a thread pool?

Me: What kind of question is that? How to set the number of core threads? All right… See the picture below:

  • The Executor definition is very simple. It defines the essence of what a thread pool does, which is execute tasks.
public interface Executor {
  
    void execute(Runnable command);
}
Copy the code

  • ExecutorService is also an interface, but it kind of lays out the thread pool framework, telling the thread pool that implements it the methods that it must provide to manage the thread pool.

  • Abstract AbstractExecutorService is a common thread pool executor and ScheduledExecutorService is a scheduled task thread pool.

Interviewer: So how do you create thread pools on a daily basis?

Me: I use ThreadPoolExecutor to create custom thread pools.

Interviewer: Do you know what parameters are used to create a thread pool?

Me: There are seven main core parameters to a thread pool, as you can see from the ThreadPoolExecutor constructor

  1. CorePoolSize: number of core threads

  2. MaximumPoolSize: indicates the maximum number of threads

  3. KeepAliveTime: The amount of time a thread has to remain in the thread pool without being destroyed. If the thread pool has too many threads and the task is small, this time the thread pool is destroyed.

    Unit: keepAliveTime Time unit. The value can be seconds or milliseconds.

  4. WorkQueue: indicates a task queue that stores tasks to be executed

  5. ThreadFactory: A task factory that creates threads, such as prefixing thread names, as described below

  6. Handler: A rejection task handler that rejects tasks when they fail to be processed

  7. AllowCoreThreadTimeOut: whether to allowCoreThreadTimeOut destruction. This parameter is not in the constructor, but is also important

Interviewer: Honestly, you probably memorized it before you came, or you wouldn’t have remembered it all.

I: [lift table], not face, return to seek what work, want what bicycle.

I just read every article on Angela’s blog before I got here.

Interviewer: : In fact, that was also a question, to check whether the interviewer is honest, let’s move on.

Interviewer: With those core parameters in mind, can you give me a basic idea of how thread pools work?

Me: Yes, here I draw a process for you, as follows:

Interviewer: Then follow the procedure above to write some pseudo code.

I: still can not good face, let the hand rip thread pool.

Well, if you look at the flowchart at 👆🏻, the code is as follows:

Interviewer: Good, how do you normally manage thread pools?

Me: I would create a ThreadPoolManager, such as ThreadPoolManager, with a Map of private variables, and give it a name for the purpose of the thread pool, such as: PreparePlateThreadPool, define the thread pool name as a constant, and place the created thread pool into the manager’s Map.

Interviewer: Is there any other way to create your own thread pool with ThreadPoolExecutor?

The Executors provided in the Java.util. concurrent package can also be used to create thread pools.

Interviewer: What kinds of Executors are defined?

I:

  • NewSingleThreadExecutos single threaded thread pool, which is a thread pool that only has one task, which I use occasionally
  • NewFixedThreadPool (int nThreads) Specifies the thread pool for fixed-size threads
  • NewCachedThreadPool () is an unbounded thread pool, where threads are created to run no matter how many tasks are performed, so queues are virtually useless.

* If you use ThreadPoolExecutor to create a thread pool, do not use Executors. * If you use ThreadPoolExecutor, do not use Executors.

Me: First, the Thread pool provided by Executors provides limited application scenarios, so it’s hard to use the common scenarios. Second, they also use ThreadPoolExecutor to create thread pools. I directly use ThreadPoolExecutor to create thread pools, so I can understand the mechanism and be more flexible.

Refer to the specification of Ali Development Manual:

4. Force: Do not use Executors to create a thread pool. Use ThreadPoolExecutor to clear the running rules of the thread pool and avoid resource depletion. 1) FixedThreadPool and SingleThreadPool: The allowed request queue length is Integer.MAX_VALUE, which may accumulate a large number of requests and result in OOM. 2) CachedThreadPool: the number of threads allowed to create is integer. MAX_VALUE, which may create a large number of threads, resulting in OOM.

Alibaba R&D Manual

Interviewer: You have a queue for tasks in your code. What queue do you use to customize your thread pool?

I: this wants to see practical application.

  1. Some tasks are at peak time at 8am and 6pm, so there are task spikes, which are usedLinkedBlockingQueue, this is an unbounded queue that does not limit the size of the task.
  2. For less important, less dependent tasksArrayBlockingQueueIf the task is exceeded, a non-core thread will be created to execute the task.

Interviewer: So how do you ensure the availability of the task queue?

Me: In several ways:

  1. My thread pool manager will have a scheduled task to regularly detect the current task queue status of the thread pool in Map. It will set a waterThreshold. If the thread pool exceeds the waterThreshold, an alarm will be generated.
  2. Daily drills, will do pressure on the thread pool, if the water level, but also according to the thread name to do demotion, dynamic adjustment of the number of core threads and queues, of course, also limited flow, demotion and other section protection.

Interviewer: So how do you properly split the thread pool, the number of core tasks and the size of the task queue?

Me: This is an old question.

[Recommendation] To know the approximate average time of each service, you can configure an independent thread pool to separate slow services from the main thread pool, so that all service threads do not die together.

Alibaba R&D Manual

  1. According to the type of the task, the task is split into different thread pools, named respectively.

  2. The CPU can be set to about the number of CPU cores. Less context switching is required. The I/O intensive task can be set to more.

  3. A thread pool variable is also available: LargestPoolSize specifies the maximum number of threads that have reached in the thread pool. For example, you can set the number of threads to be large enough at the beginning, and check the maximum value of this parameter after the pressure test. At the same time, refer to the system performance indicators, such as COU, IO, MEM, etc.

  4. Here’s another formula to use: The optimal number of threads = ((thread wait time + thread CPU time)/thread CPU time) * number of cpus

  5. There are also open source AIDS for measuring the reasonable number of threads in a thread pool.

Interviewer: What about the rejection strategy? Understand?

Me: The reject policy is when the number of tasks exceeds maximumPoolSize and you have to reject them.

Interviewer: Tell me more about it

Me: You can specify a rejection policy or implement it yourself. The JDK provides four rejection policies by default.

  • AbortPolicy

    By default, rejection policies directly behind RejectedExecutionException

  • DiscardPolicy

    The task is discarded without throwing an exception

  • CallerRunsPolicy

    The caller performs the rejected task, for example, the main thread calls the Submit task of the thread pool, but the task is rejected, then the main thread directly executes.

    But if the thread pool is already closed, the task is discarded.

    public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
  // Thread pool is not closed
  if(! e.isShutdown()) {// Run directly, without letting the thread pool execute itr.run(); }}Copy the code

  • DiscardOldestPolicy

    Discards the longest waiting task in the queue, and then tries to execute the rejected task.

    But if the thread pool is already closed, the task is discarded

    public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
  if(! e.isShutdown()) {// Discards the queue header task
    e.getQueue().poll();
    // The thread pool attempts to execute the taske.execute(r); }}Copy the code

Interviewer: Which of these rejection strategies would you choose?

Me: I choose to refuse to answer

Interviewer: I choose you to go home and wait for the announcement.