Every day, but do you know how hard Tomcat’s thread pool works?
December 28, 2023
by Olivia Smith
No Comments
This is why’s 45th original article. Talk about different thread pool execution policies and thread rejection policies, and discuss how to use up the maximum thread pool before putting tasks on queues.
Drought cavity be out of tune
Hi, I’m Why, a Sichuan programmer and a good man in Chengdu.
First of all, the characteristics of this number, before sharing technology, a brief talk about life. Let the temperature of the article be a little bit more.
The picture above was taken during one of my runs. Before the event, the race party held a message gathering activity to collect one slogan for each kilometer of road signs.
My message was chosen:
Everyone knows what you’re standing for, but you should.
I’m talking about running a marathon, but I’m also talking about other things.
I remember the sun that day. The sun was burning, and there was very little shade on the road. The bitter part is that I also sign up for an ultramarathon.
How tanned it is, let me show you the comparison:
The heat was unbearable, so that about 30 kilometers in my heart appeared two small people:
One said, I am so tired, I can’t run any more, I will quit the race.
One said: good ah good ah, I also bask in ah, quit the game quit.
I say: bah, see you two of disgraceful things, let me take you to the end
So I came across the slogan I submitted 36 kilometers away. I was so happy that I stopped to take some photos. Say to yourself: when you can’t hold on, hold on.
The last 3 kilometers uphill, I cramp I don’t know how many times. Far to see the end of the arch when I suddenly thought of dunhuang when the realization of a word: their own to their own hard, not hard, is happiness.
All right, back to the article.
Counterintuitive JDK thread pools
Let’s start with a JDK thread pool.
Using my previous article “How to Set thread Pool parameters?” meituan gave an answer that blew the interviewer’s mind. Examples from the section “Dissuade a wave first” :
Q: This is a custom thread pool. Suppose there are 100 time-consuming tasks at this time. How many threads are running?
The correct answer was answered in the previous article and will not be repeated here.
However, I met many people who were not familiar with THE JDK thread pool during the interview.
Most of these people have the common problem of guessing when faced with a question they don’t know well.
When the interviewees met this question, they smiled on the surface, but actually I already figured out their inner activities:
The number of core threads is 10 and the maximum number of threads is 30. And they know that the answer is either 10 or 30.
Multiple choice, 50 percent chance, don’t you take a chance?
Wait, 30 is the maximum number of threads? Most? I feel like this is it.
So after a flash of thought, and I looked at each other, confidently said:
So I smiled and told him, “Go down there and get to know you better. Let’s talk about something else.”
Indeed, if I didn’t know anything about the JDK thread pool rules, I would intuitively have used up all available threads in the pool before submitting the task to a queue, regardless of the core or maximum number of threads.
Unfortunately, the JDK thread pool is counter-intuitive.
Is there a thread pool that fits our intuition?
Yes, you often use Tomcat, where the thread pool runs to the maximum number of threads before submitting tasks to the queue.
Let me take you through it.
Tomcat thread pool
Tomcat server.xml:
Familiar? Who has learned Java Web and hasn’t configured this file? Who has configured this file without paying attention to the Executor configuration?
For details about configurable items, see the official document:
At the same time I found a configurable parameter description in Chinese as follows:
Notice that the first parameter is className, which is missing the letter C.
Then there are two parameters that have not been introduced, I would like to add:
PrestartminSpareThreads: a Boolean type, when the server starts, whether to create a minimum number of idle thread thread (core) threads, the default value is false.
2. ThreadRenewalDelay: long, when we configure the ThreadLocalLeakPreventionListener it will monitor whether a request to stop. When a thread is stopped, it will be rebuilt if necessary. To avoid multiple threads, this setting checks whether two threads are created at the same time. If so, the creation will be delayed according to this parameter. If rejected, the thread will not be rebuilt. The default value is 1000 ms. A negative value indicates no update.
We focus on the className parameter. If not configured, the default implementation is:
org.apache.catalina.core.StandardThreadExecutor
Let’s look at this method first (note that Tomcat source code is 10.0.0-M4 in this article) :
Lines 123 to 130 are where the Tomcat thread pool is built.
Line 123
taskqueue = new TaskQueue(maxQueueSize);
Create a TaskQueue that inherits from the self-linkedBlockingQueue:
The comments on this queue are worth noting:
This is a task queue designed specifically for a thread pool. When used with a thread pool, it is different from a normal queue.
A queue length is also passed, which defaults to integer.max_value:
Line 124
TaskThreadFactory tf = new TaskThreadFactory(namePrefix,daemon,getThreadPriority());
Build a ThreadFactory with three input arguments as follows:
NamePrefix: indicates the namePrefix. It can be specified, and the default is tomcat-exec-.
Daemon: Whether to start in daemon thread mode. The default is true.
Priority: indicates the priority of the thread. Is a number between 1 and 10. The default is 5.
Line 125
executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), maxIdleTime, TimeUnit.MILLISECONDS,taskqueue, tf);
Build a thread pool with its 6 input parameters as follows:
I won’t explain what this means, but it is the same as JDK thread pools.
Just to show you the default parameters.
It is also important to note that ThreadPoolExecuteor here is Tomcat, not JDK, even though it has the same name.
Take a look at Tomcat’s ThreadPoolExecuteor comment, which mentions two points: the total number of commits and the rejection policy. We’ll talk about that later.
Set the threadRenewalDelay parameter. It’s not the point of this article, so don’t worry about it.
Lines 127-129
if (prestartminSpareThreads) {
executor.prestartAllCoreThreads();
}
Copy the code
This parameter sets whether to prestart all core thread pools, as discussed in the previous article.
The prestartminSpareThreads parameter is false by default. But I think you set true to this one too many times. It’s totally unnecessary.
Why is that?
Because this method was already called when building the thread pool at line 125:
The prestartAllCoreThreads method is called regardless of which thread pool constructor you call.
So, is this a Bug in Tomcat? Grab your keyboard and give it some PR.
Line 130
taskqueue.setParent(executor);
This line of code is critical. Without this line of code, Tomcat’s thread pool behaves just like the JDK’s thread pool.
Examples of procedures:
Custom thread pools can hold up to 150+300 tasks.
When the 24 lines are commented, the Tomcat thread pool will run the same way as the JDK thread pool, with only 5 core programs running.
When line 24 is uncommented, the Tomcat thread pool creates up to 150 threads and submits the remaining tasks to a custom TaskQueue.
I will also provide a copy and paste direct run version, you run separately, try it, see the results:
public class TomcatThreadPoolExecutorTest {
Public static void main(String[] args) throws InterruptedException {String namePrefix = "Why not only technology-exec -"; boolean daemon = true; TaskQueue taskqueue = new TaskQueue(300); TaskThreadFactory tf = new TaskThreadFactory(namePrefix, daemon, Thread.NORM_PRIORITY); ThreadPoolExecutor executor = new ThreadPoolExecutor(5, 150, 60000, TimeUnit.MILLISECONDS, taskqueue, tf); //taskqueue.setParent(executor); for (int i = 0; i < 300; i++) { try { executor.execute(() -> {logStatus(executor, "Create task "); try { TimeUnit.SECONDS.sleep(2); } catch (InterruptedException e) { e.printStackTrace(); }}); } catch (Exception e) { e.printStackTrace(); } } Thread.currentThread().join(); } private static void logStatus(ThreadPoolExecutor executor, String name) { TaskQueue queue = (TaskQueue) executor.getQueue(); Println (thread.currentThread ().getName() + "-" + name + "-:" + "number of core threads :" + executor.getCorePoolSize() + "\ t activity threads:" + executor. GetActiveCount () + "\ t maximum number of threads:" + executor. GetMaximumPoolSize () + "\ t addition number: +" Executor.gettaskcount () + "\t current queue number :" + queue.size() + "\t remaining queue size :" + queue.remainingCapacity()); }Copy the code
Copy the code
}
Then we’ll look at what this line of code does and see how it reverses the JDK thread pool.
There are no secrets under the source code
If you’re familiar with the JDK thread pool source code, you can probably guess that Tomcat has tweaked the controls on new threads in this area:
PS: It should be noted that the screenshot above shows the EXECUTE method of the JDK thread pool. Because Tomcat thread pool commits also reuse this method. But workqueues are different.
Then you should first get familiar with the workflow and parameters, then write a Demo, and then go to the crazy Debug. Then you’ll find the place, and you’ll find it’s not hard to find.
Okay, so the top focus on what I’ve circled.
In line 1371 of the screenshot, if the task is not successfully queued (provided the thread pool is running), line 1378 of the logic is executed, and this logic is used to create non-core threads.
So, after the above derivation, it’s clear that Tomcat just needs to work with the offer method for custom queues.
So, let’s focus on this method:
org.apache.tomcat.util.threads.TaskQueue#offer
To make it more intuitive that it’s running smoothly, I’m going to make a breakpoint at line 80 and run the program as follows:
You can see some of the parameters in it, which will be used in the next tutorial:
The first if judgment
If (parent = null);
As you can see from the breakpoint run parameter screenshot, parent is Tomcat’s ThreadPoolExecutor class.
When parent is null, the original offer method is called.
So, remember what I said earlier?
Now do you see why?
Source code, that’s the source code. Truth, that’s the truth.
So, it’s not empty, it doesn’t satisfy the condition, it goes to the next if judgment.
Second if judgment
First, it should be made clear that the number of threads currently running must be greater than or equal to the number of core threads at the time of the second judgment, and less than the maximum number of threads.
So, the second if determines whether the number of running threads is equal to the maximum number of threads. If equals, all threads are working, and the task is thrown to the queue.
The breakpoint run parameter screenshot shows that the current number of runs is 5 and the maximum number of threads is 150. If the condition is not met, proceed to the next if judgment.
The third if judgment
First let’s see what getSubmittedCount gets:
GetSubmittedCount gets the number of tasks that are currently committed but not yet completed, and its value is the number of tasks in the queue plus the number of tasks that are running.
As you can see from the breakpoint run parameter screenshot, the current value is 6.
Parent-getpoolsize () is 5.
If the condition is not met, proceed to the next if judgment.
If the number of submitted tasks that have not yet been completed is smaller than the number of running threads in the thread pool, Tomcat will queue the tasks instead of executing them immediately.
In fact, this is also very logical and simple approach.
There’s a free thread, so if you throw it in the queue, it’s consumed by the free thread. Why do it at once? In addition to breaking the process, additional implementation is required.
Hard work pays no dividends. There is no need.
The fourth if judgment
This judgment is crucial.
If the number of threads currently running is less than the maximum, return false.
Note that the previous several if judgments are put into the queue if the condition is not met. In this case, false is returned if the condition is not met.
What does it mean to return false?
That means 1378 lines of code to create a thread.
So, the flowchart looks something like this:
Let’s talk about rejection strategies
The rejection strategy needs to look at this method:
If the queue is full, it waits a specified time before being queued again.
If the queue is full again, a reject exception is thrown.
The logic is similar to when you go to the bathroom and the pit is occupied. At this point, your body is telling you that your sphincter can hold out for at most a minute.
So you hold your watch by the door, take a deep breath, close your eyes and meditate for a minute.
Good luck, and then a look: ah, there is an empty pit, hurriedly occupied.
Luck is bad, go again a look: ah, still have no position, how to do? Throw an exception. I don’t know how to throw it, but imagine.
So let’s look at this place and see how the Tomcat code is implemented:
The catch part first determines whether the queue is Tomcat’s custom queue. If so, enter the if branch.
The key logic is in this if judgment.
You can see line 172:
if (! queue.force(command, timeout, unit))
The force method of the queue is called. We know that BlockingQueue does not have a force method.
So this force method is specific to Tomcat custom queues:
public boolean force(Runnable o, long timeout, TimeUnit unit) throws InterruptedException {
if (parent == null || parent.isShutdown())
throw new RejectedExecutionException(sm.getString("taskQueue.notRunning"));
//forces the item onto the queue, to be used if the task is rejected
return super.offer(o,timeout,unit);
}
Copy the code
Go in a look to discover: harm, also but so. It’s just a wrapper around the native Offer method.
If you make it to the queue, everything’s fine.
If not successfully enqueued, an exception is thrown and the submittedCount parameter is maintained.
The submittedCount parameter = number of tasks in the queue + number of tasks running.
So, we have to subtract one here.
So much for the rejection strategy.
But I led you to the source code for this place. What if you want to find it yourself?
Yes, you do. You want to test the rejection strategy. All it has to do is trigger its rejection policy.
Like this:
Submit 500 tasks to a thread pool that can only hold 450 tasks.
It then throws this exception:
You will find line 174 of the Tomcat thread pool:
Then you hit a break point and go play.
So what about waiting at the door for a minute before we go in?
We can just tell the thread pool the parameters, like the following:
Then run it again, because when the queue is full, the reject exception is triggered, and then wait 3 seconds before submitting the task. We submitted a task that could be executed in two seconds.
So, in this scenario, all tasks will be executed normally.
Now do you know how hard Tomcat works to get the tasks you give it done as quickly and fully as possible?
Small eggs
While looking at the Tomcat custom queue, I found the author’s comment:
This will set the forcedRemainingCapacity parameter to 0.
When was this parameter set?
This is when the following closing method is used:
As you can see, the author directly sets the forcedRemainingCapacity parameter to 0 before calling the setCorePoolSize method.
Written comments above reason is that the JDK ThreadPoolExecutor. SetCorePoolSize method to check whether remainingCapacity is 0.
The author of Tomcat twice said: I don’t see why. I did not understand why.
So, he faked condition.
Anyway, he says he doesn’t understand why the setCorePoolSize method in the JDK should limit the remaining queue length to 0 when reducing the core thread pool.
Don’t ask. Asking is the rule.
So I looked at the setCorePoolSize method of the JDK thread pool and found that this limitation was in JDK 1.6. Later versions of the JDK have undergone a major refactoring of the thread pool to remove this limitation:
So what’s the problem with setting Tomcat to 0?
The normal logic is that the remaining queue size = queue length – number of queued tasks in the queue.
When you monitor its thread pool (queue length is 300), it should look like this:
However, when you call the contextStopping method, you may have a problem like this:
It clearly doesn’t fit the algorithm.
Well, if you need to monitor the Tomcat thread pool in the future, and the JDK version is 1.6 or older. Well, you can remove this restriction to avoid false alarms.
All right, congratulations, my friend. I learned a basic useless knowledge point, and I added a little more strange knowledge.
Dubbo thread pool
Here’s another extension to Dubbo’s thread pool implementation.
From the code, the offer method is adjusted immediately after the failure, with no wait time.
That is to say, the interval between two offers is very short.
In fact, I do not quite understand why to write so, may be the author left a hole to expand it?
Because if I do, why don’t I just call this method?
java.util.concurrent.LinkedBlockingQueue#offer(E)
Is also the author wants in extremely short time can bet? Who knows?
Then you can see that the thread pool also does a great deal on the rejection policy:
You can see that the logs are printed in great detail at the warn level:
The dumpJStack method, known by its name to Dump threads, remains on-site:
In this method, he uses the JDK default thread pool to exception Dump threads.
Wait, didn’t the Ali spec say that the default thread pool is not recommended?
In fact, this criterion depends on how you take it. In this scenario, a native thread pool would suffice.
And look at the second red box: the shutdown method is executed after the commit, with a nice warning.
The thread pool must be shutdown or it will result in OOM.
That’s the details, my friends. The devil is in the details!
Why is shutdown not shutdownNow? What’s the difference between them? Why not call shutdown method to get OOM?
Knowledge, my friends, is knowledge!
Well, here is the end of this article to share.
In the future, when the interviewer asks you how to run the JDK thread pool, you will have to change your mind and do another one:
We could use up the maximum number of threads first and then queue tasks. By customizing the queue, you can override its offer method. Both Tomcat and Dubbo THAT I know of so far provide thread pools with this strategy.
Between the understatement and a beautiful bitch. Let the interviewer move on to the next point, so that you can show more about yourself.
One last word (for attention)
This article focuses on the running flow of the Tomcat thread pool, which is really different from the JDK thread pool flow.
Why would a Tomcat thread pool do this?
Because Tomcat handles IO intensive tasks, users are waiting for a response, and you end up waiting for users’ requests to queue up when you can handle them?
It’s not good. It’s not good.
Ultimately, it comes back to the question of whether the task type is IO intensive or CPU intensive.
If you’re interested, check out my article “How to set thread pool parameters? Meituan gives an answer that will blow your interviewer’s mind.”
Give me a thumbs up, zhou is more tired, don’t fuck me, need some positive feedback.
Thank you for reading, I insist on original, very welcome and thank you for your attention.
I am why Technology, a nice sichuan man who is not a big shot, but likes to share, warm and interesting.
Welcome to pay attention to the public number [WHY more than technology], adhere to the output of original. Share technology, taste life, wish you and I progress together.
