This is why’s 71st original article

An interview question


What do you say, guys?

I think if you’ve been working for two years or so, or if you’ve prepared for an interview, it’s probably a pretty easy question to answer. This question in the rules, the test point is clear, can say things are not a lot.

But it’s all written in blood, so why not analyze it?

Here’s an interview question:

1000 + concurrent threads, 10 machines, 4 cores per machine, design thread pool size.

The information given in this question is very simple, but the advantage of simple is that there is plenty of room for imagination.


When I first saw this question, I intuitively felt two points:

  1. Thread pool design.

  2. Load balancing policy.

I will tell you straight away that both of these points are within the range of my previous article:

How to set the thread pool parameters?

Blood Pumping: a 20,000-word guide to Five Load balancing Strategies

Below, I will analyze the two test points I feel.

Thread pool design

Let’s think simple: 1000 concurrent threads are assigned to 10 machines, so one machine is handling 100 concurrent requests.

100 concurrent requests, that’s not much.

And he doesn’t say whether 1000 concurrent threads come in every second or 1000 concurrent threads come in every once in a while.

Answer this question first from the perspective of thread pool design.

To answer this question well, you must have two basic stores of knowledge:

  1. Custom thread pool 7 parameters.

  2. JDK thread pool execution flow.

Let’s start with the first one, the seven parameters of the custom thread pool.

java.util.concurrent.ThreadPoolExecutor#ThreadPoolExecutor


Harm, these 7 parameters I really do not want to say, you go to the history article, I have written many times. If you don’t tell me what you know, you’re sorry I wrote these articles.

And the Javadoc on this class is pretty clear. This javadoc was written by Doug Lea and you didn’t even read it?

In case you get lazy, I’ll paste down what Pops wrote and we’ll read it sentence by sentence.

I will say one last time about these parameters in this article.

If the future article IF I talk about these parameters, I will not call why elder brother, after you call me Wang.

Write write, how still have a kind of angry feeling. Seems to suddenly understand that year in the podium more speak more angry of the math teacher said: this topic I have spoken how many times! Who else is wrong?


Ok, not angry, say parameters:

  1. CorePoolSize: The number of threads to keep in the pool, even if they are idle, unless {@code allowCoreThreadTimeOut} is set Whether or not they are free after creation. Thread pools need to hold the number of corePoolSize threads unless allowCoreThreadTimeOut is set.

  2. MaximumPoolSize: The maximum number of threads to allow in the pool. (Maximum number of threads: Maximum number of maximumPoolSize threads can be created in the thread pool.)

  3. KeepAliveTime: when the number of threads is greater than the core, This is the maximum time that excess idle threads will wait for new tasks before terminating. (keepAliveTime: If after keepAliveTime, the number of threads exceeding the core has not received a new task, then reclaim.)

  4. Unit: The time unit for the {@code keepAliveTime} argument (keepAliveTime unit.)

  5. WorkQueue: the queue to use for holding tasks before they are executed. This queue will hold only the {@code Runnable} tasks Submitted by the {@code execute} method Queue for tasks to be executed: When the number of submitted tasks exceeds the size of the core thread, the submitted tasks are stored here. It is only used to hold Runnable tasks submitted by the execute method. So don’t translate this as a work queue, okay? Don’t dig a hole for yourself.

  6. ThreadFactory: The factory to use when the executor creates a new thread. Thread project: Used to create thread factories. For example, the thread name can be customized so that when analyzing the virtual stack, you can look at the name and know where the thread came from without being confused.

  7. 2. The handler to use when execution is blocked because the thread bounds and queue are reached. (Reject policy: when the queue is full of tasks and the threads with the maximum number of threads are working, the thread pool of further submitted tasks can not handle it, how to implement the reject policy.)

The first knowledge store is finished, you do not begin to recite, this thing you recite down what use, you have to be combined with the implementation of the process to understand.

Let’s look at the second one: the implementation flow of the JDK thread pool.

A picture is worth a thousand words:


Seven arguments and execution flow about the JDK thread pool.

Although I haven’t participated in the interview for a long time, I think this question is a must.

So if you really do not know, please write a Demo, change a few parameters debugging. Get it under control.

And pay more attention to the interview questions derived from these points.

For example, if the number of core threads in the JDK thread pool is full, subsequent requests will be placed in the blocking queue. When the blocking queue is full, the maximum number of threads will be enabled.

But keep in mind that if we were a Web service and requests came in through Tomcat, the Tomcat thread pool would run differently.

The Tomcat thread pool works like this: if the number of core threads runs out, then the maximum number of threads is used, and finally the task is submitted to the queue. This is to ensure that response time is a priority.

So, Tomcat’s execution flow looks like this:


The technical detail is that it overrides the offer method of the queue itself. In this article said very clear, we can have a look:

Every day, but do you know how hard Tomcat’s thread pool works?

Ok, that’s the first two points.


This question, from a thread pool design perspective, I would answer like this:

As we said earlier, 10 machines, 1000 concurrent requests, an average of 100 requests per service. The server is in a 4-core configuration.

For CPU-intensive tasks, we should minimize context switching, so the number of core threads can be set to 5, the length of the queue can be set to 100, and the maximum number of threads should be the same as the number of core threads.

If the task is IO intensive, we can allocate more core threads to better use the CPU, so the number of core threads can be set to 8, the queue length is still 100, and the maximum thread pool can be set to 10.

Of course, these are all theoretical values.

We can also start with the core thread count equal to 5 and compare the results to determine the most appropriate setting.

At the same time, I think the thread pool parameters should vary with the system traffic.

Therefore, for thread pools in core services, we should use thread pool monitoring to provide early warning. Thread pool response parameters such as the number of core threads and queue length can also be dynamically modified by means.

The answers above can be summed up in four points:

  1. CPU intensive case.
  2. IO intensive case.
  3. Reasonable parameter configuration is obtained by pressure measurement.
  4. The thread pool adjusts dynamically.

The first two are textbook answers, just write them down, the interviewer wants to hear them.

The latter two are the more practical answers that will shine through the interviewer’s eyes.

Based on the limited information in this interview question, the thread pool queue length should be greater than 100.

You can even set a limit, such as a core thread count and a maximum thread count of 4, and a queue length of 96, which is just enough to handle 100 requests, but no more.

So I think from this point of view, it is not for you to give a perfect solution, but to test your understanding of the thread pool parameters and the use of technology.

In the interview I thought that was enough.

And then we diverge a little bit.

For example, the interviewer asks, “What would happen if we didn’t have thread pools in our system?

First, assume that the system we developed is a Web service that runs in a Tomcat container and provides HTTP interfaces externally.

Thread pool technology is not used in the system. So can we just resist 100 concurrent requests?

The answer is yes.

Tomcat has a thread pool. Its maxThreads default is 200 (assuming BIO mode) :


When maxThreads is used up, it is queued. The default queue length (acceptCount) is 100:


In BIO mode, Tomcat is configured to accept up to 300 (200+100) requests by default. Connection refused

So, isn’t it more than enough to handle 100 concurrent requests?

However, if you have 100 concurrent requests per second, and they keep coming in, you can’t handle it.

There are two levels of modification involved:

  1. Tuning Tomcat parameter configuration.
  2. Optimization of system code.

For tuning Tomcat parameter configuration, we can appropriately increase the value of its maxThreads and other parameters.

For optimization of system code, we can introduce thread pooling technology, or we can introduce message queuing. Overall, the goal is to increase system throughput.

Similarly, suppose we are a Dubbo service that provides an RPC interface.

By default, the fixed thread pool is used by the server, and the core thread pool and maximum thread count are both 200. The queue length defaults to 0:


That’s more than enough to handle 100 concurrent requests.

Similarly, if 100 concurrent requests come in every second, the thread pool will soon be full:


In fact, the solution is similar to the situation of Tomcat, adjust parameters, change the system, and asynchrony.

Concurrency in this case is something that most systems are resistant to.

You can also follow up by asking: If traffic doubled due to a promotion, what would you say is the first performance bottleneck in this situation?

The database must be the first place to go wrong, right?

So what do we do?

Distribute the pressure. Sub – library sub – table, read – write separation of these things on the set.

Then cut the peak and fill the valley at the entrance to the system, introduce a cache and, if possible, intercept most of the traffic at the entrance.

Critical service nodes also need to support service circuit breakers and service degradation for large volumes of traffic that cannot be blocked.

If you have to, add money and build machines. There are no problems that can’t be solved by stacking machines, and if there are, you don’t stack enough machines.


That’s how interviews work anyway. Look like a divergent topic, in fact, there are routines can be found.

Okay, first Angle I think that’s all I can think of.

First, I answered the interviewer’s question about thread pool design.

Then we talked about how if we didn’t have a thread pool in our project, we would be able to withstand 1000 concurrent requests.

Finally, the burst flow is briefly described.

Next, let’s talk about load balancing.

Load Balancing Policy

I think it’s a little hidden, but it’s easy to find.

After all, the title says: 10 machines.

And we also assume that on average one machine handles 100 cases.

Behind this assumption is a load balancing strategy: polling load balancing.

If the load balancing strategy is not polling, then our thread pool queue length design may not work.

If we are running in a Tomcat container, assuming nginx, then nginx has the following load balancing policies:

  1. (Weighted) Polling load balancing
  2. Random load balancing
  3. Minimum connections Load balancing
  4. Minimum response time load balancing
  5. Ip_hash Load balancer
  6. Url_hash Load balancing

For RPC services, Dubbo is used as an example to provide the following load balancing policies:

  1. (Weighted) Polling load balancing
  2. Random load balancing
  3. Minimum active load balancing
  4. Minimum response time load balancing
  5. Consistent hash load balancing

Oh, right. Before, another friend asked me, in the scenario of Dubbo + ZooKeeper, is the load balancing done by Dubbo or ZK?

It must be Dubbo, my friend. Source code is written in Dubbo inside, ZK is just a registry, is concerned with its own management of several services, and these services up and down.

When you want to use it, I’ll give you everything you can. Whether you use that service, the so-called load balancing strategy, is not zK’s concern.

But let’s get back to it.

Assuming we use random load balancing, we cannot guarantee that each machine will handle 100 requests.

In this case, the thread pool Settings we gave earlier do not make sense.

The table shows the advantages, disadvantages, and application scenarios of common load balancing policies.


As for the load balancing strategy, MY article “Vomit blood output: 20,000 words to take you to review the five load balancing strategies” has written more than 20,000 words, which is very clear, so I will not repeat it here.

Speaking of load balancing, I was also reminded of a programming contest that Ali held. The topic is “Design and Implementation of adaptive load Balancing”.

The background is as follows:

Load balancing is a fundamental problem in large-scale computer systems. Flexible load balancing algorithms can reasonably distribute requests to less-loaded servers.

Ideally, a load balancing algorithm should minimize service response time (RTT) to maximize system throughput and maintain high performance service capabilities.

Adaptive load balancing refers to that the load balancing algorithm automatically evaluates the service capability of the system in idle, stable, or busy state to better allocate traffic, so that the whole system always maintains good performance and avoids hunger, overload, and downtime.

Specific topics and defense of winning teams can be seen here:

Topic: https://tianchi.aliyun.com/competition/entrance/231714/information?spm=a2c22.12849246.1359729.1.6b0d372cO8oYGK
Answer: https://tianchi.aliyun.com/course/video?spm=5176.12586971.1001.1.32de8188ivjLZj&liveId=41090Copy the code

Recommend you are interested to have a look, or very interesting, can learn a lot of things.

Further reading

This section is taken from the book “Distributed System Architecture”. I think it is a good example, so I would like to share it with you:

Here’s an example of a shopping mall:

The system is deployed on a 4C/8 gb application server and data is stored on an 8C/16 gb database.

Assume that the total number of users in the system is 200,000, and the average daily active users are slightly different according to different system scenarios. Take 20% here, which is 40,000.

According to the 80-20 rule of system division, the peak of the system is 4 hours per day, and the active users account for 80% in the peak period. There are 32,000 active users in the peak period within 4 hours.

Each user sends requests to the system, for example, each user sends 30 times. During the peak period, requests initiated by 32,000 users are 960,000 times, QPS= 960,000 /(4x60x60)≈67 times, and 67 times of requests are processed per second. The processing process is shown in the following figure:


The average number of add, delete, change and check (CRUD) times of an application operation is three times that of an application operation. The specific frequency can be calculated according to the average of the system operation. How many requests can an application or database handle?

Specific analysis is as follows.

  1. First, applications and databases are deployed on servers, which are directly related to server performance, such as CPU, memory, and disk storage.

  2. Applications need to be deployed in containers, such as Tomcat, Jetty, and JBoss. Therefore, the system parameters and configurations of containers can increase or decrease the number of requests processed.

  3. Tomcat is used to deploy applications. The server has 8 gb of memory. The server is mainly used for deploying applications. Therefore, the available memory of Tomcat is set to 4GB (1/2 of the physical memory). The default maximum number of threads on the application server is 1000 (for details, see the system configuration file). Adjust the default number of Tomcat threads to 600 to achieve the maximum processing capability of the system. An application can process a maximum of 1000 requests. When the number of requests exceeds 1000, the requests are temporarily stored in the queue and processed after the thread completes.

  4. The database uses MySQL. MySQL has the concept of connection number. The default value is 100. Each connection request occupies one connection. You can adjust the default number of connections according to the configuration and performance of the database. In this case, you can process 500 requests.

If the number of active users increases from 32,000 to 320,000, 670 requests are processed per second, exceeding the default maximum of 600. In this case, high concurrency occurs. High concurrency is classified into high concurrent read operations and high concurrent write operations.

All right, so that’s the case in the book.

Drought cavity be out of tune

I went to see Kumgang Last Friday night.

It is said that the shooting period is only 2 months, but the movie as a whole is pretty good. The first half is relatively smooth, but the second half is full of climax. For me, there were a few tears.

The first tears point is “Katyusha” rocket out of the time, like fireworks, the soldiers said: this is our Katyusha?

The second tearful spot is when Zhang Fei, played by Zhang Yi, fights against a US reconnaissance plane alone and shouts from an anti-aircraft gun, “A reckless man will be honored through the ages”.

In Lao Zhang’s words: do not think this is a magic drama routine, the scene of history is even more tragic.

Zhang Yi’s acting skills are excellent. One person holds up half of the segment. Best actor book one.

The third tear point is the incendiary bomb fell on the river, and then the sound of “My Motherland” BGM:


A great river has broad waves

The wind blows the rice flowers along the banks

My family lives on the shore

I am used to hearing the boatman’s cry

Got used to the white sails of the ship

This is a beautiful motherland

It’s where I grew up

With the whole story of the bridge, it’s a bit of a teardrop.

To be honest, kumgang Is not a perfect movie. But I urge you to go to the cinema and support this film.

Because this film, shot in the 70th anniversary of the anti-AMERICAN aid to Korea at this special point. Can let fortunate to live in the era of peace we know, now the peace of Changan, the wealth of the country and the strong people are not falling from the sky, is 70 years ago, that a group of only 17 or 18 years old “the most lovely people” with their lives shot down.

I saw a short video on People’s Daily earlier. The words of a veteran were particularly touching. He said:

What is the motherland? When we crossed the Yalu River and saw the flames of war, my motherland was behind me.

Peace has not come easily and we should cherish it.

Hail to the loveliest.

One last word (for attention)

Ok, saw here to arrange a “key three even”, week is more tired, don’t whoring me, need a little positive feedback.

If you find something wrong, you can put it up in the comments section and I will correct it. Thank you for reading, I insist on original, very welcome and thank you for your attention.

I am Why, a literary creator delayed by code. I am not a big shot, but I like sharing. I am a nice man in Sichuan who is warm and interesting.