Author: Enjoy learning classroom old gu
Wechat official account: Enjoy Learning class online
One, foreword
“Sorry, the online server timed out seriously, the request is very slow, it seems that the number of connections reported too many, how to do? “Kids are giving feedback. Normally what our tech bosses do is they increase the connection count and the thread pool, reboot, and see.
Often this approach is an emergency measure, treating the symptoms rather than the root cause, because the cause of the problem is not known.
There is a serious misconception that if the thread pool Settings are too small, it will speed up requests by increasing them.
How to set the size of the thread pool reasonably?
Second, the problem of
If you have two tasks to deal with, one task A and one task B
Plan 1: One thread executes tasks A and B. After A completes, plan B: Two threads A and B execute tasks A and B at the same time
Which one will be faster? I think A lot of people would say, well, plan two, multithreading tasks A and B in parallel, that’s fast. Is that so? Before we answer that question, let’s take you through a little review.
Third, thread execution
The execution of threads is scheduled by the CPU, and each CPU only executes one thread at A time, so what we see as thread A and thread B executing concurrently.
In order to give the user the impression that these tasks are being performed simultaneously, the operating system takes advantage of time slice rotation, in which the CPU serves each task for a certain amount of time, saves the state of the current task, and continues to serve the next task after loading the state of the next task. The state of a task is saved and reloaded, a process called context switching.
The context switching process takes time; Now let’s take a look at the above question, friends and let’s see which solution is faster? Some of you might say plan one, because there’s no thread switching; Scheme 2 requires switching these two threads back and forth, which takes more time.
Friends at this time will not have doubts, that why there is multithreading? Take it easy and look down.
Why multithreading
Think about the process in our real business.
The flow of the image above:
1. Make a network request first
2. The Web server parses the request
3. Request the back-end database for data
4. After obtaining the data, process it
5. Return the processing result to the user
This is the normal request flow when we are dealing with business; Let’s take a look at what computer processing is involved.
1. Network request —–> Network IO
2. Parse the request —–>CPU
3, request database —–> network IO
4, MySQL query data —–> disk IO
5, MySQL returns data —–> network IO
6. Data processing —–>CPU
7. Return data to user —–> Network IO
In real business, we will not only involve CPU calculation, but also network IO and disk IO processing, which is very time-consuming. If the whole process of a thread is the process shown in the figure above, only 2 nodes are really involved in THE CPU, and the other nodes are all IO processing, then the CPU will be free when the thread is doing IO processing, and the CPU utilization will be low.
Now you know what multithreading is for, yes, to improve CPU utilization.
5. Improve QPS/TPS
Measure the system performance, the main indicators of the system (QPS/TPS)
QPS/TPS: Number of requests/transactions that can be processed per second
Concurrency: The number of simultaneous requests/transactions processed by the system
Response time: the average time it takes to process a request/transaction
QPS/TPS = Concurrency/response time
The above formula represents that the greater the number of concurrent requests, the greater the QPS; Therefore, many people will think that increasing the thread pool will increase the number of concurrent requests and improve THE QPS. Therefore, there will be the misunderstanding of most people mentioned in the introduction at the beginning.
In fact, QPS is inversely proportional to the response time, and the larger the response time, the smaller the QPS.
While increasing the number of concurrent threads can improve the QPS, the number of threads can also affect the response time, because we also mentioned context switching. How to set the number of threads?
How to set the number of threads
So how do we allocate threads? We offer a formula:
Optimal number of threads = ((thread wait time + thread CPU time)/thread CPU time) * number of cpus
Note This formula is also shared by predecessors, of course, I read the article on taobao’s front desk system optimization practice before, and it is similar to the above formula, but in the number of CPU, they are more detailed, the above formula is just for reference. But no matter what the formula is, it is ultimately optimized after running in production.
We continue the above task, our server CPU number is 4 cores, a task thread CPU time is 20ms, thread wait (network I/o, disk I/o) time is 80ms, the optimal number of threads :(80 + 20)/20 * 4 = 20. The optimal number of threads is 20.
From this formula we can see that the larger the waiting time of a thread, the larger the number of threads should be set, which is exactly in line with our analysis above and can improve CPU utilization. From another point of view, the number of threads is set according to our own business, we need to go to the pressure test, set a reasonable value.
Vii. Basic general standards
We may ask, because many businesses are concentrated in a thread pool, unlike the above case is relatively simple, in fact there are too many businesses, how to set? This is to stress test to adjust. However, our predecessors have helped us to summarize a basic value (ultimately, it depends on the performance)
1. CPU intensive: Memory processing services. Generally, the number of threads is set to the number of CPU cores + 1 or number of CPU cores x 2. If the number of cores is 4, it is usually set to 5 or 8
2, IO intensive: file operation, network operation, database operation, the general thread setting is: CPU core/(1-0.9), the number of cores is 4, the general setting is 40
Eight, summary
Today we introduce the setting of thread size, some mistakes of friends. By now, we have a new understanding of threads, which is not as rude as before. We should analyze why the thread is so slow, where the bottleneck of the system appears, and reduce the time consuming of the bottleneck.
In addition, I recommend you to take a look at Redis, Nginx; Why are they so fast? In fact, there is something in common with the knowledge point of this article.
Popular content:
1, the system Bug again? These tuning tips can help you solve most of your problems!
How do YOU deploy Tomcat in a production environment?
God level code annotation appreciation, like to take to use!
Open source 13 Spring Boot excellent learning projects! Over 53K stars, catch them all!
5, Huawei Haisi president letter to employees: no longer have the opportunity to change tires, will be independent science and technology!
Don’t understand these 12 grammar sugar, don’t say you can Java!
7. Overtime carnage caused by minor paging problems!
10 thousand attributes, 10 billion data, 100 thousand transactions per second, how to design the architecture?
Java backend technology (ID: JavaITWork), learn Java with 200,000 people! Reply:1024, you can get it for free! Includes SSM, Spring family bucket, microservices, MySQL, MyCat, cluster, distributed, middleware, Linux, network, multi-threading, Jenkins, Nexus, Docker, ELK and so on free learning video, continue to update!