Unified Thread Pool | Hackathon 2019 outstanding project introduction 43/100 save drafts Post TiDB_PingCAP
Author: Eric Sharry
This article, written by Eric Xia, a member of the Unified Thread Pool team, introduces the design and implementation process of the Unified Thread Pool project. This project implemented a unified adaptive thread pool to handle read requests in TiKV, significantly improved performance and predictably limited interference from large queries to small requests, and won the first prize in TiDB Hackathon 2019.
It has been more than half a month since TiDB Hackathon came to an end. I am still very excited to recall the experience of this competition and winning awards. I am a junior student in South China University of Technology. I joined the TiDB Hackathon together with Yi Lin, an intern at PingCAP. The theme of the competition is “Improve”, which means to Improve the performance and usability of TiDB and related projects. The entry point of our project design is:
-
TiKV’s existing thread pool does not perform well in mixed size query scenarios.
-
The number of thread pools needs to be configured for different environments and scenarios, which is costly to use and learn.
Therefore, Yilin and I tried to re-implement a thread pool for TiKV to solve this problem, so as to achieve the effect of improving the overall performance. In addition to optimizing the Thread Pool for read requests, we planned to use this Thread Pool to replace other Thread pools in TiKV, resulting in our entry Unified Thread Pool.
The project design
In the current thread pool scheme of TiKV, there are three thread pools: Coprocessor, Storage and Scheduler. These thread pools were originally designed to separate different tasks and reduce their interaction. This method is simple and crude, and its disadvantages are obvious, such as:
-
Because multiple thread pools coexist, each thread pool is limited to not being able to use all of the system’s resources.
-
There are two or three different priority thread pools per set of tasks, but in practical terms this isolation does not work very well.
-
In the current TiKV, each thread pool needs to be configured separately, such as Scheduler, Storage, Coprocessor.
Our Unified Thread Pool is a Thread Pool that simulates multi-level feedback queue scheduling in user mode, which can solve the disadvantages of the current Thread Pool scheme. The specific comparison between the two is shown in the following table:
| Current thread pool scheme | Our improvements |
|---|---|
| Multiple thread pools coexist, and each thread pool cannot allocate all resources | A unified thread pool that allocates as many resources as possible |
| Priority is achieved by allocating it to different thread pools, but not very well | Internal scheduling by time |
| The impact of large requests on small requests is uncontrollable | Predictable impact of large requests on the system |
The scheduling scheme of a Unified Thread Pool is based on the multi-level feedback queue algorithm. There are three queues in a Unified Thread Pool, and workers take tasks from different queues in different numbers each time to indicate their priorities. Unlike scheduling on a per-task basis in OS scenarios, a TiDB Query may generate multiple TiKV tasks across multiple regions, as shown in Figure 1:
So here we need to schedule in units of query in TiDB. To achieve this we let TiDB send a token as an identifier when sending a query, and in the thread pool we adjust the priority by token (Query) as a whole.
Another important change is that there may now be large Batches of Coprocessor requests in TiKV that are executed in batches. These may consist of hundreds of batches, each of which may be executed in seconds, making it impossible to schedule them. PingCAP will also be covered in a follow-up article on Coprocessor’s implementation of quantization. Therefore, we use Rust’s latest Async /await coroutine mechanism to hand over execution between Coprocessor batches, as shown in the figure below, which normally takes about one second. The batch of about 500 tasks will now be a number of small tasks of about a millisecond in length, with active handover of execution between each small task.
So far, a Unified Thread Pool can dynamically adjust queue parameters to allocate resources to tasks of different sizes. The parameters to be set are very simple, with only one parameter representing the percentage of computing resources that a small task should occupy when tasks of different sizes are mixed. Through testing, we can see that the effect of this allocation is more accurate, as shown in the figure below.
The game process
Hacking Time
After getting the machine (8C16G) provided by UCloud on the morning of October 26th, we started to deploy TiDB cluster for testing. The first deployment solution was 3TiDB + 3TiKV, but when the cluster was up and running we found that the bottleneck seemed to be on TiDB when the requests came in, so we took the TiKV cluster down one, the situation improved but TiKV was still unable to run to full capacity. After a lot of struggling, we shoveled the whole cluster and redeployed it. After a second deployment with 4TiDB + 1TiKV + 1Tester, the bottleneck finally appeared on TiKV.
The detailed test scenario was to use the Tester machine to send requests to four TiDB and then detect latency and QPS. Sysbench tested 32 tables with 10,000,000 entries each, with a total capacity of approximately 80GB. We simulate both small and large requests using sysbench point_selec t and read_only, and large requests using four clients continually SELECT COUNT(*) FROM.. To sweep the table. Below is our comparison of Unified Thread Pool against TiKV Master in the above test environment, which shows a 20% to 50% throughput improvement for simple small requests.
Eat and sleep with your eyes open
Hackathon is known for its constant theme of eating and drinking, and this one will be no different. PingCAP provides the contestants with three full meals, and the decor of the Guangzhou Office is like a vacation. For lunch, there are so many flavors to choose from, and in the evening, a conference table of shrimp and crab is really a visual impact. In addition to the main meal, there is an unlimited supply of snacks coffee happy water, it is so happy.
One thing I was curious about before the game was that there were three mattresses at night, so I didn’t know how difficult the scene would be. It turns out that other people don’t sleep at all when they get sleepy…
At about two o ‘clock in the evening, I felt the test effect was still good, so I found a small conference room with a bed for dreaming. However, the air of the air mattress had leaked out, and I tried to restart the air pump after I found the air device in the pantry. However, the sound of the air pump was extremely loud in the middle of the night after the plug was inserted. I don’t know how the staff had filled up the three mattresses quietly before. To prevent noise, we rolled the mattress into a small room on the next floor of the building (next to the washroom) and inflated it. Two uncles, who looked like security guards, came to see what was going on. Next and Yi Lin two people carrying two meters of mattress through the conference room through the tea room through the station, into the small bedroom that section of the road, it is very impressive.
The air conditioning effect of this building is really very strong. Although it is not the first time I was cold in your company, I still didn’t bring my coat when I came back this time. Even if the air conditioning has been turned off before going to bed, or with the coat but did not take out yi Lin shrank into two groups on the bed.
Write after Hackathon
At the last Demo Time of the competition, I looked at other people’s projects so good that I felt like running away in advance. Fortunately, IT was I who made the presentation, and I was surprised to win the first prize. Now though Hackathon has been finished, I still want to continue to improve this work. Now it can increase the maximum throughput, but it can go even further in terms of latency. During the competition, our thread pool was designed based on relatively simple Juliex. In the future, we plan to optimize it by referring to some mature thread pools such as Tokio, hoping to integrate it into master. You can continue to work on this project together in the TiDB Performance Challenge, which links to the project.
Finally, THANK Teacher Yilin for being so strong and willing to take me to play, thanks to PingCAP for letting me eat and drink freely 😀
Attached: Team Demo Show video (0:00-10:35)
pingcap.com/blog-cn/uni…
Author: Eric Sharry
This article, written by Eric Xia, a member of the Unified Thread Pool team, introduces the design and implementation process of the Unified Thread Pool project. This project implemented a unified adaptive thread pool to handle read requests in TiKV, significantly improved performance and predictably limited interference from large queries to small requests, and won the first prize in TiDB Hackathon 2019.
It has been more than half a month since TiDB Hackathon came to an end. I am still very excited to recall the experience of this competition and winning awards. I am a junior student in South China University of Technology. I joined the TiDB Hackathon together with Yi Lin, an intern at PingCAP. The theme of the competition is “Improve”, which means to Improve the performance and usability of TiDB and related projects. The entry point of our project design is:
TiKV’s existing thread pool does not perform well in mixed size query scenarios.
The number of thread pools needs to be configured for different environments and scenarios, which is costly to use and learn.
Therefore, Yilin and I tried to re-implement a thread pool for TiKV to solve this problem, so as to achieve the effect of improving the overall performance. In addition to optimizing the Thread Pool for read requests, we planned to use this Thread Pool to replace other Thread pools in TiKV, resulting in our entry Unified Thread Pool.
In the current thread pool scheme of TiKV, there are three thread pools: Coprocessor, Storage and Scheduler. These thread pools were originally designed to separate different tasks and reduce their interaction. This method is simple and crude, and its disadvantages are obvious, such as:
Because multiple thread pools coexist, each thread pool is limited to not being able to use all of the system’s resources.
There are two or three different priority thread pools per set of tasks, but in practical terms this isolation does not work very well.
In the current TiKV, each thread pool needs to be configured separately, such as Scheduler, Storage, Coprocessor.
Our Unified Thread Pool is a Thread Pool that simulates multi-level feedback queue scheduling in user mode, which can solve the disadvantages of the current Thread Pool scheme. The specific comparison between the two is shown in the following table:
The current thread pool scheme that we’ve improved is that multiple thread pools coexist, each thread pool can’t allocate all the resources one thread pool can allocate as many resources as possible by allocating them to different thread pools to achieve priority, Table 1 Comparison with current Thread pools The scheduling scheme of Unified Thread Pool is based on the multi-level feedback queue algorithm. In a Unified Thread Pool, there are three queues. Workers take tasks from different queues in different numbers each time to indicate their priorities. Unlike scheduling on a per-task basis in OS scenarios, a TiDB Query may generate multiple TiKV tasks across multiple regions, as shown in Figure 1:
Figure 1 TiDB Query versus TiKV Task so here we need to schedule in units of TiDB query. To achieve this we let TiDB send a token as an identifier when sending a query, and in the thread pool we adjust the priority by token (Query) as a whole.
Another important change is that there may now be large Batches of Coprocessor requests in TiKV that are executed in batches. These may consist of hundreds of batches, each of which may be executed in seconds, making it impossible to schedule them. PingCAP will also be covered in a follow-up article on Coprocessor’s implementation of quantization. Therefore, we use Rust’s latest Async /await coroutine mechanism to hand over execution between Coprocessor batches, as shown in the figure below, which normally takes about one second. The batch of about 500 tasks will now be a number of small tasks of about a millisecond in length, with active handover of execution between each small task.
Figure 2 divides requests into multiple executions. At this point, the Unified Thread Pool is able to dynamically adjust queue parameters to allocate resources to small and large tasks. The parameters to be set are very simple, with only one parameter representing the percentage of computing resources that small tasks should occupy when large and small tasks are mixed. Through testing, we can see that the effect of this allocation is more accurate, as shown in the figure below.
Hacking Time receives the 8C16G machine provided by UCloud on the morning of October 26, 2012, and starts to deploy the TiDB cluster for testing purposes. The first deployment solution was 3TiDB + 3TiKV, but when the cluster was up and running we found that the bottleneck seemed to be on TiDB when the requests came in, so we took the TiKV cluster down one, the situation improved but TiKV was still unable to run to full capacity. After a lot of struggling, we shoveled the whole cluster and redeployed it. After a second deployment with 4TiDB + 1TiKV + 1Tester, the bottleneck finally appeared on TiKV.
The detailed test scenario was to use the Tester machine to send requests to four TiDB and then detect latency and QPS. Sysbench tested 32 tables with 10,000,000 entries each, with a total capacity of approximately 80GB. We simulate both small and large requests using sysbench point_selec t and read_only, and large requests using four clients continually SELECT COUNT(*) FROM.. To sweep the table. Below is our comparison of Unified Thread Pool against TiKV Master in the above test environment, which shows a 20% to 50% throughput improvement for simple small requests.
As we all know, Hackathon has a constant theme of eating and drinking, and this one is the same. PingCAP provides the contestants with three full meals, and the decor of the Guangzhou Office is like a vacation. For lunch, there are so many flavors to choose from, and in the evening, a conference table of shrimp and crab is really a visual impact. In addition to the main meal, there is an unlimited supply of snacks coffee happy water, it is so happy.
One thing I was curious about before the game was that there were three mattresses at night, so I didn’t know how difficult the scene would be. It turns out that other people don’t sleep at all when they get sleepy…
At about two o ‘clock in the evening, I felt the test effect was still good, so I found a small conference room with a bed for dreaming. However, the air of the air mattress had leaked out, and I tried to restart the air pump after I found the air device in the pantry. However, the sound of the air pump was extremely loud in the middle of the night after the plug was inserted. I don’t know how the staff had filled up the three mattresses quietly before. To prevent noise, we rolled the mattress into a small room on the next floor of the building (next to the washroom) and inflated it. Two uncles, who looked like security guards, came to see what was going on. Next and Yi Lin two people carrying two meters of mattress through the conference room through the tea room through the station, into the small bedroom that section of the road, it is very impressive.
The air conditioning effect of this building is really very strong. Although it is not the first time I was cold in your company, I still didn’t bring my coat when I came back this time. Even if the air conditioning has been turned off before going to bed, or with the coat but did not take out yi Lin shrank into two groups on the bed.
It was written after Hackathon when I saw other people’s projects at the last Demo Time of the competition were so excellent that I felt like running away in advance. Fortunately, IT was not me who made the presentation and I was surprised to win the first prize. Now though Hackathon has been finished, But I still want to work on it. Now it can increase the maximum throughput, but it can go even further in terms of latency. During the competition, our thread pool was designed based on relatively simple Juliex. In the future, we plan to optimize it by referring to some mature thread pools such as Tokio, hoping to integrate it into master. You can continue to work on this project together in the TiDB Performance Challenge, which links to the project.
Finally thank Yilin teacher so strong is willing to take me to play, thanks to PingCAP let me rub eat rub drink hard pay 😄
Attached: Team Demo Show video (0:00-10:35)
pingcap.com/blog-cn/uni…
tidb.png
Markdown has selected 4778 word count 87 lines current line 87, current column 0 Article saved 11:27:01HTML 3029 word count 43 paragraphs sent to mobile phone excerpt collection