Some big shots in Taobao have done a test, on a machine with 24 GB memory, the maximum number of concurrent connections of Nginx reached 2 million. After hearing this conclusion, students are not impressed by the ultra high performance of Nginx, what is the internal architecture design? This article will introduce you to the architecture design of Nginx.
This article mainly refers to the Nginx article written by taobao technical team, and will be shared from the following aspects:
- Nginx process model
- Nginx event model
Nginx process model
Nginx starts and runs in multi-process mode by default. Of course, Nginx also supports multi-thread mode, but our mainstream mode is still multi-process mode, which is also the default mode of Nginx. Nginx’s multi-process approach has many benefits, contrary to what most of our students believe. Most of you will say that processes are heavier than threads, that processes cost more resources and so on, and there’s a whole bunch of arguments that threads are better than processes. If you think exactly this way, you have to expand your cognition. For example, our famous Redis and Oracle all have the concept of multi-process, and these software all have super high performance without exception. So, today we will mainly study Nginx multi-process mode.
What does Nginx process model look like?
After Nginx starts, there will be one master process and multiple worker processes. The master process is mainly used to manage worker processes, including: receiving signals from the operating system, sending signals to all worker processes, monitoring the running status of worker processes, and automatically restarting new worker processes when worker processes exit (under abnormal circumstances). Basic network events are handled in the worker process. Multiple worker processes are peer. They compete equally for requests from clients and are independent of each other. A request can only be processed in one worker process, and a worker process cannot process requests from other worker processes. The number of worker processes can be set, which is generally consistent with the number of MACHINE CPU cores. The reason for this is inseparable from Nginx’s process model and event processing model.
Nginx’s multi-process model gives it one big advantage: elegant restarts and uninterrupted service. How does it do that? As we know from the above, the master manages the worker process, so we only need to communicate with the master process. The master process receives signals from the operating system and does different things based on those signals. So to control Nginx, we just need to send a signal to the master process through the kill command provided by the operating system. For example, restart Nginx with kill -hup PID. What does the master process do after receiving the HUP signal? First of all, after receiving the signal, the master process will reload the configuration file first, and then start the new worker process, and send a signal to all the old worker processes to tell them that they can retire with honor. The new worker starts to receive new requests after starting, while the old worker stops receiving new requests after receiving the signal from the master, and exits after all unprocessed requests in the current process are processed.
Now that we know what Nginx is doing internally, how does the worker process handle requests on the client side? As we mentioned earlier, worker processes are equal, and each process has the same opportunity to process requests. When we provide HTTP service for port 80, a connection request comes in, and each process can potentially handle the connection. How does that happen? First, each worker process is forked from the master process. In the master process, the ServerSocket is created first, the monitoring is enabled, and then multiple worker processes are forked. All worker processes will hold the same socket handle as the master process, and the handle will become readable upon the arrival of a new connection. To ensure that only one process processes the connection, all worker processes will rob accept_mutex before registering the handle read event, and the process that obtains the mutex will register the handle read event. Call Accept in the read event to accept the connection. When a worker process accepts this connection, it generates a new socket, through which it begins to read requests, parse requests, process requests, generate data, and then return it to the client. Finally, a complete request ends in this way. We can see that a request is handled entirely by the worker process, and only within one worker process.
So what are the benefits of Nginx adopting this process model? There are so many, just to highlight a few. First of all, for each worker process, an independent process does not need to lock, so the cost of locking is saved. Meanwhile, programming and problem finding are much more convenient. Secondly, independent processes can not affect each other. After one process exits, other processes are still working and the service will not be interrupted. The master process will soon start a new worker process. Of course, the abnormal exit of worker process must be a bug in the program. Abnormal exit will lead to the failure of all requests on the current worker, but it will not affect all requests, so the risk is reduced. Of course, there are many benefits, students can slowly experience.
Nginx event model
Nginx takes the NIO approach to handling requests, which is the root cause of Nginx’s ability to handle thousands of requests simultaneously. If you think about the IO model of earlier versions of Tomcat (older versions already support NIO), each request has an exclusive worker thread, and when the number of concurrent requests reaches thousands, there are thousands of threads processing requests at the same time. This is a big challenge for the operating system. Threads take up a lot of memory, and context switching costs a lot of CPU, which makes no sense.
Why does Nginx use NIO? What’s up with NIO? The nature of IO is read and write events, and when read and write events are not ready, they must not operate, if you do not call in a non-blocking way, then you have to block the call, the event is not ready, then you have to wait, when the event is ready, the worker thread to continue. The blocking call will enter the kernel and wait, and the CPU will be let out for others to use. The worker thread can only sleep and wait foolishly, which is obviously not suitable for the single-threaded worker. When there are more network events, everyone is waiting, and the CPU is idle and no one is using it, so the CPU utilization rate naturally cannot go up, let alone the high concurrency. Therefore, for high performance, NIO must be used. About NIO, we here is a brush over, want to learn in detail students can go to see my NIO source analysis article.
Next, let’s summarize Nginx’s event handling model with a piece of pseudocode.
Date now; // Indicates the current time
while (true) {
// Process all tasks in the task queue
for task in tasks:
task.handler();
flushTime(now); // Refresh the current time
timeout = initValue; // The timeout period
// waitTasks can be interpreted as all valid tasks registered in epoll
for task in waitTasks:
// The first task in the list has the shortest timeout. If it has already timed out, the timeout handler is called
if (task.time <= now) {
task.timeoutHandler();
} else {
// Update the timeout period
timeout = t.time - now;
break;
}
// Get ready events via epoll
nevents = epoll(events, timeout);
// Process events one by one
for i in nevents:
Task task;
/ / read event
if (events[i].type == READ) {
task.handler = readHandler;
} else { / / write events
task.handler = writeHandler;
}
// prevent a special queue execution
tasks(task);
}
Copy the codeWell, that’s the end of the article. Hope Nginx process model and event model design ideas, can be helpful to friends later system design. Finally, like my article students, welcome to pay attention to my public number “small jin daemon thread”, not miss any valuable dry goods.
