Origin of C10K problem

With the popularity of the Internet, the user population of applications grows exponentially, and server performance problems arise at this time. The original server was based on the process/thread model. When a new TCP connection arrives, a process needs to be assigned. If you have C10K, you need to create 1W processes, which can not be supported by a single machine. Therefore, how to break through the single-machine performance is a problem that high-performance network programming must face, and these limitations and problems are collectively referred to as C10K problems, which were first summarized and summarized by Dan Kegel, and he also systematically analyzed and proposed solutions.

The nature of the C10K problem

The C10K problem is essentially an operating system problem. For Web 1.0/2.0 operating systems, the traditional synchronous blocking I/O model was requests per second. When many processes or threads are created, data is copied frequently (caching I/O, kernel copying data into user process space, blocking, process/thread context switching costs a lot, and the operating system crashes. This is the essence of C10K.

As you can see, the key to solving the C10K problem is to minimize these CPU consumption.

Solution to the C10K problem

From the perspective of network programming technology, the main ideas are as follows:

  1. Each connection is assigned a separate thread/process
  2. The same thread/process processes multiple connections simultaneously

Each process/thread handles one connection

This idea is the most direct, but the application process/thread needs system resources, and the system needs to manage these processes/threads, so it will occupy too much resources, poor scalability

Processing multiple connections simultaneously per process/thread (I/O multiplexing)

  1. Select: use the fd_set structure to tell the kernel to monitor which file handles are available at the same time and to check if any file handles are ready or timed out. This method has the following disadvantages: the number of file handles is limited, the throughput of checking one by one is low, and fD_set must be initialized repeatedly for each call.
  2. Poll method: This method mainly solves two disadvantages of the SELECT method, file handle upper limit problem (linked list storage) and repeated initialization problem (different field annotation events and events), but the problem of checking whether file handle is ready one by one is still not solved.
  3. Epoll: This is the killer of a C10K problem. It does not poll to see if all file handles are ready. Epoll is only interested in file handles that have changed. The way it works is that the file descriptor FD is registered with epoll_ctl as an “event” ready notification. Once the FD is ready, the kernel uses a callback-like callback mechanism to activate the FD, and epoll_wait is notified and notified to the application. Moreover, ePoll uses a single file descriptor to manage multiple descriptors, storing the user process’s file descriptor events into an event table in the kernel so that the data only needs to be copied from the kernel cache space to the user process address space once. Moreover, epoll realizes event-ready message delivery by sharing memory between kernel and user space, which is very efficient. But epoll is system dependent (Linux).
  4. Asynchronous I/O and Windows, which is well supported on Windows, will not be covered here.