Doubt appear

I’ve written about zombie and orphan processes before, and it’s better to understand zombie processes first.

Redis can persist data through RDB. Using the BGsave command forks a child process, and when the persistence ends, the child process exits. After exiting, the parent needs to “collect” the corpse to avoid becoming a zombie process, which requires the use of the wait system call.

However, in C library, the wait function is blocked. If you use wait directly, the parent process will wait for the child process to finish persisting and then recycle. Redis is a single-thread process, waiting means blocking.

Guess and verify

Thought of a solution, the use of signal asynchronous notification.

The parent registers a signal handler (SIGCHLD), and the child sends a signal to the parent when it finishes persisting and exits.

The parent process continues its business until it receives the signal. After receiving the signal, the parent process executes the SIGCHLD (previously registered signal) handler function. In the signal handler function, the parent process executes the wait function to reclaim the child process and prevent the zombie process from appearing.

In Linux, the child itself sends a SIGCHLD signal to the parent when it exits. If the parent process does not register the signal handler, the signal is ignored.

With this in mind, use Strace to verify if Redis is the solution.

$strace -f -p 16809 clone (); $strace -f -p 16809 clone (); $strace -f -p 16809 clone (); [pid 16809] clone(strace: Process 17626 attached child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, Child_tidptr = 0x7f6b5AB89250) = 40 [PID 17626] set_robust_list(0x7F6b5AB89260, 24) =0 Exit_group (0) =? [pid 17626] exit_group(0) =? [pid 16809] -- SIGCHLD {si_SIGno =SIGCHLD, si_code=CLD_EXITED, si_pid=40, si_uid=999, si_status=0, si_utime=0, si_stime=0} --- ...... The wait function then executes the wait4 system call, WIFEXITED(s) &&wexitStatus (s) == 0}], WNOHANG, NULL) = 40 # Main process continues into its own main loop [pid 16809] getPID () = 1 [pid 16877] <... futex resumed> ) = 0 [pid 16809] getpid() = 1Copy the code

At this point, I thought the Redis strategy of reclaiming the child process was as I suspected it would be, and since the system call showed roughly the same information as the inference, I was going to pull down the source code (version 6.2.3), but after looking at the source code I was still too green.

There is no SIGCHLD signal handler in the code, but there is a lot of other signal processing.

After searching for wait functions, we did not find the standard WAIT function in C library.

Wait () = waitpid() = waitpid() = waitpid() = waitpid() = waitpid()

Consult the information for the use of the waitPID function:

pid_t waitpid(pid_t pid,int *status,int options)
Copy the code

This method takes three parameters, and only status has two more parameters than wait, one pid and one options.

Pid has four main options:

  • If the PID is greater than 0, the process whose PID is the waiting process exits.
  • Equal to 0, waits for any child processes in the same process group;
  • If it is -1, wait for any child process to exit.
  • < -1, waits for all child processes whose process group identifier is equal to the absolute value of pid.

Options can be bitwise or operated. there are three flags:

  • With WNOHANG, if the state of the specified child process is unchanged, the function does not wait for blocking and returns directly.
  • Using WCONTINUED, return status information for stopped child processes that resume execution upon receiving a SIGCONT signal.
  • Using WUNTRACED, in addition to returning information to terminate child processes, but also to return child processes stopped by signals;

Status is used to pass in a pointer and cause the user to get the process status.

The practice of Redis

Redis uses this function in the following way:

waitpid(-1, &statloc, WNOHANG)
Copy the code

AeTimeEvent registers serverCron in its control loop aeEventLoop, which contains a check on the child process. It first checks whether the process has a child process, and if so, it calls this function to check whether the child process exits.

Part of the source code is as follows:

// Initialization process void initServer(void) {...... // Register the timer callback function, Join serverCron if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) { serverPanic("Can't create event loop timers."); exit(1); } } int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { ...... / / if there are the child into the branch of the if the if (hasActiveChildProcess () | | ldbPendingChildren ()) {run_with_period (1000) receiveChildInfo (); // Check if the child exits checkChildrenDone(); . }... } // Check whether there are child processes, if there are, return true int hasActiveChildProcess() {return server.child_pid! = 1; Void checkChildrenDone(void) {int statloc = 0; pid_t pid; If ((pid = waitpid(-1, &statloc, WNOHANG))! = 0) {// Other follow-up procedures....Copy the code

To verify again

Now that we can loop, let’s do something with the code and use the Strace tool to verify that a wait system call will occur:

The method is to extend the child’s running time (because in real life, the neutron process usually finishes so quickly that the parent can’t poll for the wait system call), and use Strace to check whether the parent is cyclic executing the wait system call:

The simplest and most crude way is to add sleep to the child stream after fork and suspend it for a while.

int redisFork(int purpose) { ... If ((childpid = fork ()) = = 0) {/ * child * / sleep (100); Server. in_fork_child = purpose; setOOMScoreAdj(CONFIG_OOM_BGCHILD); setupChildSignalHandlers(); closeChildUnusedResourceAfterFork(); } else {/* Parent process */.....Copy the code

Compile and run Redis, bgsave command;

$ cd src/
$ make
$ ./redis-server XXXXredis.conf
$ ./redis-cli -h localhost -p 6379
$ localhost> bgsave
Copy the code

Viewing system calls:

[pid 30708] clone(strace: Process 30834 attached child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fef4bf09250) = 30834 ....... [PID 30834] nanosleep({tv_sec=100, tv_nsec=0}, < Unfinished... [pid 30708] write(8, "+Background saving started\r\n", 28) = 28 [pid 30708] epoll_wait(5, [], 10128, 95) = 0... [pid 30708] <... Futex resumed>) = 1 # You can see that in the event loop you have wait system call [PID 30708] wait4(-1, < Unfinished... >... [pid 30708] close(11) = 0 [pid 30708] wait4(-1, 0x7ffd5793a6dc, WNOHANG, NULL) = 0 [pid 30708] epoll_wait(5, [], 10128, 100) = 0 [pid 30708] getpid() = 30708 [pid 30708] open("/proc/30708/stat", O_RDONLY) = 11 [pid 30708] read(11, "30708 (redis-server) R 19236 307"... , 4096) = 329 [pid 30708] close(11) = 0 [pid 30708] wait4(-1, 0x7ffd5793a6dc, WNOHANG, NULL) = 0 [pid 30708] epoll_wait(5, [], 10128, 100) = 0 [pid 30708] getpid() = 30708 [pid 30708] open("/proc/30708/stat", O_RDONLY) = 11 [pid 30708] read(11, "30708 (redis-server) R 19236 307"... , 4096) = 329 [pid 30708] close(11) = 0 [pid 30708] wait4(-1, 0x7ffd5793a6dc, WNOHANG, NULL) = 0 [pid 30708] epoll_wait(5, [], 10128, 100) = 0 [pid 30708] getpid() = 30708 [pid 30708] open("/proc/30708/stat", O_RDONLY) = 11 [pid 30708] read(11, "30708 (redis-server) R 19236 307"... , 4096) = 329 [pid 30708] close(11) = 0 [pid 30708] wait4(-1, 0x7ffd5793a6dc, WNOHANG, NULL) = 0 [pid 30708] epoll_wait(5, [], 10128, 100) = 0 ........Copy the code

In this case, we can see that the parent process will add a wait system call to the control loop after the child process appears. If the child process terminates, the wait process will be canceled by the loop to avoid zombie process.

summary

This article starts from the problem eliciting, records the process from the emergence of doubt to the idea and then to the solution. In general, the Redis collection subprocess does not use signal processing, but rather uses non-blocking busy polling. The general process is shown in the figure below:

Reference

  • Time Loops in Redis draP. Me/Redis-Event…
  • Linux/UNIX System Programming Manual chapter 26: Monitoring child processes
  • Redis – 6.2.3 source code