The death of a Node process

Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money

Hello, I’m Shanyue.

As human beings die, so does a Node process, and there is always some inevitability to it. From this article we will see how a process can die gracefully.

A Node process, in addition to providing HTTP services, must also run scripts. Running a script to pull configuration, process data, and schedule tasks is common. Scripts can be seen in some important processes:

1. CI, for testing, quality assurance, deployment, etc
2. Cron is used to schedule tasks
3. Docker, used to build images

If scripts fail to detect problems in these critical processes, more insidious problems can arise. Service exceptions are intolerable if you cannot catch HTTP service problems when they occur.

Recently, observing project image builds, I have occasionally noticed that one or two images have been built successfully, but the container does not run. The reason is that a Node process dies without being aware of it.

Exit Code

What is exit code?

Exit code represents the return code of a process and is triggered by the system call exit_group.

On POSIX, 0 indicates a normal return code, and 1-255 indicates an exception return code. In practice, the error code that is actively thrown is 1. Call API process.exitCode = 1 in the Node application to indicate that the process is aborted and exits due to an unexpected exception.

Here is a Appendix E. Exit Codes With Special Meanings.

Exception codes are common in operating systems. Here is an exception about the CAT process and its exit code, using Strace to trace system calls.

$ cat a
cat: a: No such file or directory

Use strace to view system calls to cat
# -e displays only write and exit_group system calls
$ strace -e write,exit_group cat a
write(2, "cat: ", 5cat: )                    = 5
write(2, "a", 1a)                        = 1
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
exit_group(1)                           = ?
+++ exited with 1 +++
Copy the code

As you can see from the last line displayed by the Strace trace process, its exit code is 1 and it prints the error message to stderr (whose FD is 2)

How do I view exit Code

The exit code of the process can be determined from Strace, but it is not convenient and redundant, let alone unable to locate the exception code in the first time.

There’s an easier way to do this, throughecho $?To confirm the return code

$ cat a
cat: a: No such file or directory

$ echo $?
1
Copy the code

$ node -e "preocess.exit(52)"
$ echo $?
52
Copy the code

Where is the pain that is not felt:throw new Error 与 Promise.rejectThe difference between

Here are two pieces of code. The first throws an exception, and the second promises. Reject.

function error () {
  throw new Error('hello, error')
}

error()

// Output:

// /Users/shanyue/Documents/note/demo.js:2
// throw new Error('hello, world')
/ / ^
// 
// Error: hello, world
// at error (/Users/shanyue/Documents/note/demo.js:2:9)
Copy the code

async function error () {
  return new Error('hello, error')
}

error()

// Output:

// (node:60356) UnhandledPromiseRejectionWarning: Error: hello, world
// at error (/Users/shanyue/Documents/note/demo.js:2:9)
// at Object.
      
        (/Users/shanyue/Documents/note/demo.js:5:1)
      
// at Module._compile (internal/modules/cjs/loader.js:701:30)
// at Object.Module._extensions.. js (internal/modules/cjs/loader.js:712:10)
Copy the code

Use echo $? For both test cases. Looking at the exit code, we see that the exit code for throw new Error() is 1, while promise.reject () is 0.

From an operating system perspective,exit codeA value of 0 indicates that the process runs successfully and exits, however, even if it doesPromise.rejectAnd the operating system will treat it as successful.

This has security implications when executing scripts in Dockerfile and CI.

Dockerfile hidden in Node image construction

When building an image or CI using Dockerfile, the build will fail if the process returns a non-zero return code.

This is an easy to understand mirror image of the promise.reject () problem that we see as the problem.

FROM node:12-alpine

RUN node -e "Promise.reject('hello, world')"
Copy the code

Build the mirroring process is as follows, the last two lines hint mirror build successful: even in the build process to print out the unhandledPromiseRejection information, but the mirror still build is successful.

$docker build -t demo. Sending build context to docker daemon 33.28kB Step 1/2: FROM node:12-alpine ---> 18f4bc975732 Step 2/2 : RUN node -e"Promise.reject('hello, world')"
 ---> Running in 79a6d53c5aa6
(node:1) UnhandledPromiseRejectionWarning: hello, world
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exitcode. Removing intermediate container 79a6d53c5aa6 ---> 09f07eb993fe Successfully built 09f07eb993fe Successfully tagged  demo:latestCopy the code

However, if you are in a Node 15 image, the image will fail to build, for reasons discussed below.

FROM node:15-alpine

RUN node -e "Promise.reject('hello, world')"
Copy the code

$docker build -t demo. Sending build context to Docker daemon 2.048kB Step 1/2: FROM node:15-alpine ---> 8bf655e9f9b2 Step 2/2 : RUN node -e"Promise.reject('hello, world')"
 ---> Running in 4573ed5d5b08
node:internal/process/promises:245
          triggerUncaughtException(err, true /* fromPromise */);
          ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "hello, world".] {
  code: 'ERR_UNHANDLED_REJECTION'
}
The command '/bin/sh -c node -e "Promise.reject('hello, world')" returned a non-zero code: 1
Copy the code

Promise.reject scripting solution

Problems that can be found at compile time should never be left at run time. Therefore, if you need to execute the Node script in a build image or CI, you need to manually specify process.exitCode = 1 for exception handling to expose the problem in advance

runScript().catch(() = > {
  process.exitCode = 1
})
Copy the code

Node also has suggestions for exception resolution when building the image:

(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise Rejection, use the CLI flag –unhandled-rejections=strict (see nodejs.org/api/cli.htm…) . (rejection id: 1)

As prompted, –unhandled-rejections=strict will set the promise. reject exit code to 1 and fix the Promise exit code in future Node releases.

The next version of Node 15.0 has been removedunhandled-rejectionsTreated as an exception and returns a non-zero exit code.

$ node --unhandled-rejections=strict error.js 
Copy the code

Signal

Externally, how do you kill a process? A: kill $pid

More precisely, a kill command is used to send a signal to a process, not to kill it. Probably because the number of people killing the process becomes kill.

The kill utility sends a signal to the processes specified by the pid operands.

Each signal is represented by a number, and the list of signals can be printed by kill -L

# list all signals
$ kill-l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV  12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR 31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 38) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8 43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13 48) SIGRTMIN+14  49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12 53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 58) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2 63) SIGRTMAX-1 64) SIGRTMAXCopy the code

Among these signals, the most contact with the terminal process is as follows: SIGTERM is the default signal to send kill, and SIGKILL is the signal to forcibly kill the process

In Node, process.on can listen for a grabable exit signal without exiting. The following example listens for SIGINT and SIGTERM signals. SIGKILL cannot be listened on and setTimeout guarantees that the program will not exit

console.log(`Pid: ${process.pid}`)

process.on('SIGINT'.() = > console.log('Received: SIGINT'))
// process.on('SIGKILL', () => console.log('Received: SIGKILL'))
process.on('SIGTERM'.() = > console.log('Received: SIGTERM'))

setTimeout(() = > {}, 1000000)
Copy the code

Run the script and start the process. The PID of the process is displayed. Run the kill -2 97864 command to send a signal

$ node signal.js
Pid: 97864
Received: SIGTERM
Received: SIGTERM
Received: SIGTERM
Received: SIGINT
Received: SIGINT
Received: SIGINT
Copy the code

Graceful handling of exit from container

When the expired Pod needs to be disabled during the K8S container service upgrade, a SIGTERM signal is sent to the main process of the container (PID 1) and 30s are reserved for cleaning. If the container does not exit after 30 seconds, K8S continues to send a SIGKILL signal. If the ancient emperor white damask death, teach you decent.

In fact, not only containers, CI scripts also gracefully handle process exit.

When you receive a SIGTERM/SIGINT signal, set aside one minute to do any unfinished work.

async function gracefulClose(signal) {
  await new Promise(resolve= > {
    setTimout(resolve, 60000)
  })

  process.exit()
}

process.on('SIGINT',  gracefulClose)
process.on('SIGTERM', gracefulClose)
Copy the code

This is the right thing to do, but what if it’s a service with a steady stream of requests coming in? Let the service shut down, calling server.close() to terminate the service

const server = http.createServer(handler)

function gracefulClose(signal) {
  server.close(() = > {
    process.exit()
  })
}

process.on('SIGINT',  gracefulClose)
process.on('SIGTERM', gracefulClose)
Copy the code

conclusion

1. If the exit code is not 0, the system considers that the process failed to execute
2. throughecho $?You can view the exit code of the process on the terminal
3. In the NodePromise.reject, exit code is 0
4. Yes in Nodeprocess.exitCode = 1Explicitly set exit code
5. It can pass in Node12+node --unhandled-rejections=strict error.jsExecute the script, seePromise.reject 的 exit codeTo 1, this problem is fixed in Node15
6. The Node process exits gracefully
7. K8s will send a SIGTERM signal when closing POD and leave 30 seconds to process the unfinished business. If POD does not exit normally, it will send SIGKILL signal after 30 seconds