K8s-graceful-shutdown: this library provides resources for implementing graceful shutdown in Node.js App using Kubernetes.

Problem description

When running microservices in Kubernetes. We need to deal with the termination signal from Kubernetes. The right way to do this is:

  1. Listening to theSIGINT.SIGTERM
  2. After receiving the signal, put the service in unhealthy mode (/healthThe route should return a status code4xx.5xx)
  3. Add a grace period before closing to allow Kubernetes to remove your application from the load balancer
  4. Close the server and all open connections
  5. Shut down

The library makes the above process easy. Simply register your Graceful Shutdown hook and add a grace period.

Please note that your grace period must be less than the grace period defined in Kubernetes!

An example using the Express framework

For example, using the Express framework:

import { Response, Request } from 'express'
import express from 'express'
import { addGracefulShutdownHook, getHealthHandler, shutdown } from '@neurocode.io/k8s-graceful-shutdown'

const app = express()
app.disable('x-powered-by')
const port = process.env.PORT || 3000
const server = app.listen(port, () = > console.log(`App is running on http://localhost:${port}`))

// Patch the NodeJS server shutdown function so that it has the correct shutdown function, as you might expect to close keep-alive connections for you!
/ / here to read more information at https://github.com/nodejs/node/issues/2642
server.close = shutdown(server)

const healthy = (req: Request, res: Response) = > {
  res.send('everything is great')}const notHealthy = (req: Request, res: Response) = > {
  res.status(503).send('oh no, something bad happened! ')}const healthTest = async() = > {// This is optional
  // You can use it for health checks
  return true
}

const healthCheck = getHealthHandler({ healthy, notHealthy, test: healthTest })
app.get('/health', healthCheck)

const sleep = (time: number) = > new Promise((resolve) = > setTimeout(resolve, time))

const asyncOperation = async () => sleep(3000).then(() = > console.log('Async op done'))

const closeServers = async() = > {await asyncOperation() // can be any asynchronous operation, such as mongo db shutdown, or sending slack messages;)
  server.close()
}

const gracePeriodSec = 5*1000
addGracefulShutdownHook(gracePeriodSec, closeServers)

server.addListener('close'.() = > console.log('shutdown after graceful period'))
Copy the code

  • The simple application shown above adds a graceful 5-second shutdown cycle, after which the hook (responsible for shutting down the server with the help of the shutdown function) is fired. When a SIGINT or SIGTERM signal is sent, the user sees a 5-second grace period followed by a 3-second wait for an asynchronous operation before a “shutdown after graceful period” message is displayed indicating that the server is shutdown.

  • The application also shows the functionality of “getHealthHandler.” When localhost:3000/health is requested, healthTest will return true and display the ‘everything is great’ message indicating that the health check is normal. The user can change healthTest to return false and see the message change to ‘Oh no, something bad Happened! ‘It shows an unhealthy state.

If you use the Koa framework, check out the demos/ folders. We have a Koa example that does something similar to the above application. Koa applications use getHealthContextHandler with FN (CTX) support for Health and notHealthy handlers, Instead of the Health and notHealthy handlers as the FN (REQ, RES) getHealthHandler.

How does it work?

Examples of how a normally closed workflow works:

  1. Kubernetes 向 PodsendSIGTERMThe signal. Manual to narrowPodOr automatically shrink during rolling deploymentPodThat’s what happens
  2. The library to receiveSIGTERMSignal and call yournotHealthyHandler. Your handler should return400 或 500 的 httpStatus code (throw error?) , which indicates that thepodNo more traffic is received. Note that this step is optional (check next)
  3. The library waits for the specifiedgrace timeTo initiate the closure of the application. The grace period shall be in5 到 20Seconds.kubernetesThe endpoint controller requires grace before it can be removed from the list of valid endpointsPod, and then removed from the servicePod(fromiptablesObtained from all nodespod 的 ipAddress).
  4. Kubernetes 从 ServiceRemove thePod
  5. The library calls all of your registered shutdown hooks
  6. After the configured grace period, the application will shut down correctly using our shutdown mechanism, as you mayExpect default work, but inNodeJS http server.express 和 Koanot