Node.js in production – Open source monitoring tool

What do you think is the most important feature that Node.js applications can have? Fancy full-text fuzzy match search, or live chat with sockets? Can you tell me what are the most advanced, amazing, and fascinating features that can be added to node.js apps?

You want to know mine? High performance and uninterrupted service. High-performance applications need to do the following three things:

1. Minimum downtime;
2. Predictable utilization of resources;
3. Scale effectively according to the load

In Part 1, Key Node.js metrics to monitor, we discussed the key Node.js metrics you should monitor to understand the health of your application. I also explained node.js errors you should avoid, such as blocking threads and causing memory leaks, and some neat tricks you can use to improve your application’s performance, such as creating worker processes using cluster modules and running long-running tasks on separate threads from the main thread.

In this article, I describe how to monitor Node.js applications using five different open source tools. They may not be as comprehensive as Sematext or Datadog, but they are open source products that can be completely self-contained.

Appmetrics

The Node Application Metrics Monitoring Kanban displays performance data for running Node.js applications. This is a simple module that is applied and initialized at the top of the Node.js entry file. You can install from NPM by running the following command from your terminal.

$ npm install appmetrics-dash
Copy the code

Appmetrics provides a very easy-to-use Web dashboard. To get a dashboard of all the HTTP services created by your application, all you need to do is add the following code snippet to your app.js(or whatever entry file you call it) file.

// Before all other 'require' statements
require('appmetrics-dash').attach()
Copy the code

Then you’ll see a lot of useful metrics in the request path/Appmetrics-Dash.

• CPU Profiling
• HTTP incoming request
• HTTP throughput
• Average response time (top 5)
• CPU
• memory
• Heap (Heap)
• Event Loop Times
• The environment
• Other requests
• HTTP outbound request

This tool does more than show metrics. It allows you to generate Node.js reports and Heap Snapshots directly from the dashboard. In addition, you can use Flame Graphs, a cool open source tool.

Express Status Monitor

Express.js is currently the framework of choice for Node.js developers. The Express Status Monitor is a very simple stand-alone module that you can add to your Express application. It exposes a /status route that reports real-time server metrics with the help of socket. IO and chart.js.

Install from NPM.

$ npm install express-status-monitor
Copy the code

After installing this module, you need to add it before any other middleware or routing.

app.use(require('express-status-monitor') ())Copy the code

Then once you run your app, you can check your Node.js metrics via the /status route.

Prometheus

Unless you lived in primitive times, you must have heard of Prometheus. This is the most well-known open source monitoring tool available today. Prometheus is 100% open source and community-driven. All components comply with the Apache 2 License and can be downloaded from GitHub. It is one of the member projects managed by CNCF (Cloud Native Computing Foundation) and has graduated, and the same member projects include Kubernetes and Fluentd, etc.

To start monitoring with Prometheus, you need to download and install the latest version.

$ tar xvfz prometheus-\*.tar.gz
$ cd prometheus-\*
Copy the code

It is then started by running the executable, but before running this command, you need to create a Prometheus.yml file. It is a configuration file that configures which targets to monitor by fetching HTTP endpoint data on which targets.

# prometheus.yml
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 1s
    static_configs:
      - targets: ['127.0.0.1:3000']
        labels:
          service: 'test-prom'
          group: 'production'
Copy the code

Now you can use Prometheus.

$ ./prometheus --config.file=prometheus.yml
Copy the code

But, I’m lazy, and I love Docker. So what I did was run the official Prometheus Docker image and avoid all the hassle of downloading it.

Prometheus and Docker

First, go to the root directory of the Node.js application. Here, create a prometheus-data directory and put the Prometheus.yml file into it. After this, run the Prometheus Docker container.

Get the official Prometheus Docker image and run it using the Docker run command.

$ docker run -d \
    --name prometheus \
    --network="host" \
    -v "$(pwd)/prometheus-data":/prometheus-data \
    prom/prometheus \
    --config.file=/prometheus-data/prometheus.yml
Copy the code

I chose to run the container with -network = “host” to make the Prometheus container accessible through the native localhost address, and in doing so, the Node.js application’s through the native HTTP port. Otherwise, if you have Prometheus and Node.js running in separate containers, you need to set up a network between them so that they are only accessible to each other.

The -v option is used to map the Prometry-data directory from the host to a directory of the same name in the container.

After the Prometheus container is running, the configured code needs to be added to the Node.js application to expose a monitoring data interface. You first need to install the Prometheus client for Node.js from NPM.

$ npm install prom-client
Copy the code

Next, add relevant Prometheus related configuration code

// after all 'require' statements
const client = require('prom-client')
const collectDefaultMetrics = client.collectDefaultMetrics
collectDefaultMetrics({ timeout: 1000 })
app.get('/metrics', (req, res) => {
  res.set('Content-Type', client.register.contentType)
  res.end(client.register.metrics())
})
Copy the code

Next, you need to run the Node. Js applications, through http://localhost:9090/graph you can see Prometheus charts

Clinic.js

Clinic. Js contains three tools to help diagnose and pinpoint Node.js performance issues. It’s very simple to use. All you need to do is install the module from NPM and run it. It will generate reports for you, making troubleshooting easier.

Install clinic.js using the following command

$ npm install clinic
Copy the code

Once installed, you can choose the type of report to generate. There are three types of reports you can choose from.

• Doctor
  1. Metrics are collected by injecting probes
  2. Assess health and heuristics
  3. Provide repair recommendations
• Bubbleprof- a new and completely unique way to analyze Node.js code
  1. Collect metrics using Async_hooks
  2. Tracking delays between operations
  3. Create bubble chart
• Flame – Use Flame diagrams to reveal bottlenecks and thermal paths in your code
  1. Collect indicators by CPU sampling
  2. Track top of stack frequency
  3. Create a flame map

Let’s start by running Doctor and testing the Node.js application.

$ clinic doctor -- node app.js
Copy the code

While the program is running, run the load test using the pressure gauge tool.

$ loadtest -n 1000 -c 100 [http://localhost:3000/api](http://localhost:3000/api)
Copy the code

Once complete, stop the server and clinic.js Doctor will open the report you can view.

In the same way, you can run Bubbleprof or Flame and get a graphical report of the corresponding tool.

PM2

Running Node.js applications in production with PM2 makes it much easier. It is a process manager that lets you easily run applications in clustered mode. In layman’s terms, it will generate one process for each CPU core on your host.

Install the PM2 first

$ npm install pm2 -g
Copy the code

After installation, if your primary source file is app.js, generate the PM2 daemon by running this command on your terminal.

$ pm2 start app.js -i 0
Copy the code

-i 0 Indicates the number of flag instances. This will run the Node.js application in clustered mode, where the number 0 represents the number of CPU cores. You can manually enter any number you want, but it’s easier for PM2 to calculate the number of cores and automatically generate the corresponding number.

Viewing Node.js monitoring data using PM2 is also easy

$ pm2 monit
Copy the code

This command opens the dashboard in the terminal. From here, you can monitor processes, logs, loop latency, process memory, and CPU.

Use open source tools to wrap monitoring node.js scenarios

Performance metrics are critical to customer satisfaction. In this article, I’ve shown you how to add monitoring to node.js applications using five different open source tools. After learning about the key metrics node.js will monitor in Part 1 of this series, adding tools to monitor real-world applications was a natural learning curve. The final part of this series will cover node.js monitoring in production using Sematext.

If you want to see sample code, here’s a repo that contains all the sample code. You can also clone it and open it with any tool of your choice.

If you need more full stack observability for software, check out Sematext. We are pushing to open source our products and make an impact.

Node.js Open-source Monitoring Tools

The resources

• Prometheus Operation Guide
• Prometheus introduction and Practice