Nodejs Memory Monitoring 1- Monitoring scheme

This is the first article in the NodeJS Memory Monitoring series, which focuses on the main nodeJS monitoring metrics and some of the leading monitoring solutions on the market. (This article is not original, some knowledge points of the article are only used as notes)

introduce

Node.js is becoming more and more important on the front end, but it’s also making its presence felt on the back end. However, only some demos can only cover some scenarios of enterprise applications, and the stable operation of services in the production environment depends on sound monitoring, logging, flow and other infrastructure.

When Node is used as the server language in production environment, excessive concurrency or code problems causing OOM (out of memory) or CPU full load are common problems in the server. In this case, it is easy to find problems by monitoring THE CPU and memory, combined with logs and Release.

Key indicators

The indicators monitored by Node.js measure the load, processing capacity, and health status of services. The following describes the definition of performance indicators.

memory

It is used to determine the server usage and whether there is a memory leak. Basic memory metrics can be obtained using the process.MemoryUsage () method.

  1. Resident Set Size Specifies the physical memory used by the RSS process (including the memory occupied by the shared library).
  2. External external memory: memory usage of C++ objects bound to Javascript, managed by V8
  3. HeapTotal Total heap memory size applied by V8
  4. HeapUsed V8 heap memory usage

Heap spatial distribution

  1. New space used: The size of the memory used by the new generation space
  2. The old space used is the size of the memory used in the old generation
  3. Map space Used hides the memory usage of the class space
  4. Code space used indicates the memory usage of code space
  5. Large object space used Indicates the size of the memory used by large objects

http

  1. Apdex Performance index
  2. QPS
  3. The response time
  4. Status code & error rate
  5. Byte size (throughput)

CPU

  1. usage
  2. Load (1, 5, 15) Average load in minutes (system or process?)

The GC statistics

  1. GcTime: The time at which GC occurs. The accuracy may need to be improved to ms level instead of second level
  2. GcType: A type of GC that can be ignored the scavenge avenge and be low in magnitude and optimizability
  3. GcPause: duration of GC outage, which needs to be sorted and collected according to different GC types, especially old markSweepCompact data
  4. SizeBefore: specifies the memory sizeBefore GC. Bytes
  5. SizeAfter: Specifies the size of the memory after GC
  6. HolesBefore: Specifies the size of the memory space before GC. Bytes
  7. HolesAfter: specifies the size of the memory space after GC. Bytes
  8. Allocated: Allocated bytes in GC
  9. Promoted: During GC, object promotion quantity bytes
  10. AllocationThroughput: Between GC, new generation memory application rate bytes/ms
  11. PromotionRatio: The percentage of memory promoted from generation to generation in the current GC
  12. IncrementalWalltime: indicates the incremental marking time ms
  13. CompactionSpeed: Specifies the memory Compacting speed bytes/ms

Event loop delay

Process lifetime

  1. Process life cycle, running time.
  2. Process Restart times

Open source solutions

swagger-stats

Support for monitoring koA and Express framework services, metrics display kanban is user-friendly, and support for Prometheus.

egg-prometheus

Egg.js monitoring plug-in, collect less indicators, only HTTP and RPC performance indicators, suitable for secondary development.

Pandora.js

Ali open source Node.js monitoring tool, used in egg.js for a period of time, but the service is not stable.

nest-status-monitor

Simple monitoring service for Nest.js.

Sandbox

Ali Node.js application monitors the Sandbox deployment configuration of Docker Compose. However, it seems that no one has maintained it for a long time, which should be the end of KPI project

PM2

Common Node.js process daemon with monitoring metrics. Monitoring metrics costs more and does not support local deployment. Data sensitive can be passed.

Express Status Monitor

Monitor plug-in for Express framework.

prom-client

Prometheus node. js client that creates metric types for generating standard metrics. It’s very powerful with Prometheus, but you could leave Prometheus and just collect data.

node-prometheus-gc-stats

Based on PROM -client, used to collect GC information.

appmetrics

Node.js application metrics, Prometheus support, and Appmetrics-Dash support kanban.

appmetrics-prometheus-client

Collect metrics sets and expose metrics for capture by the Prometheus server.

easy-monitor

Lightweight real-time Node.js kernel performance monitoring analysis tool. Pure open source project, if the requirements are not high or very recommended on this open source project.

alinode

The first choice of open source solutions! , supports local deployment and is a powerful free trial, the first choice for small and medium-sized companies. However, it is strictly semi-open source. The monitoring data will be uploaded to the background of Ali Cloud for cleaning, and those sensitive to data may not be able to try it out.

sematext

A foreign provider of monitoring infrastructure.

The blogger’s current Node monitoring scheme

Since I built my own K8S cluster and migrated most of my projects to K8S, I naturally chose the open source solution of Prometheus + Granafa + PROM-Cilent for monitoring node projects.

Based on Prometheus, the open source nodeJs dashboard was used. Additional monitoring of HTTP data is added to this foundation.

In part 2, I will show how to monitor nodeJS projects based on Prometheus + Granafa + PROM-cilent.

Nodejs Memory monitoring 2- Prometheus + Grafana monitoring implementation