Welcome to Tencent Cloud + community, get more Tencent mass technology practice dry goods oh ~
This article was published by Xiao Yi in cloud + Community
introduce
Netdata provides accurate performance monitoring through an extensible Web dashboard that displays processes and services on Linux systems. It monitors metrics related to CPU, memory, disk, network, process, and so on.
Netdata requires no additional configuration once installed, but provides important customization. The efficiency and speed of the application is designed to be comparable to native console management tools such as VMstat, iostat, and HTOP.
The steps in this tutorial cover everything you need to successfully set up an Ubuntu 16.04 server running Netdata using its built-in Web server or, optionally, using Nginx.
To prepare
To follow this tutorial, you will need:
- An Ubuntu 16.04 server, including non-root users with sudo permissions.
- Install Git on the server.
- (Optional) Install Nginx on the server. This is not mandatory, but allows for more functionality and security than Netdata’s built-in Web server.
Step 1 – Install Netdata
Before we install anything, make sure the system package index is up to date.
$ sudo apt-get update
Copy the codeNext, install Netdata’s dependencies, which include GCC (a C compiler), the GNU Autoconf tool, GUID management, and a zip library for Netdata’s internal Web server.
$ sudo apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make autoconf autoconf-archive autogen automake pkg-config curl
Copy the codeThe next set of packages is optional, but Netdata recommends, and includes Python, some Python packages, and Node.js. The stable version of Node.js bundled with the system package manager is suitable for Netdata requirements. Next, install them.
$ sudo apt-get install python python-yaml python-mysqldb python-psycopg2 nodejs lm-sensors netcat
Copy the codeTo install Netdata itself, we must use the project’s GitHub repository. Clone the Netdata repository into your home directory.
$ git clone https://github.com/firehol/netdata.git --depth=1 ~/netdata
Copy the codeMove to the newly cloned directory.
$ cd ~/netdata
Copy the codeNow build and install the application using the netdata-installer.sh shell script from this directory. Make sure sudo is attached here, otherwise the way Netdata collects system data (through the data collector) won’t work.
$ sudo ./netdata-installer.sh
Copy the codeThe initial output you’ll see contains information about where Netdata will store all of its components. You can read through this so that you are more familiar with how the program unfolds on the file system after installation.
Installer Output
. . .
It will be installed at these locations:
- the daemon at /usr/sbin/netdata
- config files at /etc/netdata
- web files at /usr/share/netdata
- plugins at /usr/libexec/netdata
- cache files at /var/cache/netdata
- db files at /var/lib/netdata
- log files at /var/log/netdata
- pid file at /var/run
. . .
Copy the codePress ENTER to continue the installation. A moment later, at the end of the output, you will see the following message:
Installer Output
. . .
^
|.-. .-. .-. .-. .-. . netdata .-. .-
| The '-' The '-' The '-' The '-' The '-' is installed and running now! -' '-' +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+---> enjoy real-time performance and health monitoring...Copy the codeNetdata is now successfully installed and running, and will run automatically when the server is restarted. If you have UFW enabled, you need to turn on port 19999 for Netdata’s internal Web server.
$ sudo ufw allow 19999/tcp
Copy the codeAt this point, you can view the default dashboard by visiting http:// your_server_IP: 19999/ in your favorite browser. You’ll see an overview of the system’s real-time metrics.
Step 2 – Configure Netdata memory usage
You can view the current Netdata configuration by visiting http://your_server_ip:19999/netdata.conf in the browser.
You will see here that all (or most) of the configurable options are commented out (that is, beginning with #). This is because Netdata is configured using a set of assumed defaults. Any disabled Settings use Netdata’s default values; If you uncomment a setting, the specified value overrides the default. This causes the configuration file to contain only what you modify.
Back to your server, the configuration file is/etc/netdata netdata. Conf. The option we will customize in this tutorial is the history parameter. It controls the size of the in-memory database used by Netdata.
Here, you need to determine how much RAM to provide for Netdata, or how long to keep the recorded chart data before losing it:
- 3600 seconds (chart data is retained for 1 hour) using 15 MB RAM
- 7200 seconds (chart data is retained for 2 hours) using 30 MB RAM
- 14400 seconds (chart data is retained for 4 hours) using 60 MB RAM
- 28800 seconds (chart data is retained for 8 hours) using 120 MB RAM
- 43,200 seconds (chart data is retained for 12 hours) using 180 MB RAM
- 86400 seconds (24 hours chart data retention) using 360 MB RAM
Keep in mind that the above estimates are based on the number of charts used by the inventory dashboard. Deleting or adding custom charts in the future will affect these estimates.
Use Nano or your favorite text editor to open Netdata’s main configuration file.
$ sudo nano /etc/netdata/netdata.conf
Copy the codeFind the History option in the [Global] section.
/etc/netdata/netdata.conf
. . .
[global]
# glibc malloc arena max for plugins = 1.# hostname = test-netdata
# history = 3600
# update every = 1.Copy the codeReplace the 3600 value with the one you decided on earlier. In this case, we are using 14400, which gives us four hours of data retention. Be sure to uncomment the line by dividing the # symbol so Netdata no longer ignores this option.
/etc/netdata/netdata.conf
. . .
[global]
# glibc malloc arena max for plugins = 1.# hostname = test-netdata
history = 14400
# update every = 1.Copy the codeSave and close the file after making this change. While this change may increase the amount of RAM that Netdata will use, the next change we’ll implement in the next step should significantly reduce it.
Step 3 – Enable kernel same-page merge
Even if Netdata runs directly and is used initially, we can make more changes to the way Netdata uses system resources, which will speed up and optimize its performance. We’ll start by enabling kernel same-page merge (KSM for short). Netdata developers estimate that this will reduce Netdata’s memory usage by 40-60%.
When enabled, the KSM daemon periodically scans memory for pages with the same content that can be replaced by a single write-protected page. In this context, it allows the same pages of memory to be shared between different processes or programs running on the system. This reduces unnecessary duplication of memory content creation.
To permanently enable this aspect of the Linux system kernel, use a text editor to open the /etc/rc.local file.
$ sudo nano /etc/rc.local
Copy the codeAfter starting all other normal Linux system services and processes, the /etc/rc.local file is executed or the control file is run. It is useful for specifying custom services – or, in our case, for enabling KSM while the server is running.
Add the two commands shown below to the end of the file, before the last exit 0 line, as follows:
/etc/rc.local
#! /bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
echo 1 > /sys/kernel/mm/ksm/run
echo 1000 > /sys/kernel/mm/ksm/sleep_millisecs
exit 0
Copy the codeEcho 1 > /sys/kernel/mm/ KSM /run The first command adds 1 to the runKSM kernel file, which enables the function. The second command, echo 1000 > /sys/kernel/mm/ KSM /sleep_millisecs, tells the KSM daemon to run once per second and evaluate 100 pages for deduplication.
After saving and closing the file, KSM will now be enabled on the next server reboot and retained on future reboots. To enable this server during its current uptime, you need to manually run the two commands you added to rc.local on the command line.
First, start and enter a new shell session as the root user of the server by using the -s flag in sudo. This is necessary because the earlier two commands used output redirection (via the > operator) and needed to be executed by the root shell to succeed.
$ sudo -s
Copy the codeNow enter the previous two commands:
echo 1 > /sys/kernel/mm/ksm/run
echo 1000 > /sys/kernel/mm/ksm/sleep_millisecs
Copy the codeThen, leave the root shell and return to the normal shell session.
$ exit
Copy the codeFinally, to apply all the changes we made in this step and the previous steps, we must restart Netdata.
$ sudo systemctl restart netdata
Copy the codeYou can verify that KSM is now enabled by looking for the newly added active KSM chart in the Netdata dashboard. This can be found under Memory > Memory Deduper in the menu tree on the right.
Now that we know that the dashboard and KSM are up and running, it’s worth accessing the dashboard through a Web server like Nginx rather than a built-in Web server. This is not entirely necessary, but is recommended, so follow the next step.
Step 4 – Through the Nginx managed Dashboard (optional)
Hosting Netdata through Nginx makes it easier to secure access to dashboards and allows you to point to Netdata dashboards on other servers in the future if needed.
You will also need to install the Apache2-utils package. This package contains the htpasswd program, which we will need later to secure access to the dashboard pages.
$ sudo apt-get install apache2-utils
Copy the codeNext, create a new Nginx host configuration file. Note that here we are using a file in the /conf.d directory, but if you are using Nginx for anything other than Netdata, you can use /sites-available and/sites-enabled.
$ sudo nano /etc/nginx/conf.d/default.conf
Copy the codeThis configuration file will tell the Nginx agent incoming requests for dashboard pages. We’ll also add a simple authentication prompt so that only people with the correct username and password can access it.
Here is the entire configuration file you want to copy and paste. Change the two red highlights of the LISTEN and server_name directives above to the IP address of the server and the assigned domain name, respectively. Note the inclusion at the end of each; . If you do not have a domain name, reserve the server_name directive as example.com.
/etc/nginx/conf.d/default.conf
Upstream netdata-backend {server 127.0.0.1:19999; keepalive 64; } server { listen your_server_ip:80; server_name example.com; auth_basic"Authentication Required";
auth_basic_user_file netdata-access;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://netdata-backend;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_set_header Connection "keep-alive"; proxy_store off; }}Copy the codeOnce you determine that your file matches the code block, you can save and exit to return to the command prompt.
Let’s explain this file so you understand what it does. Here’s the first part:
/etc/nginx/conf.d/default.conf
Upstream netdata-backend {server 127.0.0.1:19999; keepalive 64; }Copy the codeThis upstream module, named Netdata-Backend, uses the server’s loopback address 127.0.0.1 and netData’s port 19999 to locate the built-in NetData Web server. The Keepalive directive sets the maximum number of upstream idle connections that can be kept open at any given time for each Nginx worker process. The proxy_pass directive requires this upstream module definition later.
Directly after that comes the main server block.
/etc/nginx/conf.d/default.conf
server {
listen your_server_ip:80;
server_name example.com;
auth_basic "Authentication Required";
auth_basic_user_file netdata-access;
Copy the codeThe first two lines in the block define the external IP addresses that Nginx should listen to when the client tries to connect. The server_name directive tells Nginx to run this server block when the client uses the specified domain name.
The last two lines in this code snippet set up simple HTTP username and password authentication. It uses the auth_basic module to display username and password prompts. You can customize the prompt message;
In this case, it is set to Authentication Required. Auth_basic_user_file specifies the file name and location to store the login credentials for authentication prompts, as created later in this step.
The final Location block, nested within the Server block, handles the broker and passes incoming requests to Nginx.
/etc/nginx/conf.d/default.conf
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://netdata-backend;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_set_header Connection "keep-alive"; proxy_store off; }}Copy the codeIn a nutshell, this code is the code that passes the client request to the Netdata Web server, followed by the dashboard. The proxy_pass directive line that refers back to the upstream module at the beginning of the file must point to it through the name netdata-backend.
You can save and close this configuration file if it is not already installed. This is where the htpasswd program in the Apache2-Utils package helps us create user login credentials for dashboard login prompts.
First, we’ll create a username and password. Run the following command to replace Sammy with the username you want to use in Netdata. When prompted, enter the password you want to use.
$ sudo htpasswd -c /etc/nginx/netdata-access sammy
Copy the codeThis will create a file containing the username and password you provided in /etc/nginx/netdata-access.
The generated password is encrypted in the file, so it cannot be treated as plain text.
Restart Nginx to make the new configuration we added active.
$ sudo systemctl restart nginx
Copy the codeTo test the Nginx configuration for the first time, visit http://your_server_ip in your favorite Web browser. You will be prompted for authentication. Enter the username and password you provided earlier in this step to access the dashboard, which we will explore in Step 5.
Step 5 – Explore the dashboard
If you did not follow optional Step 4, please immediately visit http://your_server_ip:19999/ in your favorite browser to access the dashboard.
You’ll see the dashboard and an overview of the system’s real-time metrics. The HUD style indicator at the top of the page changes as you hover over each time period in subsequent graphics.
The graphs and charts in this overview section provide detailed descriptions of each aspect of the system, ranging from CPU to memory to network traffic.
The chart is interactive and can be dragged left or right using the mouse button to pan back and forth at different intervals.
Holding Down SHIFT and using the mouse wheel to scroll or roll over the graph will shrink or widen the time marker. Double-click the chart to reset it to the default look and view.
The fastest way to navigate the dashboard is to use the menu tree on the right side of the page. This changes focus and color depending on the part of the page you are currently viewing.
Scroll down the page or use the menu tree to access more in-depth charts. These are very detailed and controlled in the same way as the diagrams in the initial overview section.
Netdata provides many of these additional statistics-specific charts.
An important part of the GUI is the update page. Netdata receives regular updates and keeps your installation up to date. The dashboard includes an “update” button at the top that can be used to check if a new version is available for download.
Clicking this button opens a new menu with more details and opens the ** Check now ** button to manually check for updates.
If an update is available, you just need to run the provided update script from the Netdata Git repository, which we cloned into the Linux user’s home directory in the first step. That is, when updates are available, simply run sudo ~/netdata/netdata-updater.sh from the command line.
conclusion
The Netdata Wiki covers the Netdata registry, information on setting up alerts and installing custom charts/modules, and topics such as generating HTML badges or working with Netdata log files.
Reference: How to Set Up Real-time Performance Monitoring with Netdata on Ubuntu 16.04
Question and answer
What are the language requirements for AI development?
reading
How to build Minecraft server on Ubuntu
How to build ark on Ubuntu: Survival and Evolution Server
Create highly usable PostgreSQL clusters using Patroni and HAProxy
This article has been published by Tencent Cloud + community authorized by the author.Cloud.tencent.com/developer/a…
Search concern public number “cloud plus community”, the first time to obtain technical dry goods, after concern reply 1024 send you a technical course gift package!
Massive technical practice experience, all in the cloud plus community!