This is the fourth day of my participation in the August More text Challenge. For details, see: August More Text Challenge

In the current development process, Nginx is widely used as a Web server, resulting in a large number of access logs, and various analysis platforms are born. The purpose is to use the logs generated by Nginx to analyze the website user access behavior, or to monitor the platform running status. Among many tools, GoAccess is a simple and lightweight analysis tool that is very convenient to use.

Introduction to the

GoAccess is designed to be a fast, terminal based log analysis tool. The core idea is that you can quickly analyze and view Web server statistics in real time without having to go through a Web browser (which is great for those who need to use SSH for quick analysis of access logs or who just like to work in a terminal environment).

Terminal output is simply the default, and GoAccess also supports full real-time HTML reports (great for analysis, monitoring, and data visualization), as well as reports in JSON and CSV formats.

Install GoAccess

1. Install dependency libraries
yum install -y GeoIP-devel ncurses-devel
Copy the code
2. Download and install GoAccess
Wget https://tar.goaccess.io/goaccess-1.4.5.tar.gz tar - XZVF goaccess - 1.4.5. Tar. GzcdGoaccess-1.4.5 /./configure --enable-utf8 --enable-geoip=legacy make && make installCopy the code

Note: You can also install it through yum if you find it difficult

yum install goaccess
Copy the code
3. Configure the log format

Configuration file located in: / usr/local/etc/goaccess/goaccess. Conf

time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h - %^ [%d:%t %^]; "%r" %s %b "%R"- % ^"%u" - %^ %T
Copy the code
4. Analyze log results on the terminal
goaccess -a -d -f /var/log/nginx/access.log
Copy the code

Use the timer to output the analysis results in HTML format

1. Write script files
# Analysis report storage path
mkdir -p /data/www/goaccess/html
# Analysis script storage path
mkdir -p /usr/local/goaccess/scripts
Get the HTML analysis report
vim /usr/local/ goaccess/scripts/getHtmlReport - the project name. ShCopy the code
  • GetHtmlReport – Project name. sh Contents are as follows:
#! /bin/bash
Convert the page to Chinese
LANG="zh_CN.UTF-8"
# Output HTML analysis report
goaccess -a -d -f /var/log/ nginx/access log - o/data/WWW/goaccess/HTML/report - the project name. HTMLCopy the code
2, assign script executable permission:
chmod 755 /usr/local/ goaccess/scripts/getHtmlReport - the project name. ShCopy the code
3. Add scheduled tasks
# Add a scheduled task
crontab -e

Generate the nGINx log analysis HTML report for the project regularly
0 0 * * * /bin/bash /usr/local/ goaccess/scripts/getHtmlReport - the project name. ShCopy the code
4. Configure the nginx access path
Redirects it to the specified HTML using aliasLocation /report- project name. HTML {alias/ data/WWW/goaccess/HTML/report - the project name. HTML. }Copy the code

Set the nGINx access permission

(basicAuth) (basicAuth) (basicAuth)

Note: If you are using on an Intranet or are not concerned with access control, you may not need to set nginx access

1. Install the HTPassed tool
yum -y install httpd-tools
Copy the code
2, set the user name and password, and save the user name and password to the specified file:
htpasswd -c [passwfile] [username]

# example: htpasswd -c /etc/nginx-passwd you
Copy the code

Verification method: Check the generated password file /etc/nginx/passwd. The encrypted password is displayed after the semicolon.

3. Modify the nginx configuration file
server { listen 443 ssl; . location /report-armstrong.html {# Add the following two lines
            auth_basic "Please input password"; # Here is the prompt for validation
            auth_basic_user_file /etc/nginx/passwd;
            
            alias/data/www/goaccess/html/report-armstrong.html; }}Copy the code
4. Restart nginx and access

Restart nginx and access it in the browser/report-armstrong.html, enter the user name and password to view the corresponding data