[Reproduced please indicate the source] :Juejin. Cn/post / 684490…

1, Nginx status monitoring

Nginx provides a built-in status monitoring page that can be used to monitor overall access to Nginx. This function is implemented by the ngx_HTTP_stub_status_module module. Use nginx – V 2 > &1 | grep – with o – http_stub_status_module command to test whether the current nginx status function, if the output ngx_http_stub_status_module shows is some, If not, you can add this module at compile time. By default, status is turned off, and we need to turn it on and specify the URI to access the data.

server { listen 80; server_name default_server; location /status { stub_status on; Allow 114.247.125.227; }}Copy the code

Allow allows only specified Ip addresses to access the nginx status function. After Nginx is restarted, the browser visits http://{IP}/status to view the status monitoring information

  • Active Connections: indicates the number of Active client connections (including waiting client connections), which is equivalent to TCP connections in Established or SYN_ACK state
  • Accepts: The total number of client connections accepted, that is, connections accepted by the worker process
  • Handled: Number of connections that have been handled
  • Requests: specifies the total number of HTTP requests from clients
  • Reading: Number of HTTP requests currently being read (read to HTTP request header)
  • Writing: Number of connections currently prepared for the response (written to the HTTP response header)
  • Waiting: Number of idle client requests currently Waiting between Reading and Writing

After collecting Nginx data, you can use the monitoring tool to monitor it.

2. Log analysis

The default log format configuration for Nginx can be found in /etc/nginx/nginx.conf

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" $request_time $upstream_response_time';

Copy the code

A printed log instance

39.105.66.117-MP [11/Sep/ 2019-19:03:01 +0800]"POST/salesplatform - gateway/users HTTP / 1.1"200, 575,"-" "Apache HttpClient / 4.5.5 (Java / 1.8.0 comes with _161)" "-"[11/Sep/ 2019-19:03:08 +0800]"POST/salesplatform - gateway/users HTTP / 1.1"200, 575,"-" "Apache HttpClient / 4.5.5 (Java / 1.8.0 comes with _161)" "-" 0.008 0.008
Copy the code
  • $remote_ADDR: indicates the IP address of the client
  • $remote_user: Records the user name of the remote client
  • $time_local: records the access time and time zone
  • $request: Records the URL and method of the request
  • $status: indicates the response status code
  • $body_bytes_sent: number of bytes of file body content sent to the client
  • $http_referer: You can keep track of which link the user is visiting from
  • $http_user_agent: indicates the browser information used by the user
  • $http_X_forwarded_for: Records the client IP address through the proxy server
  • $request_time: specifies the time between receiving the first byte of the request and sending the response data. $request_time includes the time to receive the request data from the client, the time to send the response data to the client
  • $upstream_response_time: The time used to receive the response from the upstream server
Common Analysis Commands

1, according to the access IP statistics UV

awk '{print $1}' paycenteraccess.log | sort -n | uniq | wc -l
Copy the code

2. Query the most frequently accessed IP(top 10)

awk '{print $1}' /var/log/nginx/access.log | sort -n |uniq -c | sort -rn | head -n 10
Copy the code

3. View the IP access volume of a certain period (1-8 o ‘clock)

awk '$4 >="[25/Mar/2020:01:00:00" && $4 <="[25/Mar/2020:08:00:00"' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c| sort -nr |wc -l
Copy the code

4. View the IP addresses accessed more than 100 times

awk '{print $1}' /var/log/nginx/access.log | sort -n |uniq -c |awk '{if($1 >100) print $0}'|sort -rn
Copy the code

5. View the url and access count of the specified IP address

grep "39.105.67.140" /var/log/nginx/access.log|awk '{print $7}' |sort |uniq -c |sort -n -k 1 -r
Copy the code

6. PV statistics based on access URL

cat /var/log/nginx/access.log |awk '{print $7}' |wc -l
Copy the code

7. Query the most frequently visited URL(top 10)

awk '{print $7}' /var/log/nginx/access.log | sort |uniq -c | sort -rn | head -n 10
Copy the code

8, Check the most frequently accessed URL([exclude/API/appID])(top 10)

grep -v '/api/appid' /var/log/nginx/access.log|awk '{print $7}' | sort |uniq -c | sort -rn | head -n 10
Copy the code

9. View pages that have been visited more than 100 times

cat /var/log/nginx/access.log | cut -d ' ' -f 7 | sort |uniq -c | awk '{if ($1 > 100) print $0}' | less
Copy the code

10. View the most visited pages of the last 1000 records

tail -1000 /var/log/nginx/access.log |awk '{print $7}'|sort|uniq -c|sort -nr|less
Copy the code

Count the number of requests per hour,top10 time points (accurate to hour)

awk '{print $4}' /var/log/nginx/access.log |cut -c 14-15|sort|uniq -c|sort -nr|head -n 10
Copy the code

12. Count requests per minute,top10 time points (accurate to minute)

awk '{print $4}' /var/log/nginx/access.log |cut -c 14-18|sort|uniq -c|sort -nr|head -n 10
Copy the code

Count requests per second,top10 points in time (accurate to second)

awk '{print $4}' /var/log/nginx/access.log |cut -c 14-21|sort|uniq -c|sort -nr|head -n 10
Copy the code

14. Search for logs generated within a specified period

awk '$4 >="[25/Mar/2020:01:00:00" && $4 <="[25/Mar/2020:08:00:00"' /var/log/nginx/access.log 
Copy the code

15. List the urls whose transmission time exceeds 0.6 seconds, and display the top 10 urls

cat /var/log/nginx/access.log |awk '(substr ($NF, 2, 5) > 0.6) {print $4, $7, substr ($NF, 2, 5)}' | awk -F '"' '{print $1,$2,$3}' |sort -k3 -rn | head -10
Copy the code

16, List The Times when the/API/appID request took longer than 0.6 seconds

cat /var/log/nginx/access.log |awk '(substr ($NF, 2, 5) > 0.6 && $7 ~ / \ \ / API/appid /) {print $4, $7, substr ($NF, 2, 5)}' | awk -F '"' '{print $1,$2,$3}' |sort -k3 -rn | head -10
Copy the code

17, obtain the top 10 most time-consuming request time, URL, time

cat /var/log/nginx/access.log |awk '{print $4, $7, substr ($NF, 2, 5)}' | awk -F '"' '{print $1,$2,$3}' | sort -k3 -rn | head -10
Copy the code

[Reproduced please indicate the source] :Juejin. Cn/post / 684490…