preface

This article is about Nginx performance optimization, which is not used in the actual production environment. If you want to use it in practice, please be careful to use it after testing.

Nginx performance optimizations, primarily to reduce disk IO.

  • The request header, request body, and response body all operate in the buffer.
  • Read file information

Reduce network I/O.

  • Gzip compression. Front-end resources can also be advancedgzipCompress, so that the request does not have to compress, reduce pairscpuThe wear and tear.
  • Strong cache. Reduce requests for static resources on the back end.

HTTP links are released as quickly as possible, reducing the backlog of requests.

Linux kernel optimization. This part is mainly refer to the data and their own understanding. Nginx Module Development and Architecture Design.

After adjusting the account parameters, ab and Jmeter can be used for pressure measurement, and the tuning can be carried out according to the actual effect.

Nginx must perform performance optimizations in a safe manner. Otherwise, TCP attacks can bring your server down, using is king. Security access first, performance second.

After tuning, be sure to limit the current.

The next series is about Mysql and is already in preparation.

Linux Kernel Optimization

You can modify kernel parameters by modifying /etc/sysctl.conf. This part is carefully optimized

  • View TCP system parameters
 sysctl -a | grep 'net.ipv4.tcp' | grep -v grep
Copy the code
fs.file-max = 999999
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_keepalive_time = 15
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_rmem = 4096 32768 262144
net.ipv4.tcp_wmem = 4096 32768 262144
net.ipv4.tcp_max_orphans = 262144 
net.core.netdev_max_backlog = 262144
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
net.core.somaxconn = 262144 
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog=262144
Copy the code

After the modification, run the following command to make the configuration take effect.

sysctl -p
Copy the code

The meanings of the preceding parameters are explained as follows:

  • Fs. File – Max: 999999

    This parameter indicates the maximum number of handles that a process (such as a worker process) can open at the same time. This parameter directly limits the maximum number of concurrent connections and needs to be configured as required.

  • Net. Ipv4. Tcp_tw_reuse:

    If this parameter is set to 1, time-wait sockets are allowed to be reused for new TCP connections, which makes sense on a server where there are always a large number of time-Wait connections.

  • Net. Ipv4. Tcp_keepalive_time:

    This parameter indicates how often TCP sends Keepalive messages when Keepalive is enabled. The default is 2 hours, but setting it lower will make it quicker to clean up invalid connections.

  • Net. Ipv4. Tcp_fin_timeout:

    This parameter indicates the maximum amount of time the socket can remain in fin-WaIT-2 state when the server actively closes the connection.

  • Net. Ipv4. Tcp_max_tw_buckets:

    This parameter indicates the maximum number of TIME_WAIT sockets allowed by the operating system. If this number is exceeded, TIME_WAIT sockets will be cleared immediately and a warning will be printed. This parameter defaults to 180000, and too many TIME_WAIT sockets can slow down the Web server.

  • Net. Ipv4. Ip_local_port_range:

    This parameter defines the range of local (but not remote) ports on both UDP and TCP connections.

  • Net. Ipv4. Tcp_rmem:

    This parameter defines the minimum, default, and maximum value of the TCP receive cache for the TCP receive sliding window.

  • Net. Ipv4. Tcp_wmem except:

    This parameter defines the minimum, default, and maximum value of the TCP send cache for the TCP send sliding window.

  • net.ipv4.tcp_max_orphans

    Option to record the maximum number of connection requests that have not received confirmation from the client. The default value of this parameter is 1024 for a system with 128MB of memory and 128 for a system with small memory.

  • Net.core.net dev_max_backlog:

    When the nic receives packets faster than the kernel can process them, a queue holds the packets. This parameter represents the maximum size of the queue.

  • net.core.net.core.rmem_default:

    This parameter represents the default size of the kernel socket receive cache.

  • Net. Core. Wmem_default:

    This parameter represents the default size of the kernel socket send cache.

  • Net. Core. Rmem_max:

    This parameter represents the maximum size of the kernel socket receive cache.

  • Net. Core. Wmem_max:

    This parameter represents the maximum size of the kernel socket send cache.

The size of the sliding window and the socket cache affect the number of concurrent connections to some extent.

Each TCP connection consumes memory to maintain the TCP sliding window, which shrinks or expands depending on the server’s processing speed.

The wmem_max parameter needs to be set in a balance between the total size of physical memory and the maximum number of connections that can be concurrently processed by Nginx (as determined by worker_processes and worker_connections parameters in nginx.conf).

Of course, it is not suitable to reduce the sliding window size just to increase the concurrency and prevent the server from Out Of Memory problems, because too small sliding window will affect the speed Of large data transfer.

Rmem_default, wmem_default, rmem_max, and wmem_max must be set based on service features and actual hardware costs.

  • net.core.somaxconn

    The option represents the maximum number of packets allowed to be sent to the queue when each network interface receives packets at a rate faster than the kernel can process them.

  • Net. Ipv4. Tcp_syncookies:

    Set it to 1. This parameter is independent of performance and is used to resolve TCP SYN attacks.

  • Net. Ipv4. Tcp_max_syn_backlog:

    This parameter indicates the maximum length of the TCP three-way handshake to receive SYN requests. By default, this parameter is 1024, so that Linux does not lose client connections if Nginx is too busy to accept new connections.

Memory and disk data optimization

client_header_buffer_size 1k;

1K is sufficient for static resource access.

The buffer size of the client request header is 1K by default. When requesting the interface, the buffer size is set to 4k integer multiples of the request header data.

Getconf PAGESIZE specifies the size of the memory page

large_client_header_buffers

client_body_in_single_buffer

If set to ON, HTTP packages are written to the buffer, or to disk files if the package body exceeds client_body_buffer_size.

client_body_buffer_size

The default value for the X64 is 16K. Defines the size of the buffer in which requests are written to the cache before they are written to a temporary file.

client_max_body_size 100m;

Set the maximum size of a client request body. The default size is 1 MB. The check is content-Length. If set to 0, do not check. Location specific configuration. To prevent attack by request, set it in the place where it needs to be increased, not globally.

sendflie on;

The file content reading reduces the copy from kernel mode to user mode, and directly from kernel mode to nic device, improving the transmission efficiency.

open_file_cache

Cache file storage information. Max indicates the maximum amount of storage. If this amount is exceeded, LRU is used for elimination

Inactive Specifies a period of time during which elements that are not accessed are eliminated

open_file_cache max=65535 inactive=20s;

open_file_cache_min_uses

The default value is 1. Used in conjunction with open_file_cache inactive. If the number of accesses exceeds the number of open_FILe_cache_min_uses specified in inactive time, the cache will not be discarded.

Increase the file handle limit

Linux is all about files, but there is a limit to how many files a process can open. The number of file handles can be limited for users and processes.

  • File handle, which can be set for user and process

    • Global Settings/etc/security/limits. Conf
* hard nofile 65535
* soft nofile 65535
root hard nofile 65535
Copy the code
- Nginx process configurationCopy the code
worker_rlimit_nofile 20480;
Copy the code

Log optimization

Log operations can be reduced appropriately. Logging access to static data, for example, can be unlogged if you feel it is not important. This reduces the disk IO of the log when requesting the resource.

# disable log
access_log off;
# disable file missing errors to log
log_not_found off;
Copy the code

Reverse proxy optimization

If you use Nginx as a proxy server, reduce disk IO reads as well.

proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 16 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
proxy_set_header Host $http_host;
proxy_set_header X-REAL-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Copy the code

Nginx optimized configuration

# configure the user and user group to which the worker process belongs
user nginx nginx;

# configure the number of worker processes. To avoid CPU switching, configure the number of system cores, or auto
worker_processes auto;

Configure CPU affinity. Auto stands for automatic binding
worker_cpu_affinity auto;

# number of open file descriptors for the nginx process. This value overrides the value of ulimit -n.
worker_rlimit_nofile 65535;

events {
    # Use this model to efficiently handle asynchronous events
    use epoll;

    # On worker processes take turns accepting new links. The official recommendation is to set off. Set it to ON for high load.
    accept_mutex on;

    Whether the worker process accepts all new requests simultaneously. The default is off, which means that only one new request is accepted at a time. Official recommendation off
    multi_accept on;

    Set the number of connections to be processed by a woker process
    worker_connections 65535;
}
http {
    # disable log
    access_log off;
    Hiding information about the operating system and Web Server (Nginx) version number in the response header is good for security.
    server_tokens off;
    sendfile on;
    # When set to non-zero, limit the amount of data transferred during a single sendFile () call. Without restrictions, a fast connection can completely tie up the worker process.
    sendfile_max_chunk 5m;
    # tcp_nopush and tcp_nodeny can work together
    Sendfile will take effect when the data packets reach a certain size
    tcp_nopush on;
    # This option is enabled only when the connection transitions to the keep-alive, long connection state. Make TCP send packets as soon as possible.
    tcp_nodelay on;
    To release the connection as quickly as possible, you can set a small point. 15 to 30
    keepalive_timeout 15;

    The buffer size of the client request header is 1K by default, and 4k integer multiples are required when requesting the interface. To set the memory size to the system memory PAGESIZE, run the getconf PAGESIZE command
    client_header_buffer_size 4k;

    large_client_header_buffers 8 8k;

    The interface request can be set as large as required
    client_body_buffer_size 128k;

    Set the maximum size of the client request body to 1m by default. The check is content-Length. If set to 0, do not check. Location specific configuration
    client_max_body_size 1m;


    This parameter specifies the number of open files to be cached. By default, this parameter is disabled.
    # inactive indicates how long it has been since the file has been requested before the cache is deleted.
    open_file_cache max=262140 inactive=20s;
    How often does the cache check for valid information?
    open_file_cache_valid 30s;
    The minimum number of times a file is accessed in the inactive parameter of the # open_file_cache directive, below which the cache is cleared
    open_file_cache_min_uses 1;
    open_file_cache_errors on;

    reset_timedout_connection on;
    client_body_timeout 10;
    send_timeout 2;

    Limit the number of connections per IP
    limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;

    Limit requests per IP address per second
    limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=10r/s;

    gzip on;
    # add to the response header, Vary: accept-encoding
    gzip_vary on;
    # gzip compression level 1-9, the larger the number of compression effect is better, the longer the compression time is also higher CPU
    gzip_comp_level 5;
    If the size of the source file exceeds 8K, the size of the source file is 16*8K.
    gzip_buffers 8 128k;
    gzip_min_length 5K;
    gzip_proxied any;
    gzip_disable msie6;
    gzip_http_version 1.1;
    # Text (JS, text, CSS, XML, JSON) compression is better, images have been compressed, the effect is not very obvious, but also a waste of CPU
    gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml+rss application/rss+xml application/atom+xml image/svg+xml;

    # security-related headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header Feature-Policy "autoplay 'none'; camera 'none'" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;


    server {
        listen 80 backlog=262144;

        limit_conn conn_limit_per_ip 10;
        limit_req zone=req_limit_per_ip burst=10 nodelay;
        # assets, media
        location ~ * \. (? :css(\.map)? |js(\.map)? |jpe? g|png|gif|ico|cur|heic|webp|tiff? |mp3|m4a|aac|ogg|midi? |wav|mp4|mov|webm|mpe? g|avi|ogv|flv|wmv)$ {
            # Strong cache, for one year, browser and CDN middleware can cache
            add_header Cache-Control "max-age=31536000";
            etag off;
            access_log off;
            # disable file missing errors to log
            log_not_found off;
        }

        # svg, fonts
        location ~ * \. (? :svgz? |ttf|ttc|otf|eot|woff2?) $ {
            # Strong cache, for one year, browser and CDN middleware can cache
            add_header Cache-Control "max-age=31536000";
            etag off;
            access_log off;
            # disable file missing errors to log
            log_not_found off; }}}Copy the code

Data reference

http-security-headers

Nginx Performance Tuning — Tips & Tricks

In-depth understanding of Nginx module development and architecture design.

This article was created by Zhang Panqin on his blog www.mflyyou.cn/. It can be reproduced and quoted freely, but the author must be signed and indicate the source of the article.

If reprinted to wechat official account, please add the author’s official qr code at the end of the article. Wechat official account name: Mflyyou