Today we continue to talk about nginx, before just introduced the configuration of Nginx, this time mainly talk about the configuration of nginx production environment, how we configure in production. Source: github.com/limingios/n…

System type The IP address The node role CPU Memory Hostname
Centos7 192.168.66.110 nginx 1 2G nginx
Centos7 192.168.66.111 tomcat 1 2G tomcat1
Centos7 192.168.66.112 tomcat 1 2G tomcat2

Nginx implements caching

The scene is introduced

Suppose you are working on a large project, facing the whole country, and the company requires the architect to achieve 700+ QPS, how to deal with it?

  • Introduce some keywords that you need to know to develop a website
  1. Response time (RT)

Response time is the time it takes for the system to respond to a request.

  1. Throughput (TPS)

The number of requests processed by the system per unit of time

  1. Number of concurrent users

The number of concurrent users that a system can host who normally use system functions

  1. QPS Query rate per second

A measure of how much traffic is being processed by a particular query server over a specified period of time

  • Know the typical actual HTML size for the next web site

Open a tech blog: idig8.com

View source code

Save in the file size: 45KB, 700QPS/s = 700*45/1024 = 30MB, a single page if 700QPS need to spit out 30MB/s amount, after all, is a personal blog cannot withstand.

Let’s take some east as an example, save in the file size: 186KB, 700QPS/s = 700*186/1024 = 100MB+, a single page if 700QPS need to spit out 100MB/s amount, some east good awesome.

If let us design jingdong so much understanding how to design.

  • The product details page covers the following main services:
  1. Product details page HTML page rendering
  2. Price of service
  3. Promotional service inventory status/delivery to service adwords service
  4. Pre-sale/SEC kill service
  5. Evaluation of the service
  6. Trial service
  7. Recommended service
  8. Product introduction service Some special services related to each category

Everything up there is microservices.

  1. Ajax is used to load prices, advertising, inventory and other services dynamically
  2. Use key value to cache the detail page body HTML.

Many of the larger internal systems are designed this way.

Microservice and microservice before the Intranet are all gigabit bandwidth, the first time from the microservice. After that, it’s all through Redis. Special data via Ajax. Getting it from Redis is definitely a big improvement. If you use this situation 500qps is still ok, if you want to go up, it’s pretty hard. But our requirement goes up to 700qps.

At 500QPS it is difficult to continue to pressure up.

Analysis reason: a detail page HTML body up to an average of 150 KB then at 500QPS is close to the LAN broadband limit. 75MB+, the Intranet is generally gigabit bandwidth, millennium bandwidth 128MB/s, in fact, this is also towards a bottleneck of Intranet IO. In the diagram above, the Intranet actually goes through two times, one is through nginx to access the detailed service of the product, the other is the detailed service to access redis.

  • If you want to solve the above two Intranet communication the most ideal way, the above two nodes can not be used.

Instead of using the Redis cache or nginx to request the commodity detail page service, nginx directly uses the local hard disk cache. The bottleneck of Intranet communication is solved.

  • The solution

To reduce Intranet communication, Nginx can also be configured to cache data to the local disk, so that the next request will be directly requested by nginx internal disk cache data, which also reduces Intranet communication.

The normal process is this, with caching.

Nginx automatically adds caches when there are no caches normally

The idea is to clear the cache under Nginx through MQ message service, through the details page service, by the details page service unified request direct address.

Nginx cache configuration

  • How does nginx configure caching

Proxy_cache is a caching function for proxy mode (also known as anti-generation)

events {
    worker_connections  1024;
}


http {
      include       mime.types;
      default_type  application/octet-stream;
      sendfile        on;
      keepalive_timeout  65;
# configure cacheproxy_cache_path /data/nginx/cache_item levels=1:2 keys_zone=cache_item:200m inactive=30d max_size=10g; Upstream {server 192.168.66.111:8080 weight=5; Server 192.168.66.112:8080 weight = 5; } server { listen 80; server_name localhost; location / { root html; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; }# configure cache
        location ~*\.(html|htm)$ {
          proxy_set_header Host $host;
          proxy_set_header X-Forwarded-For $remote_addr;
          proxy_pass http://idig8;
          proxy_cache cache_item;
          proxy_cache_key $host$uri$is_args$args; proxy_cache_valid 200 304 12h; expires 7d; }}}Copy the code

  • Create a new index.html file in the tomcat directory

tomcat1

Idig8.com 192.168.66.111Copy the code

tomcat2

Idig8.com 192.168.66.112Copy the code

  • Restart the nginx
./sbin/nginx -s stop
mkdir -p /data/nginx/cache_item
./sbin/nginx
Copy the code
  • Viewing the cache path
pwd
cd /data/nginx/cache_item/3/cf
ls
Copy the code

  • Clear the cache

See the effect

Modified index. HTML

The request is still the data in the cache

Review how to configure the scheme Nginx configuration implementation

Add a cache declaration under the HTTP element. proxy_cache_path /data/nginx/cache_item levels=1:1:2 keys_zone=cache_item:500m inactive=30d max_size=10g; 2. Set cache policy for specified location. proxy_cache cache_item; proxy_cache_key$host $uri$is_args$args;# use the md5 value as the Key
proxy_cache_valid 200 304 12h; Set different cache times for different HTTP status codes
expires 7d; Total cache time
Copy the code

Demonstrates how the cache takes effect

  1. The configuration declares the cache path
  2. Configure the cache policy for Location
  3. Restart nginx (modified)
  4. View cache directory generation
The parent element The name of the describe
http proxy_cache_path Specifies the root path for the cache
levels The cache directory level contains the highest three layers, and each layer contains 1 to 2 characters. For example, 1:1:2 represents three layers.
keys_zone Cache block name and memory block size. Such as cache_item: 500 m. Cache_item specifies a cache_item whose size is 500m. The earliest data that exceeds the size will be cleared.
inactive Maximum idle time :10d If a data has been idle for 10 days, it will be cleared
max_size Maximum number of hard disks in the cache. Excess idle data will be cleared
location proxy_cache Specifies the cache area, corresponding to the value set in keys_zone
proxy_cache_key The cache key is assembled by parameters, for example, host URI is_args. Args uses the md5 value of the full path as the key
proxy_cache_valid Set the cache validity period for different status codes
  • Cache clearing:

This functionality can be implemented using the third-party module ngx_CACHE_purge. Compile ngx_cache_purge into Nginx to purge the cache for the specified URL.

  • Add the ngx_cache_purge module to nginx
cdWget ~ http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz# Check installed modules
~/nginx/sbin/nginx -V
# Go to the nginx installation package directory and reinstall --add-module is the full path to decompress the module
cd~/nginx-1.13.10./configure --prefix=/root/nginx --with-http_stub_status_module --with-http_ssl_module - add = / root/ngx_cache_purge - 2.3 - the module# recompileMake cp/root/nginx 1.13.10 / sbin/nginx ~ / nginx/sbin/nginxCheck whether the installation is successful
./nginx/sbin/nginx -t
Copy the code

Remove the configuration

location ~ /purge(/.*) {
The IP address allowed to accessAllow 127.0.0.1; Allow 192.168.0.193;Disable IP addresses
deny all;
# configure to clear specified cache and path (with proxy_cache_key 1 to)
proxy_cache_purge cache_item $host$1$is_args$args;
}
Copy the code

How to protect Nginx from theft

Lift to ask:

What is picture anti-theft chain? This site pictures, CSS and other resources only this site can access, do not allow other sites to open! How to implement this function in JAVA? It’s as simple as checking whether the referer attribute in the request header is the specified domain name. Nginx works similarly.

location ~* \.(gif|png|jpg|swf|flv)$ {
root html;
valid_referers none *.tl.com;
if ($invalid_referer) {
rewrite ^/ http://www.tl.com/image/403.png;
#return 404;}}Copy the code

If the valid_referers condition fails, nginx assigns invalid_referer to true

Grammar: valid_referers none | blocked | server_names | string… ; Parameter Description:

none

Blocked does not allow the “Referer” value to be null when the “Referer” source header is null. Server_names The “Referer” source header must contain the current server_names (current domain name). It can be multiple

Nginx implements subdomain site configuration, similar to hasten city site configuration (3)

Sometimes there is a requirement that each subdomain corresponds to a static site (such as 58 home, youzan Mall, etc.). If you add a domain name every day

It can be quite troublesome. In nginx, you can connect directly to the corresponding directory based on $host. The configuration is as follows:

server {
listen 80;
server_name *.tl.com;
root /data/www/$host;
access_log logs/$host.access.log; location / { index index.html; }}Copy the code

Make sure to modify the host file, which depends on the host file.

PS: Nginx is usually used as a reverse proxy. In fact, many special configurations are often used in large Internet e-commerce companies. So caching and anti-theft is also a nice feature.