Keepalived+Nginx to build an active/standby high availability load balancing cluster

What is Keepalived?

Keepalived is a based on the high availability cluster of VRRP protocol software, through the virtual IP (VIP) to provide services to foreign, real time monitoring in the cluster server running state and fault isolation automatically, these servers are started with the same service, failure occurs when the primary server, virtual IP drift to the backup server will automatically, In this way, services are highly available.

High availability cluster architecture diagram

Experimental scheme planning

App-01, APP-02 and APP-03 are all on the same VIRTUAL machine. Three Docker containers are used to simulate the scenario of three back-end servers. The system environment of the three virtual machines is Centos+Docker

Setting up a back-end server environment

Use the following command to quickly create three back-end services

docker run -d --name APP-01 -p 8080:80 nginx:alpine \ && docker run -d --name APP-02 -p 8081:80 nginx:alpine \ && docker  run -d --name APP-03 -p 8082:80 nginx:alpineCopy the code

Modify/usr/share/nginx/HTML/index. The HTML files, marked the current container name, facilitate subsequent testing

Modify/etc/nginx/conf. D/default. The conf file, added to the server block add_header LB_IP $remote_addr, identify the load balancing server IP, to facilitate subsequent tests

Setting up an active and standby load balancing server environment

Install Nginx

Docker run -d –name load-balancer –restart=always -p 8080:80 nginx:alpine docker run -d –name load-balancer –restart=always -p 8080:80 Modify the /etc/nginx/nginx.conf configuration file as follows:

user nginx; worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; Upstream backend-server {server 192.168.31.242:8080; Server 192.168.31.242:8081; Server 192.168.31.242:8082; } server { server_name localhost; listen 80 ; location / { proxy_pass http://backend-server; }}}Copy the code

You can access back-end services by accessing 192.168.31.240:8080 (lB-01) or 192.168.31.241:8080 (LB-02)

Install Keepalived

Here use Docker image osixia/ Keepalived installation Keepalived service, specific instructions to see the documentation

Lb-01 Primary server

Use the following command to create the container

docker run --cap-add=NET_ADMIN --cap-add=NET_BROADCAST --cap-add=NET_RAW --net=host -d --name keepalived --restart=always \ -e KEEPALIVED_INTERFACE='enp0s3' \ -e KEEPALIVED_PASSWORD='d0cker' \ -e KEEPALIVED_STATE='BACKUP' \ -e KEEPALIVED_ROUTER_ID='51' \ -e KEEPALIVED_PRIORITY='120' \ -e KEEPALIVED_UNICAST_PEERS='192.168.31.241' \ -e KEEPALIVED_VIRTUAL_IPS = '192.168.31.250 \ osixia/keepalived: 2.0.20Copy the code

Keepalived configuration is as follows:

global_defs { default_interface enp0s3 } vrrp_script check_nginx { script "/ container/service/keepalived/assets/check_nginx. Sh" # test script file interval 15 # 7 # test interval weight - the weight} vrrp_instance VI_1 { Enp0s3 # enp0s3 # enp0s3 # enp0S3 # Virtual_router_id 51 # Virtual_router_id must be the same as virtual_router_id 51 # Priority 120 # Unicast_peer {# unicast mode, set peer IP 192.168.31.241} virtual_ipaddress {# set VIP, 192.168.31.250} authentication {# auth_type PASS # Support PASS and AH auth_pass D0cker # authentication password} track_script {# set trace script check_nginx} notify "/container/service/keepalived/assets/notify.sh" }Copy the code

Lb-02 Standby server

Use the following command to create the container

docker run --cap-add=NET_ADMIN --cap-add=NET_BROADCAST --cap-add=NET_RAW --net=host -d --name keepalived --restart=always \ -e KEEPALIVED_INTERFACE='enp0s3' \ -e KEEPALIVED_PASSWORD='d0cker' \ -e KEEPALIVED_STATE='BACKUP' \ -e KEEPALIVED_ROUTER_ID='51' \ -e KEEPALIVED_PRIORITY='110' \ -e KEEPALIVED_UNICAST_PEERS='192.168.31.240' \ -e KEEPALIVED_VIRTUAL_IPS = '192.168.31.250 \ osixia/keepalived: 2.0.20Copy the code

Keepalived configuration is as follows:

global_defs { default_interface enp0s3 } vrrp_script check_nginx { script "/container/service/keepalived/assets/check_nginx.sh" interval 7 weight -15 } vrrp_instance VI_1 { interface enp0s3 State BACKUP virtual_router_id 51 priority 110 nopreempt unicast_peer {192.168.31.240} virtual_ipAddress { 192.168.31.250} authentication {auth_type PASS auth_pass d0cker} track_script {check_nginx} notify "/container/service/keepalived/assets/notify.sh" }Copy the code

Create a detection script file

Lb-01 and LB-02 keepalived are configured to be non-preemptive. When the Nginx service of LB-01 is faulty, the priority is 105. When the Nginx service of LB-02 is normal, the priority is 110. Lb-02 does not preempt to the Master state, so you need to add a command to disable Keepalived to check the Nginx state, so that the Nginx can complete the active/standby switchover in case of a problem. / container/service/keepalived/assets/check_nginx. Sh is as follows:

#! /bin/bash nc -z localhost 8080 || kill 1Copy the code

Using chmod + x/container/service/keepalived/assets/check_nginx. Sh add script execute permissions

Access to the test

After the configuration is complete, restart the load-balancer and Keepalived containers of LB-01 and LB-02 to perform the following tests

• whenLB-01,LB-02In normal operation,LB-01The primary server receives client access traffic

• whenLB-01Go down,LB-02In normal operation,LB-02It automatically takes over the program and receives client access traffic

• restartLB-01Server and open Nginx, Keepalived service container,LB-02Works well, Keepalived is configured as non-preemptive, so it is still availableLB-02Receives client access traffic

• whenLB-01Working normally,LB-02When the Nginx service fails, the script to check the Nginx status is added, which automatically shuts down the serviceLB-02Keepalived service, so byLB-01Receives client access traffic

Optimized (LVS+Keepalived+Nginx)

The above solution is still single point of access. If the access traffic exceeds the upper limit of Nginx performance and the service still cannot reach high availability, LVS (Linux Virtual Server) can be used, which is a virtual server cluster system in the Linux systemIP_VSKernel module, can be used to do load balancing, performance is much stronger than Nginx, can undertake more client access traffic, architecture diagram is as follows: