preface

Keepalived is based on Virtual Router Redundancy Protocol (VRRP), which is a Protocol for highly available routers. Keepalived consists of multiple routers providing the same functionality in a Router group.

Principle: A Keepalived cluster consists of a MASTER and multiple backups. The MASTER node has a Virtual IP(VIP) that provides services to the outside world. The MASTER sends multicast heartbeat messages. In this case, you need to elect a BACKUP as the MASTER based on the VRRP priority. When the MASTER recovers, the BACKUP releases the IP resources and services taken over by the MASTER and restores to the original standby role, thus ensuring high availability of the router.

The environment that

  • CentOS 7 (Minimal Install)
# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
Copy the code
  • The demo environment
VIP IP The host name
10.10.0.10 10.10.0.11 master
10.10.0.10 10.10.0.12 backup

The deployment of

Replace the server source warehouse

# mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
# curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
# yum makecache
# yum -y update
Copy the code

Install Keepalived

Yum install

Keepalived can be installed directly using YUM and executed on master and Backup servers:

# yum -y install keepalived
Copy the code

Source code compilation and installation

Reference documentation: Keepalived official documentation

Installing dependent libraries

# yum -y install openssl-devel libnl3-devel ipset-devel iptables-devel file-devel net-snmp-devel glib2-devel json-c-devel pcre2-devel libnftnl-devel libmnl-devel
Copy the code

Download Keepalived

#Wget HTTP: / / https://github.com/acassen/keepalived/archive/v2.0.18.tar.gz
Copy the code

Unpack the Keepalived

#The tar - ZXVF v2.0.18. Tar. Gz
# cdKeepalived - 2.0.18
Copy the code

Start the installation

# ./build_setup/build_setup: line 3: aclocal: command not found./build_setup: line 4: autoheader: command not found./build_setup: line 5: automake: Command not found./build_setup: Line 6: autoreconf: command not foundCopy the code

If the preceding error occurs, install autoTools

# yum -y install aclocal autoheader automake autoreconf
Copy the code

Continue to

# ./configure
# make && make install
Copy the code

Finally, copy related configuration files to the default system path

# mkdir /etc/keepalived
# cp ./keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
# cp ./keepalived/etc/init.d/keepalived /etc/init.d/
# cp ./keepalived/etc/sysconfig/keepalived /etc/sysconfig/
Copy the code

Modify/usr/lib/systemd/system/keepalived. PIDFile values in the service of/var/run/keepalived pid.

Configuration Keepalived

Keepalived offers two modes

  • preemptive: on the MASTER and BACKUP nodesstateIf the MASTER node goes down, the BACKUP node takes over the VIP and service of the MASTER node. If the MASTER node recovers, the BACKUP node takes over the VIP and service of the MASTER node.
  • The preemptive: on the MASTER and BACKUP nodesstateConfiguration isBACKUPAnd, invrrp_instanceAdd both nodes under the blocknopreempt“, indicating no competition for VIP. The default value for both nodes isBACKUPStatus, after both parties send multicast messages, they will elect one according to the priorityMASTERCome out, because both are configurednopreempt, soMASTERRecovering from a failure does not preempt viPs, it doesAvoid service delays caused by VIP switching.

The MASTER node

First, let’s confirm the network card and IP

# ip addr show | grep inetInet 127.0.0.1/8 scope Host LO INET6 ::1/128 Scope Host INET 10.10.0.11/8 BRD 10.255.255.255 scope Global NoPrefixRoute ens192 inet6 fd08:815:48b2::e91/128 scope global noprefixroute inet6 fd08:815:48b2:0:d419:f3f5:85de:b72/64 scope global noprefixroute inet6 fe80::49a2:321d:8cf6:651a/64 scope link noprefixrouteCopy the code

Ens192 (IP: 10.10.0.11) is used. Then edit keepalived configuration file

# vim /etc/keepalived/keepalived.conf
Copy the code

The configuration is as follows:

! Configuration File for Keepalived global_defs {# email recipient notification_email {[email protected] [email protected] [email protected]} # email Notification_email_from [email protected] # email SMTP server IP address smtp_server 192.168.200.1 smtp_connect_TIMEOUT 30 # Indicates the ID of the local node. The value is usually hostname router_id akiya01 Vrrp_skip_check_adv_addr VRrp_strict vrrp_garp_interval 0 VRrp_gna_interval 0} # Vrrp_instance VI_1 {# Specifies a keepalived role, "MASTER" indicates that the host is the primary server, "BACKUP" indicates that the host is the secondary server. Ens192 interface ENS192 # Virtual route id, this id is a number, the same VRRP instance uses the unique id # Virtual_router_id 51 # define priority; In the same vrrP_instance, the priority of "MASTER" must be greater than that of "BACKUP". Priority 100 Advert_int 1 # unit is the second set the authentication type and password authentication {# set authentication type, mainly have two PASS and AH the auth_type PASS # set authentication codes, under the same vrrp_instance, MASTER and BACKUP must use the same password to communicate properly auth_pass akiya} # enable email notification in case of failure #smtp_alert # Disable preemption of service # By default, when the MASTER service is down, # if nopreempt is configured for MASTER, then nopreempt is configured for MASTER. # If nopreempt is configured for MASTER, then nopreempt is configured for MASTER, then nopreempt is configured for MASTER. Services are no longer preempted. #nopreempt # virtual IP, both nodes must be set the same. Virtual_ipaddress {# virtual_ipAddress = 10.10.0.10/8; The binding interface is ENS192. 10.10.0.10/8 dev ens192 label HA :net}}Copy the code

The BACKUP node

The BACKUP configuration is basically the same as the Master configuration except some changes

  • The state role forBACKUP
  • Interface indicates the ID of a nic. Set this parameter based on actual conditions
  • Virtual_route_id to andMASTERThe default value is 51
  • Priority thanMASTERsmall

Modify the Keepalived configuration of the BACKUP node. The deployment configuration is as follows:

! Configuration File for keepalived ... Rrp_instance VI_1 {# specify Keepalived role, BACKUP indicates that the host is the standby node "MASTER" and "BACKUP" must be the same as virtual_router_id 51 # priority, Priority 99 advert_int 1 authentication {auth_type PASS auth_pass 1111} # Virtual IP address. Virtual_ipaddress {# virtual_ipAddress = 10.10.0.10/8; The binding interface is ENS192. 10.10.0.10/8 dev ens192 label HA :net}}Copy the code

Start the service

With the MASTER and BACKUP nodes configured, we can start and test the service

Adding firewall Rules

VRRP uses the multicast address 224.0.0.18

#Firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface ens192 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
#Firewall-cmd --direct --permanent --add-rule ipv4 filter OUTPUT 0 --out-interface ens192 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
# firewall-cmd --reload
Copy the code

View the rules

# firewall-cmd --direct --get-rules ipv4 filter INPUT
0 --in-interface ens192 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
# firewall-cmd --direct --get-rules ipv4 filter OUTPUT
0 --out-interface ens192 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
Copy the code

Starting Keepalived

Start Keepalived and add it to boot automatically

# systemctl start keepalived
# systemctl enable keepalived
Copy the code

Then we look at the IP address of the MASTER node again and see that there is a new IP address

# ip addr show | grep inetInet 127.0.0.1/8 scope Host LO INET6 ::1/128 Scope Host INET 10.10.0.11/8 BRD 10.255.255.255 scope Global NoPrefixRoute Ens192 INET 10.10.0.10/32 Scope Global HA :net INet6 FD08:815:48B2 :: E91/128 Scope Global noPrefixRoute inet6 fd08:815:48b2:0:d419:f3f5:85de:b72/64 scope global noprefixroute inet6 fe80::49a2:321d:8cf6:651a/64 scope link noprefixrouteCopy the code

The IP address displayed on the BACKUP node is

# ip addr show | grep inetInet 127.0.0.1/8 scope Host LO INET6 ::1/128 Scope Host INET 10.10.0.12/8 BRD 10.255.255.255 scope Global NoPrefixRoute ens192 inet6 fd08:815:48b2::1ca/128 scope global noprefixroute inet6 fd08:815:48b2:0:b840:33aa:f6de:253b/64 scope global  noprefixroute inet6 fe80::a96d:fe89:d95:3dfd/64 scope link noprefixrouteCopy the code

Test Keepalived

Install the tcpdump tool

# yum -y install tcpdump
Copy the code

Run the following command on the MASTER node

# tcpdump -i ens192 vrrp -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
10:10:24.193943 IP 10.10.0.11 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
10:10:25.194972 IP 10.10.0.11 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
10:10:26.196009 IP 10.10.0.11 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
10:10:27.197038 IP 10.10.0.11 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
...
Copy the code

If Keepalived is disabled on MASTER, there is no packet to catch and VIP will drift to BACKUP accordingly.

To configure the log

Keepalived default logs are output to /var/log/messages, because there are many system logs, which are relatively troublesome to query.

We can separate the Keepalived log, here we need to modify the log path.

  1. Modify Keepalived configuration
# vim /etc/sysconfig/keepalived
Copy the code

The changes are as follows:

# Options for keepalived. See `keepalived --help' output and keepalived(8) and # keepalived.conf(5) man pages for a list  of all options. Here are the most # common ones : # # --vrrp -P Only run with VRRP subsystem. # --check -C Only run with Health-checker subsystem. # --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop. # --dont-release-ipvs -I Dont remove IPVS topology on daemon stop. # --dump-conf -d Dump the configuration data. # --log-detail -D Detailed log messages. # --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON) # KEEPALIVED_OPTIONS="-D"Copy the code

Change KEEPALIVED_OPTIONS=” -d “to KEEPALIVED_OPTIONS=” -d -d -s 0″, -s specifies the syslog facility

  1. Modify the/etc/rsyslog.conf, added at the end
. local0.* /var/log/keepalived.logCopy the code
  1. Restart the logging service
# systemctl restart rsyslog
Copy the code
  1. Restart Keepalived
# systemctl restart keepalived
Copy the code
  1. See the log
# ls -lh /var/log/keepalived.log-rw-------. 1 root root 14K 9月 30 13:22 /var/log/keepalive.log# head -n 10 /var/log/keepalived.logSep 30 13:22:52 master Keepalived[30707]: Starting Keepalived V1.3.5 (03/19,2017), Git commit V1.3.5-6-G6FA32f2 Sep 30 13:22:52 Master Keepalived[30707] Opening file '/etc/keepalived/keepalived.conf'. Sep 30 13:22:52 master Keepalived[30708]: Starting Healthcheck child process, pid=30709 Sep 30 13:22:52 master Keepalived[30708]: Starting VRRP child process, pid=30710 Sep 30 13:22:52 master Keepalived_healthcheckers[30709]: Initializing ipvs Sep 30 13:22:52 master Keepalived_healthcheckers[30709]: Opening file '/etc/keepalived/keepalived.conf'. Sep 30 13:22:52 master Keepalived_healthcheckers[30709]: ------< Global definitions >------ Sep 30 13:22:52 master Keepalived_healthcheckers[30709]: Router ID = ha01 Sep 30 13:22:52 master Keepalived_healthcheckers[30709]: Smtp server = 192.168.200.1 Sep 30 13:22:52 Master Keepalived_healthcheckers[30709]: Smtp server port = 25Copy the code

Keepalived+Nginx

In a real situation, if the service is stopped while Keepalived service is still working, the VIP cannot find the corresponding service. In this case, you need to write a daemon script. The following uses Nginx as an example.

Nginx installation

  • Add the corresponding Nginx source
# rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
Copy the code
  • Yum install Nginx
# yum -y install nginx
Copy the code
  • NGINX starts, stops, restarts, and starts automatically
# systemctl start nginx   # Start Nginx service
# systemctl stop nginx    # Stop Nginx service
# systemctl restart nginx Restart the Nginx service
# systemctl enable nginx  # Start Nginx service
# nginx -t                Check whether the configuration file is correct
# nginx -s reload         # Smooth overloaded configuration
Copy the code
  • Check whether the startup is successful
# curl -i localhost
Copy the code

Create Nginx service detection scripts

Create the nginx_check.sh script in the /etc/keepalived directory of the active and standby servers as follows:

#! /bin/bash
# author:akiya
A=$(ps -C nginx --no-header | wc -l)
if [ $A -eq0];then
    systemctl start nginx
    sleep 2
    if [ $(ps -C nginx --no-header | wc -l) -eq0];then
        systemctl stop keepalived
    fi
fi
Copy the code

Add executable permissions to the script

# chmod +x /etc/keepalived/nginx_check.sh
Copy the code

This script is used by Keepalived to periodically check the status of Nginx service. If Nginx stops, it will try to restart Nginx. If the start fails, Keepalived service will stop, making IP drift to the standby node.

Modify Keepalived configuration

In the/etc/keepalived/keepalived. Conf add test script configuration

global_defs { ... }... Keepalived executes the script periodically and analyzes the result of the script execution, dynamically adjusting the priority of vrrp_instance # If the result of the script execution is 0 and the weight configuration is greater than 0, the priority is increased accordingly. If the script execution result is non-zero and the weight configuration is less than zero, the priority is reduced accordingly. In other cases, maintain the original configured priority, that is, the value corresponding to priority in the configuration file. Vrrp_script chk_nginx {script "/etc/keepalived/nginx_check.sh" interval 2 # Check nginx status every 2 seconds weight-20 # failed, Set your own priority to -20} vrrp_instance VI_1 {... Virtual_ipaddress {10.10.0.10/8 dev ens192 label ha:net} track_script {# Nginx chk_nginx}}Copy the code

Problem processing

Unable to access script

After compiling vrrp_script block in the configuration file, I encountered a problem Unable to access script when starting the service. After checking the information, I found that this problem was mentioned in Git Issues, and the new version has been solved now.

Some error logs are as follows:

Sep 30 14:25:42 master Keepalived_vrrp[30930]:     chk_nginx no match, ignoring...
Sep 30 14:26:04 master Keepalived_vrrp[30944]:     nginx_check no match, ignoring...
Sep 30 14:44:18 master Keepalived_vrrp[30980]: Unable to access script `/etc/keepalived/nginx_check.sh`
Sep 30 14:44:18 master Keepalived_vrrp[30980]: Disabling track script chk_nginx since not found
Copy the code

If you use yum to install the package, check the package information before installation

# yum info keepalived
Copy the code

default user…

After compiling the installation (installation version 2.0.18), adding the Nginx detection script and starting the Keepalived service, Default user ‘keepalived_script’ for script execution does not exist – please create.

Solution: Add the user or group that runs the detection script to the configuration file

! Configuration File for keepalived

global_defs {
...
   script_user root
   enable_script_security
}
...
Copy the code