preface

Cluster is a common method to solve the problem of high concurrency and mass data. When a server’s processing capacity, storage space is insufficient, do not attempt to change a more powerful server, for large websites, no matter how powerful the server, can not meet the continuous growth of the business needs of the website. In this case, it is more appropriate to add a server to share the access and storage burden of the original server.

Through the load balancing scheduling server, the access request from the browser is distributed to any server in the application server cluster. If there are more users, more application servers will be added to the cluster, so that the load pressure of the application server will no longer become the bottleneck of the whole website.

From “Technical Architecture for Large Web sites – Core Principles and Case Studies”

Environment to prepare

  • 192.168.0.221: Nginx + Keepalived Master

  • 192.168.0.222: nginx + Keepalived backup

  • 192.168.0.223: tomcat

  • 192.168.0.224: tomcat

  • Virtual IP address (VIP):192.168.0.200, also known as the floating IP address

The relationship between the components is shown below:

Tomcat is used as the application server

The installation of Tomcat is not within the scope of this blog. For details, please refer to virtualBox to install centos and set up Tomcat. Remember to put your own application under tomcat webapps. Then the IP in index.jsp needs to be its own


Start tomcat on 192.168.0.223 and 192.168.0.224. The tomcat path may be different from mine, so you need to write it as your own

# cd /usr/local/tomcat7/bin
# ./startup.sh
Copy the code

Visit myWeb below

Nginx does load balancing

Nginx installation is not covered in this article.

Nginx. conf contains the following contents

user  root;            # run user
worker_processes  1;        Start the process, usually set to equal the number of cpus

Global error log and PID file
error_log  /usr/local/nginx/logs/error.log;
error_log  /usr/local/nginx/logs/error.log  notice;
error_log  /usr/local/nginx/logs/error.log  info;
pid        /usr/local/nginx/logs/nginx.pid;

# Working mode and connection number online
events 
{
    use epoll;            #epoll is a way of Multiplexing IO(I/O Multiplexing), but only for linux2.6 + kernels, can greatly improve nginx performance

    worker_connections  1024;    # Maximum number of concurrent links for a single background worker process
}

Configure the HTTP server to provide load balancing support with its reverse proxy capabilities
http 
{
    include       mime.types;
    default_type  application/octet-stream;

    Set request buffering
    server_names_hash_bucket_size  128;
    client_header_buffer_size   32K;
    large_client_header_buffers  4 32k;
    # client_max_body_size 8m;

    # sendFile specifies whether nginx calls sendfile (zero copy) to output files. For common applications,
    If it is used for applications such as downloads, set it to off to balance disk and network I/O processing speeds and reduce uptime.
    sendfile        on;
    tcp_nopush     on;
    tcp_nodelay    on;

    Connection timeout
    keepalive_timeout  65;

    # Enable gzip compression to reduce transmission traffic
    gzip  on;
    gzip_min_length    1k;
    gzip_buffers    4 16k;
    gzip_http_version  1.1;
    gzip_comp_level  2;
    gzip_types  text/plain application/x-javascript text/css  application/xml;
    gzip_vary on;

    Add tomcat to the list where real application servers are stored
    upstream tomcat_pool 
    {
       The larger the weight, the more likely it is to be assigned.Server 192.168.0.223:8080 weight=4 max_fails=2 fail_timeout=30s; Server 192.168.0.0.2:8080 weight=4 max_fails=2 fail_timeout=30s; } server { listen 80;# monitor port
        server_name  localhost;

    # Default request Settings
    location / {
        proxy_pass http://tomcat_pool;    Switch to Tomcat processing
    }

    All JSP pages are handled by Tomcatlocation ~ \.(jsp|jspx|dp)? $ { proxy_set_header Host$host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://tomcat_pool;    Switch to Tomcat processing
    }

    # all static files are read directly by Nginx without Tomcatlocation ~ .*\.(htm|html|gif|jpg|jpeg|png|bmp|swf|ioc|rar|zip|txt|flv|mid|doc|ppt|pdf|xls|mp3|wma)$ { expires 30d; } location ~ .*\.(js|css)? $ { expires 1h; }# Define error pageerror_page 500 502 503 504 /50x.html; location = /50x.html { root html; }}}Copy the code

Nginx. conf configuration can be complex or simple. You can configure the nginx.conf configuration according to your own situation.

Once configured, start nginx and write your own path

# cd /usr/local/nginx/sbin
# ./nginx
Copy the code

To access nginx, it looks like this:

Two Nginx servers are working properly, there is no master slave, they are of the same level, when configured keepalived there is a master slave.

Keepalived Implement Nginx High Availability (HA)

The installation of Keepalived is not covered in this article. Keepalived function in fact has been reflected in the first picture, mainly play two roles: to achieve the MAPPING of VIP to local IP; And checking nginx status.

Keepalive.conf on master reads as follows:

global_defs {
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server smtp.hysec.com
    smtp_connection_timeout 30
    router_id nginx_master        # Set the nginx master ID, which should be unique on a network
}
vrrp_script chk_http_port {
    script "/usr/local/src/check_nginx_pid.sh"    Finally, execute the script manually to ensure that the script works
    interval 2                          # check the interval between script execution, in seconds
    weight 2
}
vrrp_instance VI_1 {
    state MASTER            Keepalived specifies a keepalived role, MASTER as primary, BACKUP as BACKUP
    interface eth0            Network interface card (current centos nic)
    virtual_router_id 66        # Indicates the number of the virtual route
    priority 100            # priority, the higher the value, the higher the priority of obtaining and processing requests
    advert_int 1            Check interval, default is 1s(VRRP multicast cycle seconds)
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
    chk_http_port            # (call detection script)} virtual_ipaddress {192.168.0.200# Define virtual IP (VIP), can be set more than one, each line one}}Copy the code

Keepalive.conf on backup reads as follows:

global_defs {
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server smtp.hysec.com
    smtp_connection_timeout 30
    router_id nginx_backup              # Set nginx backup ID that should be unique on a network
}
vrrp_script chk_http_port {
    script "/usr/local/src/check_nginx_pid.sh"
    interval 2                          # check the interval between script execution
    weight 2
}
vrrp_instance VI_1 {
    state BACKUP                        Keepalived specifies a keepalived role, MASTER as primary, BACKUP as BACKUP
    interface eth0                      Network interface card (current centos nic)
    virtual_router_id 66                # Indicates the number of the virtual route
    priority 99                         # priority, the higher the value, the higher the priority of obtaining and processing requests
    advert_int 1                        Check interval, default is 1s(VRRP multicast cycle seconds)
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_http_port                   # (call detection script)} virtual_ipaddress {192.168.0.200# Define virtual IP (VIP), can be set more than one, each line one}}Copy the code

The nginx check_nginx_pid.sh script contains the following contents:

#! /bin/bash
A=`ps -C nginx --no-header |wc -l`        
if [ $A -eq0];then                            
    /usr/local/nginx/sbin/nginx                # restart nginx
    if [ `ps -C nginx --no-header |wc -l` -eq0];then    #nginx restart failed
        exit 1
    else
        exit 0
    fi
else
    exit 0
fi
Copy the code

Starting keepalived

# service keepalived start
Copy the code

Visit VIP, the effect is as follows:

Let’s take a look at keepalived’s logs

Master (192.168.0.221) :

Backup (192.168.0.222) :

Once we’ve stopped keepalived on the master (simulated downtime), take a look at the Keepalived log

Master (192.168.0.221) :

Original backup (192.168.0.222) :

VIP can access the service normally, the front-end request can not feel the switch of the back-end Nginx; Reawaken the original master (192.168.0.221) test will not be performed here

Pay attention to the point

/bin/sh^M: bad interpreter: Does not have that file or directory

Because the operating system is Windows, I edit the script under Windows, so there may be invisible characters. The script file is in DOS format, that is, the end of each line is identified by a carriage return character and a newline character, with ASCII codes 0x0D and 0x0A respectively. There are many ways to see if the file is in DOS, UNIX, or MAC format

Solutions:

vim filename
:set ff? You can see the words DOS or Unix. If it is indeed in DOS format.
:set ff=unix Force it to Unix and save it to exit.
Copy the code

Run the script again.

Be careful when editing files from Windows and then copying them to Linux. In addition, scripts need to be given executable permissions to execute them. An intuitive representation of executable files is the green color of the files themselves.

2, load balancing it is best to test multiple browsers, some browsers will cache, there will be no load balancing effect.

For example, in my test, Google Browser had a similar situation (it is not clear whether the cache is the reason), Firefox, 360, IE are normal load balancing effect.

3, request direction

Access virtual IP(VIP), Keepalived maps the request to local Nginx, which forwards the request to Tomcat, for example: http://192.168.0.200/myWeb/, are mapped into http://192.168.0.221/myWeb/, port is 80, and 221 in nginx port is 80; Once mapped to Nginx, nginx forwards the request.

Keepalived server IP state

Vips are always on one or only one of keepalived servers; Keepalived transfers the VIP to backup and promotes backup to Master when the VIP server goes down.

4. VIP, also known as the floating IP address, is a public IP address that maps to the domain name to provide external services. Other IP addresses are generally Intranet IP addresses and cannot be accessed directly from external devices