In the girlfriend’s repeatedly questioning, so there is this article!!
The cluster structures,
If you don’t know docker, you can learn it by yourself
- The network required to create RabbitMQ
docker network create rabbtimane
Copy the code- Install three RabbitMQ components, one active and two slave
# Three commands are executed in batches. If the download is slow, it is recommended to replace ali Cloud Docker image to accelerate
docker run -d --name=rabbitmq_master -p 5672:5672 -p 15672:15672 -e RABBITMQ_NODENAME=rabbitmq_master -e RABBITMQ_ERLANG_COOKIE='1111111'-h rabbitmq_master --net=rabbtimanet -v ~/docker/rabbitmq_master/data:/var/lib/rabbitmq -e RABBITMQ_DEFAULT_USER=admin - e RABBITMQ_DEFAULT_PASS = admin rabbitmq: 3.7.7 - management
docker run -d --name=rabbitmq_slave1 -p 5673:5672 -p 15673:15672 -e RABBITMQ_NODENAME=rabbitmq_slave1 -e RABBITMQ_ERLANG_COOKIE='1111111'-h rabbitmq_slave1 --net=rabbtimanet -v ~/docker/rabbitmq_slave1/data:/var/lib/rabbitmq -e RABBITMQ_DEFAULT_USER=admin - e RABBITMQ_DEFAULT_PASS = admin rabbitmq: 3.7.7 - management
docker run -d --name=rabbitmq_slave2 -p 5674:5672 -p 15674:15672 -e RABBITMQ_NODENAME=rabbitmq_slave2 -e RABBITMQ_ERLANG_COOKIE='1111111'-h rabbitmq_slave2 --net=rabbtimanet -v ~/docker/rabbitmq_slave2/data:/var/lib/rabbitmq -e RABBITMQ_DEFAULT_USER=admin - e RABBITMQ_DEFAULT_PASS = admin rabbitmq: 3.7.7 - management
Copy the code- Enable the management plug-in of the three components
docker exec -it rabbitmq_master bash
rabbitmq-plugins enable rabbitmq_management
exit
docker exec -it rabbitmq_slave1 bash
rabbitmq-plugins enable rabbitmq_management
exit
docker exec -it rabbitmq_slave2 bash
rabbitmq-plugins enable rabbitmq_management
exit
Copy the code-
The browser accesses the following addresses with the user name and password admin
http://localhost:15672/#/ http://localhost:15673/#/ http://localhost:15674/#/ If the login is successful, the component is installed successfully, but this is not cluster mode, we need to manually join the cluster
-
The secondary node is added to the primary node to implement cluster mode
# salve1 joins the cluster
docker exec -it rabbitmq_slave1 bash
rabbitmqctl stop_app
# node name @user name
rabbitmqctl join_cluster rabbitmq_master@rabbitmq_master
abbitmqctl start_app
exit
# salve2 joins the cluster
docker exec -it rabbitmq_slave2 bash
rabbitmqctl stop_app
# node name @user name
rabbitmqctl join_cluster rabbitmq_master@rabbitmq_master
abbitmqctl start_app
exit
Copy the codeNow open the admin page again and find the cluster mode appears:
So far, the cluster mode has been set up, but this is not enough. RabbitMQ builds normal clusters by default
Common mode cluster
Normal-mode clusters do not copy the queue created to other nodes, although we can see the node on other nodes; This is because the other nodes only keep a copy of the queue’s metadata, showing us that the actual queue still exists on the node that created it; If we close the node that created the queue, the queue will not be available. To solve this problem rabbitMQ provides another clustering mode: mirroring mode
Mirror mode
Mirror mode is an addable policy mode provided by Rabbit. If the created queue matches the created policy, the queue is synchronized to other nodes instead of just copying metadata.
-
There are two ways to add a policy pattern
-
Name: indicates the policy name. If the created queue name matches the wildcard, the policy will be used. For example, ^ Matches all queues Definition: Setting the replication mode The configuration I set here is as follows
- You can add it by command
The following command can be executed on any node, in this case from node 1, and it will be automatically synchronized across the cluster
docker exec -it rabbitmq_slave1 bash
Copy the code
Rabbitmqctl set_policy my_ha “^” ‘{“ha-mode”:”all”}’ exit ‘rabbitmqctl set_policy my_ha “^” ‘{“ha-mode”:”all”}’ exit’
SpringBoot integrates RabbitMQ clusters
- Add the dependent
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
Copy the code- Creating a Configuration File
spring:
rabbitmq:
# host: 127.0.0.1
# port: 5672
Address = host+port = host+port
Addresses: 127.0.0.1:5672127.00 0.1:5673127.00 0.1:5674
# virtual-host: my_vhost
username: admin
password: admin
publisher-returns: true
publisher-confirm-type: simple
Copy the codeSpecific projects can see https://gitee.com/yangleliu/code_learning.git
Some issues with SpringBoot directly integrating RabbitMQ clusters
The rabbitMQ cluster has been integrated by SpringBoot, but the following problems have been detected:
- Springboot will always use one connection, even if three addresses are configured
- Attempts to connect to other nodes are made only after the connected node hangs up
“Summary” : If the cluster mode is configured, but the client is still connected to the same node, there will be a waste of resources and a huge load pressure on a single node, that is, “load imbalance”. Springboot does not have a solution, but we can use the load balancing framework “HAProxy” to help us achieve this
RabbitMQ implements load balancing with HAProxy
HAProxy is what
HAProxy provides high availability, load balancing, and proxy based on TCP and HTTP applications, and supports virtual hosting. It is a free, fast, and reliable solution. HAProxy is especially useful for heavily loaded Web sites that require session persistence or seven-tier processing. HAProxy runs on current hardware and can support tens of thousands of concurrent connections. And its operating mode makes it easy and secure to integrate into your current architecture, while protecting your Web server from being exposed to the network.
Integrated HAProxy
I’ll use Docker to install HAProxy, because it’s really nice to use Dokcer to install software.
- Adding a Configuration File
mkdir ~/docker/haproxy/5677
mkdir ~/docker/haproxy/5678
Modify configuration file 1
vim ~/docker/haproxy/5677/haproxy.cfg
#logging options
global
log 127.0.0.1 local0 info
maxconn 5120
chroot /usr/local/etc/haproxy
uid 99
gid 99
daemon
quiet
nbproc 20
pidfile /var/run/haproxy.pid
defaults
log global
# Use layer 4 proxy mode. "mode HTTP" is set to layer 7 proxy mode
mode tcp
#if you set mode to tcp,then you nust change tcplog into httplog
option tcplog
option dontlognull
retries 3
option redispatch
maxconn 2000
timeout connect 5s
If the client timeout period is 60 seconds, HA initiates the reconnection mechanism
timeout client 60s
If the server connection timeout is 15 seconds, HA initiates reconnection
timeout server 15s
#front-end IP for consumers and producters
listen rabbitmq_cluster
bind 0.0.0.0:5677
Configure the TCP mode
mode tcp
#balance url_param userid
#balance url_param session_id check_post 64
#balance hdr(User-Agent)
#balance hdr(host)
#balance hdr(Host) use_domain_only
#balance rdp-cookie
#balance leastconn
#balance source //ip
# Simple polling
balance roundrobin
Rabbitmq cluster node configuration
Inter performs a health check on the MQ cluster every five seconds, verifies that the server is available twice, and verifies that the server is unavailable twice, and configures the active/standby mechanism
Server rabbitmq_master 192.168.0.103:5672 Check inter 5000 Rise 2 Fall 2
Server rabbitmq_salve1 192.168.0.103:5673 Check inter 5000 rise 2 Fall 2
Server rabbitmq_salve2 192.168.0.103:5674 Check Inter 5000 rise 2 Fall 2
Configure haProxy web monitoring to view statistics
listen stats
bind 0.0.0.0:8100
mode http
option httplog
stats enable
# set haproxy monitor address to http://localhost:8100/rabbitmq-stats
stats uri /rabbitmq-stats
stats refresh 5s
stats auth admin:123456
listen rabbitmq_admin Listen for port 8000 to be forwarded to rabbitMQ clients
bind 0.0.0.0:8000
Server rabbitmq_master 192.168.0.103:15672 Check inter 5000 Rise 2 Fall 2
Server rabbitmq_salve1 192.168.0.103:15673 Check inter 5000 rise 2 Fall 2
Server rabbitmq_salve2 192.168.0.103:15674 Check Inter 5000 rise 2 Fall 2
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
Modify configuration file 2
vim ~/docker/haproxy/5678/haproxy.cfg
#logging options
global
log 127.0.0.1 local0 info
maxconn 5120
chroot /usr/local/etc/haproxy
uid 99
gid 99
daemon
quiet
nbproc 20
pidfile /var/run/haproxy.pid
defaults
log global
# Use layer 4 proxy mode. "mode HTTP" is set to layer 7 proxy mode
mode tcp
#if you set mode to tcp,then you nust change tcplog into httplog
option tcplog
option dontlognull
retries 3
option redispatch
maxconn 2000
timeout connect 5s
If the client timeout period is 60 seconds, HA initiates the reconnection mechanism
timeout client 60s
If the server connection timeout is 15 seconds, HA initiates reconnection
timeout server 15s
#front-end IP for consumers and producters
listen rabbitmq_cluster
bind 0.0.0.0:5678
Configure the TCP mode
mode tcp
#balance url_param userid
#balance url_param session_id check_post 64
#balance hdr(User-Agent)
#balance hdr(host)
#balance hdr(Host) use_domain_only
#balance rdp-cookie
#balance leastconn
#balance source //ip
# Simple polling
balance roundrobin
Rabbitmq cluster node configuration
Inter performs a health check on the MQ cluster every five seconds, verifies that the server is available twice, and verifies that the server is unavailable twice, and configures the active/standby mechanism
Server rabbitmq_master 192.168.0.103:5672 Check inter 5000 Rise 2 Fall 2
Server rabbitmq_salve1 192.168.0.103:5673 Check inter 5000 rise 2 Fall 2
Server rabbitmq_salve2 192.168.0.103:5674 Check Inter 5000 rise 2 Fall 2
Configure haProxy web monitoring to view statistics
listen stats
bind 0.0.0.0:8101
mode http
option httplog
stats enable
# set haproxy monitor address to http://localhost:8100/rabbitmq-stats
stats uri /rabbitmq-stats
stats refresh 5s
stats auth admin:123456
listen rabbitmq_admin Listen for port 8000 to be forwarded to rabbitMQ clients
bind 0.0.0.0:8001
Server rabbitmq_master 192.168.0.103:15672 Check inter 5000 Rise 2 Fall 2
Server rabbitmq_salve1 192.168.0.103:15673 Check inter 5000 rise 2 Fall 2
Server rabbitmq_salve2 192.168.0.103:15674 Check Inter 5000 rise 2 Fall 2
Copy the code- Start two HaProxies
docker run --name rabbitmq-haproxy5677 -p 5677:5677 -p 8100:8100 -p 8000:8000 -d --net=rabbtimanet -v ~/docker/haproxy/5677/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro haproxy:latest
docker run --name rabbitmq-haproxy5678 -p 5678:5678 -p 8101:8101 -p 8001:8001 -d --net=rabbtimanet -v ~/docker/haproxy/5678/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro haproxy:latest
Copy the codeI installed two haProxies for high availability, in case one of them fails and the external service cannot be provided.
The springboot configuration file is modified as follows:
spring:
rabbitmq:
# host: 127.0.0.1
# port: 5672
Addresses: 127.0.0.1:5677127.00 0.1:5678
# virtual-host: my_vhost
username: admin
password: admin
publisher-returns: true
publisher-confirm-type: simple
Copy the codeHere we have changed to connect to two haProxies, test it by yourself, the code has been uploaded to the code base
conclusion
Installing RabbitMQ can be tricky and requires a variety of environmental issues; But fortunately, I use Docker here, so I can complete the installation smoothly. Then put the main time on the debugging cluster above, the installation process also encountered a lot of pits, we suggest that in accordance with this practice, after all, only their own through these pits, will let their own behind will not encounter pits ah, if you have encountered can not solve the problem, you can private message me oh
There will be a large number of interview materials and architect must-see books waiting for you to choose, including Java foundation, Java concurrency, micro services, middleware and more information waiting for you to take oh.