1, the preface

Message queue middleware is an important component in distributed system, mainly solving application coupling, asynchronous message, traffic cutting and other problems. It can achieve high performance, high availability, scalable and ultimately consistent architecture, and is an indispensable middleware for large distributed systems.

Message queue plays a key role in e-commerce system, message communication, log collection and other applications. For example, Alibaba’s RocketMQ (message queue), developed by alibaba, has supported trillions of data flood peaks in previous Tmall “Double 11” activities, providing a strong guarantee for large-scale transactions.

As an important means to improve application performance, distributed message queue technology has been paid more and more attention in the field of Internet. This article introduces four commonly used distributed message queue open source software: Kafka, ActiveMQ, RabbitMQ, and RocketMQ.

The table of contents is as follows:

  • Kafka

  • ActiveMQ

  • RabbitMQ

  • RocketMQ

  • Comparison of several message queues

  • reference

2, Kafka

In the world of distributed message queues, Kafka plays a very important role because of its excellent performance. It was originally developed by LinkedIn, which contributed to the Apache Foundation in 2010 and has since become a top open source project.

Kafka profile

Kafka is a Kafka that has been copied, edited, and interpreted in many different ways, so let’s take a look at the definition:

Apache Kafka is a distributed streaming platform.

Kafka as a streaming platform has three capabilities:

  1. Publish and subscribe record flows, similar to message queues or enterprise messaging systems;

  2. It has fault tolerance and can store record streams in a persistent manner.

  3. When a flow of records is generated (occurs), it can be processed in a timely manner.

Kafka is suitable for two types of applications:

  1. Establish real-time streaming data pipelines to reliably obtain data between systems or applications;

  2. Build real-time streaming applications that transform or react to data streams.

Kafka contains four core apis.

  • The Producer API: Based on this API, applications can publish record streams to one or more Kafka Topics;

  • Consumer API: Based on this API, an application can subscribe to one or more topics and process the record flow corresponding to the topics;

  • Streams API: Based on this API, applications can effectively transform input Streams into output Streams by acting as a stream processor, consuming input Streams from one or more topics and generating output Streams to output one or more topics;

  • Connector API: Allows you to build and run reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector for a relational database might capture every change to a table.

Kafka characteristic

As a high-throughput distributed publish-subscribe messaging system, Kafka has the following features:

  1. Fast persistence, which can persist messages with O(1) overhead;

  2. High throughput, up to 10W/s on a common server;

  3. Fully distributed system, brokers, producers and consumers all automatically support distribution and automatically realize load balancing.

  4. Supports synchronous and asynchronous replication HA.

  5. Support data batch sending and pulling;

  6. Zero-copy: reduces I/O operations.

  7. Data migration and capacity expansion are transparent to users.

  8. The machine can be expanded without stopping;

  9. Other features include strict message ordering, a rich message pull model, efficient subscriber horizontal scaling, real-time message subscription, multi-billion message stacking capability, and a periodic deletion mechanism.

Kafka deployment environment

The operating system

  • Windows: Kafka runs on some Windows systems, but is not officially recommended.

  • Unix: Supports all versions of Unix, as well as Linux and Solaris.

Environmental requirements

  • Java Development Kit (JDK) : The latest version of Kafka is 2.0.0.

  • ZooKeeper: The Kafka cluster depends on ZooKeeper. You need to install ZooKeeper based on the Kafka version (Kafka will be removed from ZooKeeper in the future).

Kafka architecture

Kafka architecture

As shown in the figure above, a typical Kafka architecture consists of producers, brokers (Kafka supports horizontal scaling, the more brokers there are, the higher the cluster throughput), Consumer groups, and producers. And a Zookeeper cluster. Kafka uses Zookeeper to manage cluster configuration, elect the Leader, and Rebalance when the Consumer Group changes. Producer uses Push mode to publish messages to the Broker, and Consumer uses Pull mode to subscribe to and consume messages from the Broker.

Please see the table below for explanations of the terms:

Kafka core components

Kafka high availability solution

The guarantee of Kafka’s high availability comes from its robust Replication policy. To improve throughput, each Topic in Kafka is divided into several Partitions; To ensure availability, multiple Replicas are set for each Partition. To ensure data consistency, the Zookeeper mechanism is introduced. Based on Zookeeper, Kafka finds a node for each Partition to act as the Leader, and the rest of the backups act as followers. Only the Leader can process client requests, while the followers act only as copies to synchronize the Leader’s data. The diagram is as follows: TopicA is divided into two partitions, and two copies are configured for each Partition.

Kafka high availability solution

Based on the above architecture, when a message pushed by Producer is written to a Partition, the Leader’s Broker (Kafka node) writes the message to its own Partition and copies the message to each Follower for synchronization. If a Follower dies, the Leader finds a replacement and synchronizes the message. If the Leader fails, a new Leader is elected from the followers to continue services. This is done by ZooKeeper.

Kafka pros and cons

The advantages mainly include the following:

  1. Rich client language, support Java,.NET, PHP, Ruby, Python, Go and other languages;

  2. Excellent performance, single-machine write TPS of about 1,000,000 / SEC, message size of 10 bytes;

  3. It provides a completely distributed architecture and Replica mechanism, which has high availability and reliability and theoretically supports unlimited accumulation of messages.

  4. Batch operations are supported.

  5. Consumers use Pull mode to obtain messages, messages are orderly, through control can ensure that all messages are consumed and only once;

  6. There are excellent third-party Kafka Web management interface Kafka-Manager;

  7. It is mature in the logging space and is used by multiple companies and open source projects.

The main disadvantages are:

  1. When Kafka single machine has more than 64 queues/partitions, Load will increase significantly. The more queues, the higher the Load, the longer the response time to send messages.

  2. In short polling mode, real-time performance depends on polling interval time.

  3. Retry is not supported when consumption fails.

  4. Message ordering is supported, but when an agent goes down, messages are out of order.

  5. Community updates are slow.

3, ActiveMQ

ActiveMQ is a subproject under Apache. It was placed second because of the description on its website:

❝

Apache ActiveMQ is the most popular and powerful open source messaging and Integration Patterns server.

❞

It does not, immodestly, put it second, as a admonition.

Introduction of ActiveMQ

ActiveMQ is produced by Apache. According to the official website, it is the most popular and powerful open source message bus. ActiveMQ is a JMS Provider implementation that fully supports JMS1.1 and J2EE 1.4 specifications. It is very fast, supports multi-language clients and protocols, and can be easily embedded into the enterprise application environment with many advanced features.

ActiveMQ is developed based on Java language, and the latest version is 5.1.5.6.

Characteristics of ActiveMQ

The Features of ActiveMQ are explained in detail in the Features column on the official website. I have translated them as follows:

  1. Support multiple languages and protocols to write clients. Languages include Java, C, C++, C#, Ruby, Perl, Python, and PHP. Application protocols include OpenWire, Stomp REST, WS Notification, XMPP and AMQP.

  2. Full support for JMS1.1 and J2EE 1.4 specifications (persistence, XA messages, transactions);

  3. Full support for enterprise integration patterns in JMS clients and message brokers;

  4. Support for many advanced features, such as message groups, virtual destinations, wildcards, and composite destinations;

  5. With Spring support, ActiveMQ can be easily embedded into Spring applications and configured using Spring’s XML configuration mechanism.

  6. Having passed the tests of common J2EE servers such as Geronimo, JBoss4, GlassFish, WebLogic, with JCA 1.5 Resource Adaptors configured, You can have ActiveMQ automatically deployed to any J2EE 1.4 compliant commercial server;

  7. Supports multiple transport protocols, such as VM, TCP, SSL, NIO, UDP, Multicast, JGroups, and JXTA.

  8. Support for high-speed message persistence via JDBC and Journal;

  9. From the design to ensure high performance cluster, client-server, point-to-point;

  10. REST apis provide technology-independent and language-based Web apis for messages;

  11. AJAX allows Web flow support for Web browsers using pure DHTML, allowing Web browsers to become part of the messaging structure;

  12. Get support from CXF and Axes so ActiveMQ can easily be embedded in either of the Web services stacks to provide reliable messaging;

  13. It is easy to call the embedded JMS Provider and test it.

ActiveMQ deployment environment

Compared with Kafka, ActiveMQ is much simpler to deploy and supports multiple versions of Windows and Unix systems. In addition, ActiveMQ is developed from the Java language, so it requires JRE support.

Hardware requirements:

  • ActiveMQ 5.x requires 60M space if installed as a binary file. Of course, additional disk space is required to persist messages;

  • If you download the ActiveMQ 5.x source file and compile and build by yourself, you need 300M space.

Operating system:

  • Windows: Supports Windows XP SP2, Windows 2000, Windows Vista, and Windows 7.

  • Unix: Supports Ubuntu Linux, Powerdog Linux, MacOS, AIX, HP-UX, Solaris, or any other Unix platform that supports Java.

Environmental requirements:

  • Java Runtime environment (JRE), version 1.7 or later, if the source code to build, you also need to install JDK;

  • You need to configure an environment variable for the JRE, usually named JAVA_HOME.

  • To build from source, Maven 3.0.0 or later needs to be installed, and the dependent JAR packages need to be added to your classpath.

ActiveMQ architecture

The main architecture of ActiveMQ is shown in the figure below.

ActiveMQ architecture

Transmission protocol: communication between messages undoubtedly requires protocols. Starting an ActiveMQ will open a listening port. ActiveMQ provides a wide range of connection modes, including SSL, STOMP, and XMPP. The default use protocol of ActiveMQ is OpenWire and port number is 61616.

Communication mode: ActiveMQ has two communication modes: point-to-point Model, Publish/Subscribe Model, In the Publich/Subscribe mode, there are two message processing modes: persistent subscription and non-persistent subscription.

Message storage: In practice, important messages often need to be persisted to a database or file system to ensure that information is not lost in the event of a server crash.

Cluster: The most common clustering methods include Network of Brokers and Master Slave.

Monitor: ActiveMQ is generally monitored by JMX.

ActiveMQ in default configuration is only suitable for learning but not for actual production environment. ActiveMQ’s performance needs to be mined through configuration, and its performance improvement includes code-level performance, rule performance, storage performance, network performance and multi-node cooperative method (cluster scheme). Therefore, the central idea of optimizing ActiveMQ is as follows:

  1. Optimize ActiveMQ single node performance, including NIO model selection and storage selection.

  2. Configure the ActiveMQ cluster (ActiveMQ’s high performance and high availability need to be demonstrated through the cluster).

In the production environment, ActiveMQ cluster can be deployed in the following two ways.

  • Master Slave mode: Implements high availability. When the Master server is down, the Slave server becomes the Master server to ensure service continuity.

  • Broker Clusters pattern: Load balancing, where messages are synchronized between multiple brokers to maximize server load.

ActiveMQ high availability solution

ActiveMQ high availability solution

In the production environment, High Availability (HA) is “just needed”. The High Availability architecture of ActiveMQ is based on Master/Slave model. ActiveMQ provides a total of four configuration schemes to configure HA, among which Shared Nothing Master/Slave is no longer used after version 5.8. The ZooKeeper-based LevelDB Store HA scheme was introduced in ActiveMQ 5.9.

For a detailed description of several HA schemes, please refer to the official website. Here, I will give a brief introduction.

Solution 1: Shared Nothing Master/Slave

This is the simplest and most typical master-slave mode. The Master and Slave have their own storage systems and do not share any data. The “Shared Nothing” pattern has many limitations, such as lost messages and “dual masters”. At present, it is almost not applied in the strict production environment, which is a tendency to be eliminated. Therefore, this paper will not introduce it.

Scheme 2: Shared Storage Master/Slave

This is a very common architecture. “Shared storage” means that data is shared between the Master and Slave. To achieve data sharing, there are two ways:

  1. Shared Database Master/Slave

  2. Shared File system Master/Slave

(1) Shared File System Master/Slaves

This is the Master/Slaves model based on shared file systems. The so-called “shared file system” can only be POSIX interface accessible file systems, such as local file systems or SAN distributed shared file systems (such as glusterFS). For the Broker, the file lock of the storage engine will be acquired at startup. If it is successfully acquired, the transportConnector will continue to be initialized, otherwise it will always try to obtain a tryLock. This requires strict assurance that only one process can obtain an exclusive lock at any time for a shared file system. If you choose a SAN file system that does not guarantee this condition, it will not function as a shared storage engine for Master/Slavers.

“Shared File System” is the most commonly used mode with simple architecture, reliability and practicality. All we need is a SAN file system.

(2) JDBC Store Master/Slaves:

Obviously, the data storage engine is Database, ActiveMQ interacts with Database through JDBC, and the exclusive lock uses Database’s table-level exclusive lock. JDBC stores are generally considered inefficient relative to log files, and although data visibility is good, the Database is poorly scalable and does not adapt well to high concurrency, big data (strictly speaking, a single group OF M-S architectures cannot support big data). Besides, the messages of ActiveMQ are usually stored for a short time, frequently written and deleted, which are the influence points of performance. We usually use JDBC Store when studying the storage principle of ActiveMQ, or in small and medium-sized application environments that require high data consistency (reliability and visibility), such as transaction process support system in order system. However, because the JDBC architecture is easy to implement and manage, we still tend to prefer this approach.

Before using the JDBC Store, you must have a stable Database and grant “create table” and plain CRUD permissions to linked users in AcitveMQ. The Master and Slave configuration files are basically the same, and developers need to be aware that brokerName and brokerId are globally non-repeatable. Copy the JDBC-connector Jar package to the ${acitvemq}/lib/optional directory.

Scenario 3: Replicated LevelDB Store

The LevelDB Store, based on replication, is ActiveMQ’s latest HA solution, supported in 5.9+. Compared with the two “Shared Storage” modes in scheme 2, this scheme is more consistent with the “master-slave” model in terms of Storage and communication mechanism.

The Replicated LevelDB also allows for multiple Slaves and has a binding limit on the number of Slaves due to its use of ZooKeeper to elect the Master. For an election to take place, a majority of “participants” are needed. Because there are multiple brokers in the Replicated LevelDB Store, one is elected from multiple brokers to become Master and the others to become slaves. Only the Master receives connections from the Client. The Slave connects to the Master and receives data from the Master (synchronous or asynchronous). Each Broker instance stores message data locally (similar to “Shared Nothing”) and does not share any data between them, so it is not appropriate in a sense to classify the “Replicated LevelDB” as “Shared Storage”.

Special note: ActiveMQ website warns that LevelDB is no longer the recommended storage solution and KahaDB is used instead.

Network Bridges mode of ActiveMQ HA scheme

The HA schemes I have introduced above all have only one Master node in nature, which cannot meet the commercial scenarios with high concurrency and large throughput. Therefore, ActiveMQ officially launched the “bridge” architecture mode, namely the real “distributed message queue”. This pattern can handle scenarios with large Clients and high message increments; It carries applications with a large amount of data in a cluster mode.

Network Bridges mode of ActiveMQ HA scheme

As shown in the figure above, the cluster consists of multiple sub-groups. Each Group is in M-S mode and shares storage. The connection between multiple Groups is established based on the “Network Connector” (master-slave protocol), which is usually two-way connection. All Groups are connected to each other and form a “subscription” relationship between Groups. For example, G2 is logically a subscriber to G1 (the strategy of subscription is to sort the list of consumer destinations on each Broker), and the principle of message forwarding is based on this. For clients, Failover is still supported. The Failover protocol can contain the addresses of “most” nodes in the cluster.

Messages for Topic subscribers are replicated and stored across all groups, and messages for Queue are forwarded between Brokers and eventually reach the node where the Consumer resides.

Producers and Consumers can establish connections and communicate messages with masters in any Group. When the topology of Brokers cluster changes and the number of Producers or Consumers changes, The Clients connection location will be dynamically balanced. Brokers use an “Advisory” mechanism to synchronize connection information between Clients. If a new Consumers joins, the Broker sends an Advisory message (an internal channel) to notify other Brokers.

Clustering provides a better guarantee of availability, but there may be trade-offs in certain features, such as the orderliness of Queue messages breaking because multiple consumers in the same Queue may be in different groups. If a Group is implemented, Messages saved on it will be visible to Clients only if they are restored.

The network forwarding bridge cluster mode is complex to construct and costly to maintain. Therefore, it can be used in production environments.

Advantages and disadvantages of ActiveMQ

The advantages are mainly as follows.

  1. Cross-platform (Java is written platform-independent and ActiveMQ can run on almost any JVM);

  2. You can persist data to a database using JDBC. Although using JDBC degrades ActiveMQ’s performance, databases have always been the storage medium that developers are most familiar with. Store the information in a database so it can be seen and touched. And the company has a special DBA to tune the database, the separation of master and slave;

  3. Support for JMS unified interface;

  4. Support automatic reconnection;

  5. Security mechanism: Supports multiple security configuration mechanisms, such as Shiro and JAAS, to authenticate and authorize Queue/Topic.

  6. A complete monitoring system, including Web Console, JMX, Shell command line, and Jolokia REST API;

  7. User-friendly: The Web Console is available for most of your needs, and there are many third-party components available, such as Hawtio.

Its disadvantages mainly include the following points:

  1. Low community activity, slow update, increased maintenance costs;

  2. Network data show that ActiveMQ has some puzzling problems and will lose messages.

  3. At present, the official focus is on The next generation of ActiveMQ 6.0 product Apollo, and there is less maintenance for 5.x.

  4. Not suitable for applications with thousands of queues.

4, the RabbitMQ

RabbitMQ is open source software developed and supported by RabbitMQ Technologies Ltd. The company was acquired by SpringSource, a division of VMWare, in April 2010. It was incorporated into Pivotal in May 2013. In fact, VMWare, Pivotal, and EMC are all part of the same family. The difference is that VMWare is an independent, publicly traded subsidiary, while Pivotal combines some of EMC’s resources and isn’t currently publicly traded.

Introduction of the RabbitMQ

RabbitMQ is a popular open source message queue system, latest version 3.7.8. RabbitMQ is a standard implementation of the Advanced Message Queuing Protocol (AMQP). Supports a variety of clients, such as Python, Ruby,.NET, Java, JMS, C, PHP, ActionScript, XMPP, STOMP, etc., supports AJAX and persistence. It is used to store and forward messages in distributed systems, and has good performance in ease of use, scalability, and high availability.

RabbitMQ is developed in the Erlang language. Erlang is a general-purpose programming language for concurrent running environments. The language was developed by Ericsson in 1986 to create a programming language and operating environment that could handle large-scale concurrent activities. Erlang was launched in 1987, and after a decade of development, it was released as an open source version in 1998.

Erlang is a structured, dynamically typed programming language with built-in parallel computing support. Application runtimes written in Erlang typically consist of thousands of lightweight processes that communicate with each other through messaging. Interprocess context switching is just one or two steps in Erlang and is much more efficient than thread switching in C programs. The Erlang runtime environment is a virtual machine, sort of like a Java virtual machine, so that once compiled, code can also run anywhere. Its runtime system even allows code to be updated without interruption. In addition, bytecode can also be compiled to run native code.

The RabbitMQ characteristics

RabbitMQ is officially the most widely deployed message broker with the following features:

  • Asynchronous messaging, supporting a variety of messaging protocols, message queues, delivery confirmation mechanisms, flexible routing messages to queues, a variety of exchange types;

  • A good developer experience that runs on many operating systems and cloud environments and provides a variety of development tools for most popular languages;

  • Pluggable identity authentication and authorization, supports Transport Layer Security (TLS) and Lightweight Directory Access Protocol (LDAP). Lightweight and easy to deploy to internal, private, or public clouds;

  • Distributed deployment: supports cluster deployment and cross-region deployment to meet high availability and high throughput application scenarios.

  • There are DEDICATED HTTP-apis, command-line tools, and UIs for administration and oversight;

  • A wide array of tools and plug-ins that support continuous integration, operational measurement, and integration into other enterprise systems. RabbitMQ can be flexibly extended by plug-ins.

RabbitMQ is a “well-developed” message broker system with high performance, security, reliability, distribution, support for multi-language clients and a dedicated o&M management tool.

RabbitMQ deployment environment

RabbitMQ supports multiple versions of Windows and Unix, and ActiveMQ is developed from the Erlang language and therefore requires Erlang environment support. In a sense RabbitMQ has the potential to run on all erlang-enabled platforms, from embedded systems to multi-core clusters and cloud-based servers.

Operating system:

  • Windows series: Support Windows NT, Windows 2000, Windows XP, Windows Vista, Windows 7, Windows 8, Windows Server 2003/2008/2012, Windows 95, Windows 98;

  • Unix family: supports Ubuntu and other debian-based Linux distributions, Fedora and other Linux distributions managed by RPM packages, openSUSE and derived distributions, Solaris, BSD, MacOSX, and more.

Environmental requirements:

  • RabbitMQ is developed using Erlang, which requires installation.

  • Different JDK versions support different versions of Erlang and RabbitMQ Server. You are advised to use older JDK versions to avoid compatibility problems.

The RabbitMQ architecture

According to the official documentation, the RabbitMQ architecture diagram is as follows:

The RabbitMQ architecture

A few important concepts are explained.

  • Broker: Message queue server entity.

  • Exchange: message switch, which specifies the rules by which messages are routed to which queue.

  • Queue: Message Queue carrier, each message is put to one or more queues.

  • Binding: Binds exchanges to queues according to routing rules.

  • Routing Key: The Key used by Exchange to deliver messages.

  • Vhost: Virtual host. Multiple vhosts can be set up within a Broker to separate permissions between different users.

  • Producer: A program that delivers messages.

  • A Consumer is a program that receives messages.

  • Channel: Message Channel. In each connection of the client, multiple channels can be established. Each Channel represents a session task.

The process of using message queues is as follows:

  1. The client connects to the message queue server and opens a Channel.

  2. The client declares an Exchange and sets the related properties.

  3. The client declares a Queue and sets its properties.

  4. The client uses the Routing Key to establish a binding relationship between the Exchange and Queue.

  5. The client posts a message to Exchange. After Exchange receives a message, it routes the message to one or more queues based on the Key and Binding set.

  6. There are three types of Exchange, Direct, Fanout, and Topic, each of which implements a different Routing Algorithm.

Direct Exchange: Deliver by Key. If the Routing Key matches, the Message is passed to the corresponding Queue. When a Queue is created, it will automatically bind Exchange with the name of the Queue as the Routing Key. For example, if the Routing Key is set to ABC during binding, the message submitted by the client will be sent to the queue only when the Key is set to ABC.

Fanout Exchange: This type of Exchange does not require a Key. It operates in broadcast mode, and when a message comes in, it is delivered to all queues bound to the switch.

Topic Exchange: Post keys after pattern matching. Like the symbol “#” matching one or more words, the symbol “.” It fits exactly one word. For example, “ABC.#” matches “abc.def.ghi”, and “ABC.” only matches “abc.def”.

RabbitMQ high availability solution

As far as distributed systems are concerned, the strategy to achieve High Availability (HA) is basically the same, that is, the replica idea. When the primary node goes down, the standby node that acts as the replica quickly “jumps up” to continue providing services. In addition, the throughput of a single machine is extremely limited, and in order to improve performance, “crowd tactics”, also known as the cluster mode, are often used.

The RabbitMQ cluster configuration modes are as follows:

  • Cluster: used for lans on the same network segment but not across network segments. Can dynamically increase or decrease at will; The nodes need to run the same version of RabbitMQ and Erlang between them.

  • Federation: Applied to a wide area network, allowing a switch or queue on a single server to receive messages posted to a switch or queue on another server, either on a single machine or in a cluster. Federation queues are similar to one-way point-to-point connections in that messages are forwarded between Federation queues any number of times until they are accepted by consumers. Federation is commonly used to connect intermediate servers on the Internet for subscription distribution messages or work queues.

  • Shovel: Connects in a similar way to Federation, but works at a lower level. Can be applied to wide area networks.

The RabbitMQ node types are as follows.

  • Memory nodes: Memory nodes store all metadata definitions for queues, switches, bindings, users, permissions, and vhosts in memory, with the benefit of better speeding up operations such as switch and queue declarations.

  • Disk node: Stores metadata on disks. A single-node system allows only disk nodes to prevent configuration information loss during RabbitMQ restart.

RabbitMQ requires at least one disk node in the cluster, and all other nodes can be memory nodes. When a node joins or leaves the cluster, at least one disk node must be notified of the change. If the only disk node in the cluster crashes, the cluster can continue to operate, but cannot perform operations (add, delete, change, or check) until the node recovers.

Solution: Set up two disk nodes, at least one of which is available to hold metadata changes.

Erlang cookies:

The Erlang Cookie is a key that ensures that different nodes can communicate with each other. To ensure that different nodes in the cluster can communicate with each other, they must share the same Erlang Cookie. Specific directory in/var/lib/rabbitmq /. Erlang. Cookies.

Its origins begin with how the rabbitmqctl command works. RabbitMQ is based on the Erlang architecture, so rabbitmqctl starts the Erlang node and connects to the RabbitMQ node using the Erlang system. The correct Erlang Cookie and node name are required during connection, and Erlang nodes exchange Erlang cookies for authentication.

Mirror queue:

RabbitMQ can be grouped into two Cluster modes: common mode and mirrored mode.

  • Common mode: The default cluster mode. The following uses Rabbit01 and Rabbit02 as an example. For a Queue, the message entity exists on only one of the nodes, Rabbit01 (or Rabbit02). Rabbit01 and Rabbit02 have the same metadata, i.e. the structure of the Queue. When a message enters the Rabbit01 Queue and a Consumer consumes it from Rabbit02, RabbitMQ will temporarily transfer the message between Rabbit01 and Rabbit02. Take the message entity from A and send it through B to the Consumer. So the Consumer should try to connect to each node and fetch messages from it. That is, physical queues are created on multiple nodes for the same logical Queue. Otherwise, both Consumer and Rabbit02 will always exit to Rabbit01, causing a bottleneck. When Rabbit01 fails, Rabbit02 cannot fetch unconsumed message entities from Rabbit01. If message persistence is made, it will wait for Rabbit01 to recover before it can be consumed. Without persistence, message loss occurs.

  • Mirrored mode: This allows RabbitMQ HA to be implemented by making the queues that need to be consumed mirrored across multiple nodes. The message entities are actively synchronized between mirrored nodes, rather than being read temporarily while the Consumer consumes the data, as in normal mode. However, it also has disadvantages. The synchronous communication within the cluster will occupy a large amount of network bandwidth.

The RabbitMQ pros and cons

The advantages are as follows:

  • Due to the nature of the Erlang language, RabbitMQ has high performance and concurrency.

  • Robust, stable, easy to use, cross-platform, multi-language client support, complete documentation;

  • With message confirmation mechanism and persistence mechanism, high reliability;

  • Highly customizable routing;

  • The management interface is rich, and it is also widely used in Internet companies.

  • The community is active and updates quickly.

The main disadvantages are:

  1. Although combined with the concurrency advantages of Erlang language itself, the performance is good, but it is not conducive to secondary development and maintenance;

  2. The broker architecture is implemented, meaning messages can be queued on the central node before being sent to the client. This feature makes RabbitMQ easy to use and deploy, but slow to run because of the added latency of the central node and large message encapsulation;

  3. Complex interfaces and protocols need to be learned, which is costly to learn and maintain.

5, RocketMQ

RocketMQ is a distributed queue developed by Alibaba’s r&d team. It focuses on the sequential delivery of messages and features high throughput and reliability. RocketMQ was opened source in 2013, donated to the Apache Software Foundation in 2016, and became an Apache Foundation Top project in September 2017.

RocketMQ profile

RocketMQ, implemented in the Java language, has been designed with Kafka in mind and has made its own improvements to be better than Kafka in terms of message reliability. The latest version is 4.3.1. RocketMQ has been adopted by several of the industry’s largest Internet companies.

Within Alibaba, RocketMQ serves thousands of applications, large and small. On The day of Singles’ Day every year, an incredible trillions of messages flow through RocketMQ (on the day of Singles’ Day in 2017, The whole Alibaba Group’s online message flow through RocketMQ reached trillions, with a peak TPS of 56 million), playing a pivotal role in Alibaba’s strategy of greater China and Taiwan.

RocketMQ characteristics

RcoketMQ is a low latency, highly reliable, scalable, and easy to use messaging middleware. Has the following features:

  1. Support for publish/subscribe (Pub/Sub) and point-to-point (P2P) messaging models;

  2. The queue has reliable first-in, first-out (FIFO) and strict sequential delivery;

  3. Support for Pull and Push message modes;

  4. Stacking capacity of millions of messages in a single queue;

  5. Support a variety of messaging protocols, such as JMS, MQTT, etc.

  6. A distributed, highly available deployment architecture that meets at least one messaging semantics;

  7. Provide Docker images for isolation testing and cluster deployment;

  8. Provides dashboards with rich functions such as configuration, indicators, and monitoring.

RocketMQ deployment environment

Operating system:

You are advised to use a 64-bit operating system, including Linux, Unix, and Mac OX.

Installation environment:

JDK: RocketMQ is based on the Java language, requires JDK support, version 1.8 or later; Maven: Builds require Maven support, version 3.2.x and above.

RocketMQ architecture

RocketMQ is a scalable distributed messaging and flow platform with high performance, high reliability, low latency, distributed trillion-scale capacity. It consists of Name Servers, Brokers, Producers and Consumers. Its architecture is shown below (from the official website).

RocketMQ architecture

NameServer cluster

NameServer is a full-featured server that acts like ZooKeeper in Kafka and supports dynamic registration and discovery of brokers. It mainly includes two functions:

  • The Broker management. The NameServer accepts the registration information of the Broker cluster and stores it as the basic data for routing information. A heartbeat detection mechanism is then provided to check that the Broker is still alive.

  • Route information management. Each NameServer will hold the entire routing information about the Broker cluster and the queue information for client queries. The Producer and Conumser can then use NameServer to know the routing information of the entire Broker cluster and deliver and consume messages.

NameServer is also typically deployed in clusters, where instances do not communicate with each other. The Broker registers its routing information with each NameServer, so each NameServer instance has a complete routing information stored on it. When a NameServer goes offline for some reason, the Broker can still synchronize its routing information with other Nameservers. The Produce and Consumer can still dynamically perceive the routing information of the Broker.

The Broker cluster

The Broker is responsible for storing, delivering, querying messages and ensuring high availability of services. To implement these functions the Broker contains several important sub-modules.

  • Remoting Module: The entity of the entire Broker that handles requests from Clients;

  • Client Manager: Responsible for managing Topic subscription information of clients (Producer, Consumer) and consumers;

  • Store Service: provides convenient and simple API interfaces to Store messages to physical disks and query messages.

  • HA Service: A high availability Service that provides data synchronization between Master brokers and Slave brokers.

  • Index Service: Indexes messages delivered to the Broker based on a specific Message Key to provide quick lookup of messages.

Producer cluster

Act as message producers and support distributed cluster deployment. Producers use MQ’s load-balancing module to select the corresponding Broker cluster queue for message delivery. The delivery process supports fast failure and low latency.

Consumer cluster

Act as message consumers and support distributed cluster deployment. Messages can be consumed in Push and pull modes. It also supports clustering and broadcast consumption. It provides real-time message subscription mechanism, which can meet the needs of most users.

RocketMQ high availability implementation principles

Unsurprisingly, RocketMQ’s approach to high availability (HA) is still based on the most primitive “replica idea,” but it differs from the replica mechanisms employed by Kafka, Redis, and Etcd: RocketMQ’s Master and Slave have no Election mechanism and no Failover mechanism.

RocketMQ does not have the election function. In cluster mode, the roles of Master and Slave are fixed. Master and Slave matchmaking is done by specifying the same brokerName parameter, where the Master BrokerId must be 0 and the Slave BrokerId must be a number greater than 0. Multiple slaves can be mounted under a Master and are distinguished by specifying different Brokerids for the same Master. When the Master node goes down, consumers can still consume from the Slave node to ensure that the messages pushed by the producer are not lost. Because the Master is down, producers Push messages to other masters without affecting availability. RocketMQ brokers can be deployed in four ways.

1. Single Master mode

There are no advantages other than simple configuration.

It has the disadvantage of being unreliable. The machine restarts or goes down, making the entire service unavailable, so this is rarely used in production environments.

2. Multiple Master modes

Its advantages are simple configuration and highest performance.

Its disadvantages are: a small amount of messages may be lost (a small amount of messages may be lost in asynchronous disk flushing, but not lost in synchronous disk flushing). During the restart or downtime of a single machine, the messages not consumed by the machine cannot be subscribed before the machine recovers, affecting the real-time performance of messages.

Note: Master brokerRole configuration must be ASYNC_MASTER when using multi-master no-slave cluster broker. If SYNC_MASTER is set to SYNC_MASTER, the SendStatus returned by producer will always be SLAVE_NOT_AVAILABLE when the producer sends messages.

3. Multi-master, multi-slave mode: asynchronous replication

The advantages are: even if the disk is damaged, the message loss is very small, and the real-time performance of the message is not affected, because when the Master is down, consumers can still consume from the Slave. This process is transparent to the application, does not require human intervention, and the performance is almost the same as the multi-master mode.

The disadvantage is that a small number of messages are lost when the Master is down or the disk is corrupted.

4. Multi-master, multi-slave mode: Synchronous double-write

The advantages are: no single point of data and service, no message delay in the case of Master outage, and very high service availability and data availability.

The performance of the asynchronous replication mode is lower than that of the asynchronous replication mode by about 10%. The RT for sending a single message is higher than that of the asynchronous replication mode. When the active replication mode is down, the standby replication mode cannot be switched over to the host automatically.

Advantages and disadvantages RocketMQ

The advantages mainly include the following points.

  1. More than 10,000 persistent queues are supported on a single machine.

  2. All RocketMQ messages are persistent, written to the system Page Cache first, then flushed to ensure that both memory and disk have a copy of the data, when accessed, directly from memory;

  3. The model is simple and the interface is easy to use (JMS interface is not very practical in many cases).

  4. Performance is very good, can heap messages in the Broker;

  5. Support multiple consumption modes, including cluster consumption, broadcast consumption, etc.

  6. Distributed and extended design for each link, master-slave HA;

  7. The community is active and the version is updated quickly.

The main disadvantages are:

  1. The supported client languages are not many, currently Java, C++ and Go, the latter two are not mature;

  2. The CLI (command line interface) is used to query, manage, and diagnose various problems without Web management interface.

  3. Interfaces such as JMS are not implemented in the MQ core.

6. Comparison of several message queues

There is a wide variety of open source software related to message queues, and this article covers only the four most common in production environments. Each of these message queues has its own advantages, and none of them has the advantage of “unifying all rivers and lakes”, which to some extent increases the difficulty of selection. Unlike distributed caches and distributed locks, Redis and Etcd have the “absolute” advantage of being effort-free.

RocketMQ official review

Practice is the sole criterion for testing truth, and the expression in practice is more convincing than words. In the Official RocketMQ documentation, the development background of RocketMQ states: in our study, the ActiveMQ IO module quickly reached a bottleneck as the use of Queue and Topic increased. We tried to solve this problem with throttling, circuit breakers, or downgrades, but it didn’t work very well. So we started looking at Kafka, a messaging solution that was popular at the time. Unfortunately, Kafka didn’t meet our requirements, especially in terms of low latency and high reliability.

In short, the performance of ActiveMQ and Kafka cannot meet ali’s super-scale application scenarios. In this context, Alibaba developed RocketMQ himself and donated it to the open source community, where more than 100 businesses now use its open source version. A comparison of ActiveMQ, Kafka, and RocketMQ is shown below (from the RocketMQ official documentation) :

Comparison of ActiveMQ, Kafka and RocketMQ

Compare the four message queues

Message queue uses efficient and reliable messaging mechanism to communicate platform-independent data and integrate distributed systems based on data communication. At present, there are many MQ products in the industry, such as RabbitMQ, RocketMQ, ActiveMQ, Kafka, ZeroMQ, MetaMq, etc. There are also cases that directly use Redis database as message queue. These message queue products, each has its own focus, in the actual selection, need to combine their own needs and MQ product characteristics, comprehensive consideration.

Here is a comparison of the four message queues:

The RabbitMQ/ActiveMQ/RocketMQ/Kafka contrast

7. References

  • The RabbitMQ homepage: https://www.rabbitmq.com/

  • ActiveMQ homepage: http://activemq.apache.org/

  • RocketMQ homepage: https://github.com/alibaba/RocketMQ

  • Kafka home page: http://kafka.apache.org/

PS: In case you can’t find this article, please click “like” to browse and find it.