Basic standards for message queue products

1.1. Must be open source products

Open source means that if one day you are using the message queue met a business affect your system Bug, at least you still have a chance to modify the source code to quickly repair or avoid this Bug, solve the problem of your system’s fire, rather than a stranded waiting for developers is not necessarily released when the next version to solve.

1.2. The product must be popular in recent years and have a certain degree of community activity

The good thing about popularity is that, as long as your usage scenario is not too popular, your chances of encountering bugs are very low, because most of the bugs you might encounter have already been encountered and fixed by someone else. Some of the problems you encounter in the process of use are also easier to find similar problems on the Internet, and then quickly find solutions.

Another benefit is that popular products have better integration and compatibility with the surrounding ecosystem, like Kafka and Flink,

1.3 message reliability to ensure that no message is lost

1.4 support the cluster, to ensure that the service will not be unavailable because of the failure of a node, of course, can not lose messages

1.5 Good performance, can meet the performance requirements of most scenarios

2. Comparison of several common message queues

2.1, the Rabbitmq

Written in Erlang, it was originally designed for reliable communication between systems in the telecommunications industry and is one of the few message queues that support the AMQP protocol. With light weight, convenient, out of the box characteristics

One feature of RabbitMQ is its flexible routing configuration. Unlike other message queues, it has an Exchange module between Producer and Queue, which you can think of as a switch. The Exchange module also acts much like a switch, distributing messages from producers to different queues based on configured routing rules. Routing rules are flexible, and you can even implement them yourself.

RabbitMQ clients support probably the most programming languages of all message queues.

Disadvantages:

The first problem is that RabbitMQ does not support message stacking very well. Message queues are designed to be conduits and large backlogs are an abnormal condition to be avoided. When a large number of messages are backlogged, RabbitMQ performance degrades dramatically.

The second problem is that RabbitMQ has the worst performance of any of the message queues, and can handle anywhere from tens of thousands to hundreds of thousands of messages per second, depending on the hardware configuration, based on official test data and our experience in daily use. This is more than enough for most applications, however, if your application requires very high performance on message queues, don’t use RabbitMQ.

The last question is the programming language used by RabbitMQ, Erlang. If you want to do extensions and secondary development on RabbitMQ, you are advised to think very carefully about sustainable maintenance.

2.2, RocketMQ

RocketMQ is a message queue product that Alibaba opened source in 2012 and later donated to the Apache Software Foundation. Alibaba also uses RocketMQ internally as the message queue to support its business. It has experienced many “Double 11” tests, and its performance, stability and reliability are trustworthy. As an excellent home-made message queue, it has been used more and more by many big factories in China in recent years.

Advantages:

There is a very active Chinese language community and most questions can be answered in Chinese.

Written in the Java language, most of the sharers are Also Chinese, the source is relatively easy to read, easy to RocketMQ extension and secondary development.

The online business response latency has been optimized for milliseconds in most cases. If your application scenario is concerned with response latency, you should use RocketMQ.

RocketMQ is an order of magnitude better than RabbitMQ and can process hundreds of thousands of messages per second

Disadvantages:

As a domestic message queue, compared with the similar products that are popular abroad, it is not so popular in the world, and its integration and compatibility with the surrounding ecosystem is a little less than that.

2.3, kafka

Kafka was originally developed by LinkedIn and is currently a top-level project at Apache. Kafka was originally designed to handle huge volumes of logs. In earlier versions, there were many design sacrifices for maximum performance, such as poor message reliability, loss of messages, lack of clustering, and poor functionality, that were acceptable for the specific scenario of handling massive logs. Kafka at this time couldn’t even be called a proper message queue. Kafka has developed into a very mature message queue product, which can meet the requirements of most scenarios in terms of data reliability, stability and functional features.

Advantages:

Kafka is one of the most compatible ecosystems around, especially in big data and streaming computing, where Kafka is a priority for almost all open source software systems.

Kafka is developed in Scala and the Java language. It is designed with a lot of batch and asynchronous thinking, which allows Kafka to achieve extremely high performance. Kafka’s performance, especially for asynchronous sending and receiving, is the best of the three, but not of an order of magnitude different from RocketMQ, which can process hundreds of thousands of messages per second.

Disadvantages:

The problem with Kafka’s asynchronous bulk design, however, is that the response time for synchronous sending and receiving messages is high, because when a client sends a message, Kafka does not send it immediately. Instead, Kafka waits for a batch of messages to be sent. A lot of places use this “collect a wave and deal with it together” design. When your business scenario doesn’t have as many messages per second, Kafka’s latency can be high. As a result, Kafka is not well suited for online business scenarios. Kafka is ideal for handling large volumes of messages, such as collecting logs, monitoring information or burying points on the front end, or if your application scenario makes heavy use of big data or open source products related to streaming computing, Kafka is the perfect message queue for you