A complex distributed system must consider using MQ to solve the problem of communication decoupling. More and more open source MQ is available on the market, which makes it more and more difficult to choose, such as Kafka, RabbitMQ, RocketMQ. It seems that each MQ is powerful, and it is difficult to know which one to use. In one technology selection, we chose RocketMQ, not Kafka or anything else. Here’s a summary of the selection and how RocketMQ compares to Kafka.

performance

In terms of performance, Kafka is superior to RocketMQ, which can achieve several hundred thousand TPS. RocketMQ can hardly exceed 10W TPS, but this is advanced. Advanced means that Kafka cannot have too many topics or partitions (preferably no more than 64). This is because Kafka performs well because it uses sequential writes. However, if the number of topics or partitions increases, the number of writes to multiple files becomes random. RocketMQ, on the other hand, almost never suffers from this problem, and even in the case of many topics (5W is officially supported), there is no significant performance degradation. Therefore, in the case of many topics, RocketMQ can be used, and tens of thousands of TPS is also a good performance.

reliability

In terms of reliability, RocketMQ is better than Kafka’s asynchronous flush mechanism due to its support for master/slave synchronous flush mechanism. Although performance degrades, RocketMQ is recommended if you have high reliability requirements. Of course, if you are only sending unreliable messages, such as notifications or SMS messages, synchronous flushing is not recommended, but RocketMQ can be used asynchronously.

features

RocketMQ has many rich features such as automatic message retries, delayed messages, etc. For example, we use automatic message retries and an exploration of distributed task idempotent processing to implement automatic retries of failed tasks.

conclusion

There is no silver bullet in the world, so is the choice of middleware, only the most appropriate is not the best, the completion of architecture design is the most correct choice!