RocketMQ is a unified messaging engine and lightweight data processing platform. In layman’s terms, when you need a message processing framework, consider it.

This article is basically from the reference documents listed in the appendix. For my notes, you can skip directly to the reference documents or jump directly to the official Github RocketMQ documentation and skip this article

RocketMQ has those features

Message type

  • Transaction messages: Applying a local transaction and sending a message operations can be defined in a global transaction and either succeed or fail simultaneously. The ultimate consistency of distributed transactions can be achieved through transaction messages
  • Timed messages: After a message is sent, it is not consumed immediately, but at a specified point in time
  • Delayed messages: A message is not consumed immediately after it is sent, but after a specified period of time has passed (a fixed amount of time has elapsed since it was sent)
  • Sequential messaging: The first published message is always consumed first, supporting both global sequential and partitioned sequential messages
    • Global sequential messages: All messages are published and consumed in a strict first-in, first-out order
    • Partition order messages: Messages of the same partition are published and consumed in a strict first-in, first-out order. The partition field is Sharding Key
  • Common message: None of the above message features

Message features:

  • The message is delivered At least once: an ACK is returned only after the consumer has finished consuming

  • Message retry: After a consumption failure, considering that it takes some time to recover from an exception, multiple retry levels are set. Each retry level has a corresponding redelivery delay. The more retry times, the longer the delivery delay will be

  • Message filtering: Consumers can filter messages based on tags

  • Message query: When a consumer encounters a problem, messages can be queried by Message ID, Message Key, and Topic

  • Message backtracking: The ability to re-consume consumed messages or discard accumulated messages at a custom time or location

  • Flow control: Flow control can be performed when production (after flow control, message redelivery is not attempted) or consumption (reduced pull frequency) reaches a bottleneck

  • Dead letter queue: After the maximum number of retries is reached, if the message still fails, the message is sent to the dead letter queue instead of being discarded immediately. The message in the dead letter queue can be processed specifically

Architecture and core concepts of RocketMQ

concept

  • Message: The carrier of Message delivery in a Message queue, the smallest unit that produces and consumes data
  • Message Id: A globally unique identifier of a Message that uniquely identifies a Message. It is generated by RocketMQ
  • Message Key: Service id of a Message. The Message producer is set to uniquely identify a service logic
  • Topic: A collection of a class of messages, each containing several messages, each of which must belong to one Topic and only one Topic, and is the basic unit of Message subscription for RocketMQ
  • Tag: Used to distinguish different types of messages within the same Topic
  • Producer: indicates message producers
  • Producer Group: A Group of producers of the same kind who send the same kind of messages logically
  • Consumer: message sender
  • Consumer Group: A collection of consumers of the same type who consume the same type of messages with consistent logic
  • Group Id: indicates the Group Id

architecture

  • Producer: A message Producer that produces and sends messages
  • Consumer: Message Consumer, responsible for receiving and consuming messages
  • NameServer:Topic Routing registration messages. Each NameServer holds the entire routing information about the Broker cluster and the queue information for clients to query
  • Broker: Stores, delivers, and queries messages and ensures high availability of services
  • When sending a message, you need to know which Broker is being sent to. The post is taken from the local cache by default. If the cache does not exist, it is pulled from NameServer again (Consumer is similar).
  • Routing Info: After the Broker starts, it registers itself with NameServer and reports Topic Routing information to NameServer at regular intervals

Why RocketMQ

The RocketMQ team started with ActiveMQ, but as queues and topics increased, the ActiveMQ IO model reached its bottleneck and began to consider whether Kafka was appropriate. However, kafka did not meet the requirements in terms of low latency and high availability. So the decision was made to develop new RocketMQ.

The explanation comes from the RocketMQ team, which generally wants to compare “absolutely fair and just”, but from the RocketMQ team, it also needs to think about whether it is “biased”.

RocketMQ vs ActiveMQ vs Kafka

reference

RocketMQ Github Basic concept RocketMQ Github features RocketMQ Github architecture design RocketMQ Github Design Ali Cloud -RocketMQ documentation – terms explain the RocketMQ documentation – features Apache RocketMQ- Why Use RocketMQ