I’m participating in nuggets Creators Camp # 4, click here to learn more and learn together!

Apache RocketMQ predecessor

With the success of SpringCloud Alibaba’s mature micro-service solution, RocketMQ, a messaging middleware that has supported Alibaba’s internal business over the years, has basically replaced other messaging middleware (Kafka, RabbitMQ, ActiveMQ) in selection.

The reference to metamorphosis in the title is actually intended to draw people in, to give them a curious idea. The reason for the Metamorphosis is that RocketMQ was called Metaq inside Ali, which is the short for the original title (‘Metamorphosis’). It was a novella written by The author of KafKa’s masterpiece, in homage to KafKa.

RocketMQ certainly borrowed some of KafKa’s best design early on. Ali developed RocketMQ, a messaging middleware called Notify, to meet the need for customization in the Java-dominated ali.

In 2016, open source contributed to the Apache Foundation, making its contribution to open source. And that’s what RocketMQ is now.

Apache RocketMQ is a distributed message and

Apache RocketMQ architecture and details

As you can see from the architecture diagram on RocketMQ’s website:

  • Producer: A role that produces messages and supports distributed cluster deployment. It does not matter which Broker queue messages are delivered through; RocketMQ does its own load balancing.

  • Consumer: A role that consumes messages and supports distributed cluster deployment. It supports both Push and Pull consumption modes (both of which are essentially pulled, and Push is also acquired in a long polling mode and then entered into the monitor to achieve the effect of Push), and supports CLUSTERING and BROADCASTING consumption modes.

  • Registry (NameServer): A simple to use Topic Router registry, RocketMQ implements its own registry, making it easy to maintain and upgrade extensions without relying on any registry. Used for Broker management and routing information management. Although clustered, the Broker registers with each NameServer.

    • The Broker management

    When the Broker configuration NameServer address then start, will be the registered information of Broker cluster into the NameServer and save up, and then use Ping | Pong heartbeat mechanism to check the Broker available;

    • Routing Information Management

    When a Broker registers with a NameServer, the entire route information and queue information in the Broker is stored for access by Producer and Consumser.

  • BrokerServer: Stores, delivers, and queries messages. And recommended distributed cluster deployment, also do a lot of design for high availability criteria:

    • The Remoting Module is responsible for receiving Clients requests.
    • Client Manager: Responsible for managing production and consumption clients and maintaining all Topic information;
    • Store Service: Asynchronously or synchronously Store messages to physical disks and query messages from physical disks.
    • HA Service: High Available. Ensure data synchronization between Master and Slaver so that it is ok to switch to Slaver if Master is unavailable.
    • Index Service: Index messages according to a specific MessageKey to provide a better query;

    Therefore, when sending messages, you are advised to set a unique IDENTIFIER for each message to facilitate location.

// orderId String orderId = “20034568923546”; message.setKeys(orderId);

What are the similarities between RocketMQ transaction messages and delayed messages

This is a small point, and an interesting design that works for RocketMQ:

Transaction messages are sent half Message in order not to be consumed by the consumer, and delayed messages are sent within the delay time in other topics, only when the conditions are met, and then cut back to the topic to be consumed by the consumer.

Seemingly complex things can always find a simple and convenient design to achieve, but lack of discovery.