preface

Shared subscriptions are a new feature introduced in MQTT 5.0 and act as a load balancing function on the subscriber side.

We know that the general unshared subscription message publishing process looks like this:

In this structure, if the subscribing node fails, the publisher’s messages will be lost (QoS 0) or stacked in the Server (QoS 1, 2). In general, the solution to this problem is to directly add subscription nodes, but this will produce a large number of repeated messages, not only waste performance, in some business scenarios, the subscription nodes also need to be self-deduplication, further increasing the complexity of the service.

Secondly, when the production capacity of the publisher is strong, the consumption capacity of subscribers may not be able to keep up with the situation in time. At this time, subscribers can only achieve load balancing to solve the problem, which increases the development cost of users again.

Protocol specification

Now, in MQTT 5.0, you can solve the above problems with the shared subscription feature. When you use a shared subscription, the message flow becomes:

Like a non-shared subscription, a shared subscription contains a topic filter and a subscription option. The only difference is that the topic filter format for a shared subscription must be $share/{ShareName}/{filter}. The meanings of these fields are:

  • $shareThe prefix indicates that this will be a shared subscription
  • {ShareName}Is a string that does not contain “/”, “+” or “#”. Subscribe to the session by using the same{ShareName}Indicates that the same subscription is shared, and messages matching this subscription are published to only one session at a time
  • {filter}That is, topic filters in non-shared subscriptions

Note that if a service is sending a QoS 2 message to its selected subscriber and the network is interrupted before the distribution is complete, the service will continue to complete the distribution of the message when the subscriber reconnects. If the subscriber’s session terminates before it reconnects, service! The endpoint discards the message without attempting to send it to another subscriber. In the case of QoS 1 messages, the server can wait until the subscriber reconnects to complete the distribution, or it can attempt to distribute the message to another subscriber as soon as the subscriber disconnects. The MQTT protocol is not mandatory, so it depends on the server implementation. However, if the session terminates while waiting for a subscriber to reconnect, the server will attempt to send the message to another subscriber.

Sharing policy

While shared subscriptions enable load-balanced consumption of messages on the subscriber side, the MQTT protocol does not specify what load-balancing policy the Server should use. For reference, EMQ X provides four strategies: random, round_robin, sticky, and hash for users to choose.

  • Random: Randomly selects one of all shared subscription sessions to send a message
  • Round_robin: the selection is based on the subscription order in rotation
  • Sticky: Use the random policy to randomly select a subscribed session and continue to use it until the session is unsubscribed or disconnected and then repeat the process
  • Hash: Hashes the ClientID of the sender and selects a subscription session based on the hash result

Results demonstrate

Finally, let’s demonstrate the effect of shared subscriptions with a comprehensive example.

The server uses EMQX-V3.2.4, and the client uses EMQTT. The sharing subscription distribution policy of EMQX is the default random:

broker.shared_subscription_strategy = random

Use./emqx start to start emqx, and then use emqtt to start three subscription clients, subscribing to $share/ A/Topic, $share/ A/Topic, and $share/ B/Topic

Start a publishing client to publish to the Topic topic.

$share/ A /topic and $share/ B /topic belong to different session groups. Non-shared subscription topic topics are load balanced across all session groups. Client Sub3 receives all messages because it has only one session in the group, while client Sub1 and sub2 receive messages randomly according to the random policy configured by us.

For more information, visit our official website, emqx. IO, or visit our open source project, github.com/emqx/emqx. For more information, visit our official documentation.