Message queues are one of the oldest middleware, and have naturally emerged since the need to communicate between systems. If you haven’t used message queues yet, it’s time to learn. This article tells you what message queues are, why they are needed, what common message queues are, and how to deploy and use RabbitMQ.

What is a message queue

Message queue if you break it down, it’s message plus queue, what is a message? Messages are generated from producers, queued, installed with rules designed to be queued, and consumed by consumers. That’s all.

Why message queues

Message queue, the most important thing is queue, you can imagine the scenario without queue, you go to the bank to do business, no one in line, everyone in a pile, too small and too weak to do business.

Without message queues, your system will be heavily coupled and will be tied up in maintenance upgrades.

Without message queues, many of the functions of your system are synchronous, which means that you can only perform subsequent operations after the previous events have been completed, and the front-end users will feel sluggish and have a poor experience.

Without message queues, web systems suddenly face high levels of concurrent access requests and may crash.

With message queues, system decoupling, asynchronous communication, traffic peaking, delayed notification, ultimate consistency assurance, sequential messaging, streaming processing, and more can be easily addressed.

Common message queues

The most common message queue products are ActiveMQ, RabbitMQ, ZeroMQ, Kafka, RocketMQ, etc.

ActiveMQ

Apache ActiveMQ is an open source messaging middleware developed by Apache Software Foundation. As ActiveMQ is a pure Java program, it can be executed only when the operating system supports Java virtual machine.

  • Java Message Service (JMS) version 1.1 is supported
  • Spring Framework
  • Cluster (Clustering)
  • Supported protocols include OpenWire, REST, STOMP, WS-Notification, MQTT, XMPP, and AMQP

RabbitMQ

RabbitMQ is open source message broker software (also known as message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ servers are written in the high-performance, robust and scalable Erlang language and support all major operating systems such as Linux, Windows and MacOS. The client supports all major programming languages.

  • Scalability: Cluster services
  • Message persistence: Persist messages from memory to disk and load them from disk to memory

ZeroMQ ZeroMQ (also spelled 0MQ or ZMQ) is a high-performance asynchronous messaging library designed for scalable distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, ZeroMQ does not require a dedicated Message Broker to run. The library is designed as a common socket style API.

ZeroMQ was developed by iMatix and a large community of contributors. ZeroQ supports most popular programming languages through many third-party software, from Java and Python to Erlang and Haskell.

Kafka

Kafka is an open source stream processing platform developed by the Apache Software Foundation and written in Scala and Java. The goal of the project is to provide a unified, high-throughput, low-latency platform for processing real-time data. Its persistence layer is essentially a “massive publish/subscribe message queue based on a distributed transaction logging architecture,” which makes it valuable as an enterprise-level infrastructure for handling streaming data. In addition, Kafka can be connected to external systems (for data input/output) through Kafka Connect, and a library for Kafka Streams is provided. This design is heavily influenced by transaction logging.

RocketMQ

RocketMQ is a distributed messaging and streaming data platform with low latency, high performance, high reliability, trillion-scale capacity, and flexible scalability. RocketMQ is the third generation distributed messaging middleware of Alibaba open source in 2012. On November 21, 2016, Alibaba donated RocketMQ to Apache Software Foundation. On February 20 of the following year, the Apache Software Foundation announced Apache RocketMQ as a top-level project.

The message queue selection depends on the specific application requirements: ZeroMQ is small and beautiful, RabbitMQ is large and stable, Kakfa and RocketMQ are fast and powerful.

Deployment and use of RabbitMQ

Docker deployment is recommended. In an environment where Docker is installed, run the following command:

docker run -d --hostname my-rabbit -p 15672:15672 -p 5672:5672 -- name rabbit-server -e RABBITMQ_DEFAULT_USER=user -e RABBITMQ_DEFAULT_PASS=password rabbitmq:3-management
Copy the code

Now open localhost:15672 in your browser. The user name is user and the password is password

Next, let’s create a queue called my_app

Create an Exchange called my_exchange

Click the Exchange TAB, click my_exchange to enter the details page, bind my_exchange and my_app, and set the routing key to test:

Write producers in Python

You can now write producers in Python to produce a message to queue and watch the Web page change:

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'.5672.'/', pika.PlainCredentials('user'.'password')))
channel = connection.channel()

channel.basic_publish(exchange='my_exchange', routing_key='test', body='Test! ')

connection.close()
Copy the code

Execute the code above to put the message on the queue. Here I execute it four times and see four messages:

The message will remain in the queue until the consumer retrieves it, and then we write a program that consumes the message.

Python authoring consumer

import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'.5672.'/', pika.PlainCredentials("user"."password")))
channel = connection.channel()

def callback(ch, method, properties, body) :
    print(f'{body} is received')

channel.basic_consume(queue="my_app", on_message_callback=callback, auto_ack=True)
channel.start_consuming()
Copy the code

Perform:

At this point, the queue is empty:

This code provides a minimal demonstration of how to publish messages to RabbitMQ, but for more information please go to the official documentation.

The last word

Message queues can decouple system modules, but become critical nodes in their own right, requiring significant attention from system administrators in terms of cluster deployment and failover. This article provides a brief introduction to what message queues are, why they are needed, what are the common message queues, and how to deploy and use RabbitMQ.