What is the MQ
Message Queue (MQ) is a first-in, first-out data structure of basic data structures.
Why use MQ
Its application scenarios mainly include the following three aspects
The application of decoupling
The higher the coupling of the system, the lower the fault tolerance of the system. Taking e-commerce applications as an example, after users create an order, if they call the inventory system, logistics system and payment system in a coupling way, any subsystem fails or becomes temporarily unavailable due to upgrading or other reasons, ordering operation will be abnormal and user experience will be affected.
Use message queues to decouple. For example, when the logistics system breaks down, it takes several minutes to repair. During this time, the data to be processed by the logistics system is cached in the message queue, and the user’s order operation is completed normally. When the logistics system recovers, the order messages in the message queue can be processed, and the terminal system cannot perceive that the logistics system has broken down for several minutes.
Traffic peak clipping
For example, a second kill system, when the second kill starts, the whole system traffic will suddenly increase, this time, it is possible to overwhelm the database.
With message queuing, a large number of requests can be cached and processed over a long period of time, which greatly improves system stability and user experience.
- In general, to ensure system stability, user requests will be blocked if the system load exceeds the threshold, which will affect the user experience.
- It is better to use message queues to cache requests and wait for the system to complete the process before notifying the user that the order has been placed than not placing an order at all.
- During normal hours of the service system
QPSIf it is 1000 and the peak traffic is 10000, it is obviously not cost-effective to configure a high-performance server to cope with the wandering peak. In this case, message queues can be used to truncate the peak traffic.
Data distribution
Not using the MQ
Let’s start with a scenario like this:
System A requires frequent code changes, which is clearly not A good design, and MQ can be used to solve this problem.
Using the MQ
Message queues allow data to flow between multiple systems, and producers don’t care who uses the data, just send the data to the message queue, where the data is directly retrieved.
The disadvantage of MQ
The availability of the system decreases
The more external dependencies a system has, the worse the system stability will be. Once MQ breaks down, services will be affected.
System complexity increases
The addition of MQ has greatly increased the complexity of the system, from synchronous remote calls between systems to asynchronous calls via MQ:
- How do I ensure that messages are not re-consumed?
- How to handle message loss?
- How to ensure the timeliness of message delivery?
Consistency problem
After processing services, system A sends messages to system B, C, and D through MQ. If system B and C process the messages successfully, system D fails to process the messages:
- How to ensure consistency of message data processing?
A variety ofMQComparison of products
Common MQ products include Kafka, ActiveMQ, RabbitMQ, and RocketMQ.
| features | ActiveMQ | RabbitMQ | RocketMQ | Kafka |
|---|---|---|---|---|
| Development of language | java | erlang | java | scala |
| Single machine throughput | All level | All level | The class of 100000 | The class of 100000 |
| timeliness | Ms level | Us level | Ms level | Ms than level |
| availability | High (master-slave architecture) | High (master-slave architecture) | Very high (distributed architecture) | Very high (distributed architecture) |
| features | Mature products, used in many companies; There are more documents; Good support for various protocols | Based on Erlang development, so the concurrent ability is strong; Excellent performance, low delay; The management interface is rich | MQ function is relatively perfect, scalability is good | Support for major MQ functions; Functions like message query, message callback, etc., are not provided; After all, it’s for big data, and it’s widely used in big data |
Welcome to follow the wechat official account to read more articles:
