I’m sure message queuing (MQ) is familiar to any Java programmer. If so, you’re probably working with small projects. Traditional projects don’t split services, and as business grows, they split multiple microservices.
Take our company’s APP user order as an example. The user requests to first arrive at the order service, and then notify the merchant to execute the business after the inventory destocking service succeeds in reducing the inventory, and then settle the coupons through the coupon system, and then settle the points through the point system.
Assuming that the response time of each microservice interface is 200ms, then these service calls will take 600ms, plus the 200ms business processing time of the order service itself, the response time of the entire order submission interface will be nearly 1s. If the system is connected to other systems as services develop, the time is longer and longer. An interface responds one second, which is a very low throughput. When the QPS of the system is slightly higher, the server resources are insufficient. When the resources are insufficient, the subsequent user requests will wait for the server to release resources to process, and the user will feel stuck. Once the user feels the card, they are more likely to go back and refresh the page, or click submit order again, and then the request comes back again, and then access the database, and they will get more card. The system could end up crashing.
If you think about it carefully, you will find that the operations of merchant business, coupon business and integral business can be executed in parallel, and there is no need to request them one by one. It saves a lot of time if these are in parallel, so we can consider using MQ.
The introduction of MQ
Before introducing MQ, we need to understand the model of MQ as a messaging middleware that connects microservices at both ends: producers and consumers.
Producers asynchronously send messages that complete a function to MQ, which pushes messages to consumers, who perform their own business.
The above figure is the process after using MQ. We only need to visit the order service, and then I send a message to the corresponding service to do the things in the order service. For example, if I want to pay coupons, I send a message to the coupon service, if I want to deduct and increase the points, I send a message to the points system. As a result, only the order service takes 200ms, and the messaging to MQ only takes about 5ms, greatly improving the throughput of the interface. And the 200ms response is insensitive to the user. This reflects the number one advantage of message queues: asynchrony.
Now, you might say, well, can’t I use multiple threads, or even a pool of threads to do it asynchronously? Don’t worry, at first glance using threads can be asynchronous, but you can think about how many threads are open for a single order request. If 50 people are placing orders, then 50 * 5 = 250 threads need to be opened to process the business. We all know that threads consume CPU resources, and there are many thread context switches involved. With MQ, if our new business needs to access the new system in the future, we can just send a message to inform the new system directly, and hardly need to change any order code. This kind of system-level decoupling is really NICE. This is MQ’s second great advantage: decoupling.
For example, in the first graph, we said that a single interface serial series of microservers requires 1s of response. Assume a large number of users in the case of concurrent order, or directly consider the seckill interface scenario. Instantly tens of thousands or even hundreds of thousands of traffic, if according to the throughput of the previous server was knocked down, even if your service cluster is a lot, not down, then your database can not withstand so many requests. After the introduction of the message queue, we will give the instantaneous flow all of MQ, MQ is a professional accept high concurrency, generally will not be crushed, downstream subsystem and partial equilibrium from MQ consumption, to resist instantaneous peak flow, wait until after the peak, consumption remains slow speed, it is introduced the MQ’s third big advantage: peak peel.
These are some of the benefits of using MQ. I’m sure everyone will be able to point out the advantages of MQ in the interview, such as asynchronous, decoupled, peak clipping. But the interviewer doesn’t want to hear those words, and will want you to relate them to the business context of your project. And when you’re done, the interviewer is sure to ask you a follow-up question.
- Where did you use MQ in your project and what problems did you use it to solve?
- You’ve talked about the benefits of using MQ, but are you aware of the problems with using it?
- How did you solve these problems with the introduction of MQ?
- Which MQ product did you choose? Why is that?
Problems with the introduction of MQ
Using MQ does solve some of the problems, but adding an MQ service for no apparent reason is definitely something to be considered for maintenance, and it’s important to keep MQ highly available first.
- Otherwise MQ hangs up and the message doesn’t go out. What if the downstream system of several coupons and points after I create an order does not perform business settlement?
- What if MQ is highly available, the message is sent, but the coupon settlement service reports an error? Since this is asynchronous, it is not easy to roll back. Has this issue been considered?
- When the message is sent normally and the consumers receive it, the merchant system and coupon system are executed normally, but the points are not settled due to the error of the points service, then the data of this order is inconsistent. How to solve this problem?
- Points system repeated consumption of the message, resulting in two settlement how to do?
So with the introduction of MQ, the complexity of the system has increased and many of the above issues have to be considered. The interviewer will certainly ask you how to solve, or how to avoid a few of the above problems, wait for the next ~~~~
MQ Product Selection
I do not have much experience in this thing, after all, my work experience is still shallow, so I really have not studied it in depth, but can you refer to the comparison of several popular MQ products on the Internet
| attribute | RabbitMQ | ActiveMQ | RocketMQ | Kafka |
|---|---|---|---|---|
| Development of language | Erlang | Java | Java | Scala |
| Languages supported by the client | Official support for Erlang,Java,Ruby, etc., community produced API, almost all languages support | Java, C, C++,Python, PHP,Perl,.NET, etc | Java, c + + | Java is officially supported, and the community produces various apis, such as PHP,Python, etc |
| Single machine throughput | Ten thousand class (second) | Ten thousand class (worst) | Class 100,000 (best) | Class 100,000 (second best) |
| The message delay | microsecond | millisecond | millisecond | Within milliseconds, |
| features | Strong concurrency, excellent performance, low latency, active community, and rich management interface | Old – brand products, high maturity, more documents | MQ has complete functions and good expansibility | Only major MQ functions are supported, after all, for the big data world. |
The business volume of our company is not very large, and there is no need for secondary development on MQ for the time being. After considering simple integration and low message latency, we finally choose RabbitMQ. It is very convenient to use Springcloud-stream to integrate RabbitMQ