First, write first

As anyone who has worked with distributed middleware knows, it is not complicated for programmers to use. There are only a few common client apis, far fewer than the ones we use to write programs. But distributed middleware is not used much by small and medium-sized r&d teams. Why is this so?

The reason is that the responsibilities of middleware are relatively single, and while the client side is simple to use, the overall environment is not easy to set up. So for the series of several middleware articles, we focus on solving the threshold problem, the server environment set up (later can be cloud or operation and maintenance solution), the basic responsibilities and functions of the middleware, the client Demo written well, let programmers lift feet, in the debugging code can easily start.

RabbitMQ message queue RabbitMQ message queue RabbitMQ message queue RabbitMQ

Why message queue MQ

1. Business system often requires strong response ability, which can play the role of peak cutting and valley filling.

2. Decoupling: If one system is down, it does not affect the continued operation of other systems.

3. Business systems often have high reliability requirements for messages and complex functions such as Ack.

4. Enhance the asynchronous processing capability of the business system, and reduce or even almost impossible concurrency:

Using message queues is like building Gezhouba for flood control, with the capacity to accumulate large amounts of data, and then reliably asynchronously output it. Such as:

The traditional practice has the following problems, as shown in the figure above:

  1. If the service processing time exceeds the timer interval, orders will be missed.
  2. If new threads are used to obtain data, the server will easily break down due to a large number of new threads.
  3. The database is under great pressure and easy to be concurrent.

The benefits of using MQ are shown in the figure above:

  1. Services can be registered and configured.
  2. The data acquisition rule is configurable.
  3. Messages in MQ are Ack only if they are successfully consumed, improving reliability.
  4. Greatly enhanced ability to process business jobs asynchronously:

After the data is periodically obtained from the database and stored in the MQ message queue, the Job periodically scans the MQ message queue. Assume that the Job prefetches five messages and processes them asynchronously, that is, the five messages may be processed at the same time.

3. Introduction to RabbitMQ

RabbitMQ is an open source messaging component based on AMQP for storing and forwarding messages on distributed systems. It is written in the Erlang language, which is known for its high performance, high availability, and high scalability.

Advanced Message Queuing Protocol (AMQP) is an application-layer Protocol specification used for asynchronous messaging, and is designed for message-oriented middleware.

RabbitMQ has the following features:

High reliability: RabbitMQ offers a variety of features that allow you to make trade-offs between reliability and performance, including persistence, send and reply, release confirmation, and high availability.

High availability queue: supports cross-machine cluster, supports queue security mirror backup, message producers and consumers no matter which side of the problem, will not affect the normal sending and receiving of messages.

Flexible routing: All messages are forwarded to message queues via routers. RabbitMQ has several common routers built into it and can use a combination of routers and custom router plug-ins to perform complex routing.

Support for multiple clients: Client implementations for major development languages (such as Python, Ruby,.NET, Java, C, PHP, ActionScript, etc.).

Clustering: Multiple servers within a local network can be aggregated to form a logical broker.

Scalability: support load balancing, simple and convenient to dynamically add or remove servers.

Permission management: Flexible user role permission management, Virtual Host is the smallest granularity of permission control.

Plugin system: supports a variety of plugin extensions, as well as custom plugins. The most common plugin is the RabbitMQ_Management Web UI access address:

http://139.198.13.12:6233/,

The login account is flight and password is yyabc123.

How RabbitMQ works

The RabbitMQ message flow from sender to receiver is shown below:

The working mechanism of sending and receiving messages

RabbitMQ basic usage

There are six basic usages: one-to-one, one-to-many, publish-subscribe, send and receive by routing rules, topics, and RPC (remote storage calls). We will cover the use of one-to-one, one-to-many, and themes.

1. Single to single: single sending and single receiving. See the picture below.

2. One-to-many: one sender and multiple receivers, such as distributed task distribution. Please see the picture below:

Topic: Exchange Type is topic. When sending messages, the switch and Routing Key need to be specified. The message queue of consumers is bound to the switch and matches the Routing Key to realize the message subscription. A message can be received only if the consumer binds the queue to the switch and the specified Routing Key matches the matching rules.

Routing Key can be set to a wildcard character, such as * or # (* indicates that the message matches a word in the Routing Key, and # indicates that any message with Routing Key can be received). If the Routing Key consists of multiple words, use the Routing Key between the words. To separate.

Naming conventions:

Ex{AppID}.{user-defined ExchangeName} is recommended for the switch name, and MQ{AppID}.{user-defined QueueName} is recommended for the QueueName.

6. Demo download and more information

RabbitMQDemo download: github.com/das2017/Rab…

RabbitMQ’s official website: www.rabbitmq.com

The list of topics covered in this series (not published in that order) is as follows. If you are interested, please follow:

  • Introduction: Small and medium-sized R&D team architecture practice three points
  • Cache Redis
  • Message queue RabbitMQ
  • Centralized log ELK
  • Task scheduling Job
  • Apply monitoring Metrics
  • Microservices framework MSA
  • Solr
  • Distributed coordinator ZooKeeper
  • Gadgets: Dapper.NET/EmitMapper/AutoMapper/Autofac/NuGet
  • Release tool Jenkins
  • Overall architecture design: How to do e-commerce enterprise overall architecture?
  • Single project architecture design
  • Uniform application layering: How to standardize all application layering in a company?
  • Debugging tool WinDbg
  • Single sign-on (sso)
  • Enterprise Payment Gateway
  • “Article

The authors introduce

Zhang Huiqing, an IT veteran of more than 10 years, has successively served as architect of Ctrip, chief architect of Gooda Group, CTO of Zhongqing E-Travel and led the upgrading and transformation of the technical architecture of the two companies. Focus on architecture and engineering efficiency, technology and business matching and integration, technology value and innovation.

Yang Li has years of experience in Internet application system research and development. She used to work in Gooda Group and now works as the system architect of Zhongqing E-travel. She is mainly responsible for the architecture design of the business system of the company’s R&D center, as well as the accumulation and training of new technologies. The current focus is on open source software, software architecture, microservices and big data.

Thanks to Yuta Tian guang for correcting this article.