preface

The text has been included in my GitHub repository. Welcome to Star: github.com/bin39232820… The best time to plant a tree was ten years ago, and the second best time is now

omg

Messaging middleware is a necessary skill for our background developers. Generally speaking, I know and use two kinds of messaging middleware, one is RabbitMQ and one is RocketMQ. Before, I can only say that I have used messaging middleware in general, but I think it is necessary to learn more about messaging middleware. So I’m going to write well, why rocketMQ, because it’s written in Java, so it looks like the source code is a little bit easier, so I’m going to learn. Of course, he still has a lot of advantages, then slowly to learn

MQ is introduced

Why use MQ

Message queues are a first-in, first-out data structure

The application scenarios include the following three aspects

  • The application of decoupling

The higher the coupling of the system, the lower the fault tolerance. Take the e-commerce application as an example. After a user creates an order, if the inventory system, logistics system and payment system are coupled, and any one of the subsystems is faulty or temporarily unavailable due to upgrade or other reasons, the ordering operation will be abnormal and the user experience will be affected.With message queue decoupling, the coupling of the system is improved. For example, when the logistics system fails, it takes several minutes to repair. During this time, the data to be processed by the logistics system is cached in the message queue and the user’s order operation is completed normally. When the logistics system replies, the order messages in the message queue can be processed, and the terminal system does not perceive the failure of the logistics system for several minutes.

  • Traffic peak clipping

Application systems may be overwhelmed by a sudden surge in system request traffic. With message queues, large numbers of requests can be cached and processed over long periods of time, which can greatly improve system stability and user experience.

In general, in order to ensure the stability of the system, if the system load exceeds the threshold, it will prevent user requests, which will affect the user experience, rather than using message queues to cache requests and wait for the system to finish processing to notify the user of the completion of the order, which is not a good experience.

For economic purposes:

If the QPS of the service system is 1000 during normal hours, the traffic peak is 10000. It is not cost-effective to configure a high-performance server to cope with the peak traffic. In this case, you can use message queues to peak the peak traffic

  • Data distribution

Message queues allow data to flow more widely across multiple systems. The producer of the data does not need to care about who uses the data, but simply sends the data to the message queue, where the consumer gets the data directly from the message queue

The advantages and disadvantages of MQ

Advantages: Decoupling, peak-cutting, and data distribution disadvantages include the following:

  • The system availability decreases

    The more external dependencies introduced into the system, the worse the stability of the system. When MQ goes down, there is an impact on business.

    How to ensure high availability of MQ?

  • System complexity increases

    The addition of MQ has greatly increased the complexity of the system, which used to be synchronous remote calls between systems, and now asynchronous calls are made through MQ.

    How do YOU ensure that messages are not consumed repeatedly? How do I handle message loss? What about sequentiality of message delivery?

  • Consistency problem

    After processing services, system A sends message data to systems B, C, and D through MQ. If systems B and C process data successfully, system D fails to process data.

    How to ensure the consistency of message data processing?

Comparison of various MQ products

Common MQ products include Kafka, ActiveMQ, RabbitMQ, and RocketMQ.

Check out the rocketMQ website

In fact, learning a thing, of course, the first thing is to go to its official website, let’s take a look

  • Apache RocketMQ™ is a unified messaging engine, lightweight data processing platform.
  • Apache RocketMQ™ is a standardized messaging engine, lightweight processing platform
  • Unified Messaging Engine: Standardized messaging engine
  • Lightweight Data Processing platform: a lightweight processing platform

  • Low Latency

Low latency

  • More than 99.6% response latency within 1 millisecond under high pressure.

Under high pressure, it can respond to more than 99.6% of requests with 1 millisecond delay

  • Finance Oriented

Financial direction, oriented to finance (stability of financial business)

  • High availability with tracking and auditing features.

The system has high availability to support tracking and auditing features

  • Industry Sustainable

Industry development

  • Trillion-level message capacity guaranteed.

A trillion-level message capacity is guaranteed.

RocketMQ version of message queue is a distributed messaging middleware with low latency, high concurrency, high availability and high reliability built by AliYun based on Apache RocketMQ. The RocketMQ version of message queues provides asynchronous decoupling and peak load shifting capabilities for distributed applications, as well as features such as massive message stacking, high throughput, and reliable retry required for Internet applications.

RocketMQ is the core of MetaQ on the shoulders of giants (Kafka) and has been optimized to make it more suitable for Internet companies. It is pure Java development, with high throughput, high availability, suitable for large-scale distributed system applications. RocketMQ is currently widely used in Alibaba Group for transaction, recharge, flow calculation, message push, log streaming processing, binglog distribution and other scenarios.

Rocketmq messaging middleware functionality

  • Publish/subscribe messaging model
  • Financial level trading news
  • Various cross-language clients such as Java, C/C ++, Python, Go
  • Pluggable transport protocols such as TCP, SSL, AIO
  • Built-in message tracing, but also support for open tracing
  • Multi-functional integration of big data and streaming ecosystems
  • Time or offset trace messages
  • Reliable FIFO and strictly ordered messaging in the same queue
  • Efficient push and pull consumption model
  • The cumulative capacity of millions of messages in a single queue
  • Multiple messaging protocols, such as JMS and OpenMessaging
  • Flexible distributed scale-out architecture
  • Lightning-fast batch message exchange system
  • Various message filter mechanisms, such as SQL and Tag
  • Docker images for isolation tests and cloud isolation clusters
  • Rich management dashboard for configuration, metrics, and monitoring
  • Authentication and Authorization

  • Rocketmq-broker: Receives messages from the producer and stores them (by calling RocketMQ-Store), where consumers get the messages
  • Rocketmq-client: Provides a client API for sending and receiving messages.
  • Rocketmq – Namesrv: NameServer, similar to Zookeeper, where the message’s TopicName, queue and other runtime meta information are stored.
  • Rocketmq-common: Common classes, methods, data structures, etc.
  • Rocketmq-remoting: Client /server based on Netty4 + FastJSON serialization + custom binary protocol
  • Rocketmq-store: Message, index store, etc.
  • Rocketmq-filtersrv: Message filter Server. Note that to implement this filter, you need to upload code to MQ!

A quick start to RocketMQ

Download RocketMQ

Download address

Environmental requirements

  • On a system

  • JDK1.8(64位)

  • Source code installation requires Maven 3.2.x installed

Install RocketMQ

Install in binary package mode

  1. Decompress the Installation Package
  2. Going to the installation directory

catalogue

  • Bin: startup scripts, including shell scripts and CMD scripts
  • Conf: instance configuration files, including broker configuration files and logback configuration files
  • Lib: relies on JAR packages, including Netty, Commons-lang, and FastJSON

Start the RocketMQ

  1. Start the NameServer
# 1. Start NameServer nohup sh bin/ mqNamesrv & # 2 Check the startup log tail -f ~ / logs/rocketmqlogs/namesrv logCopy the code
  1. Start the Broker
Nohup sh bin/ mqBroker -n localhost:9876 & # 2. Check the startup log tail - f ~ / logs/rocketmqlogs/broker. The logCopy the code
  • Problem description:

The default RocketMQ VM has a large memory. If the Broker fails to start due to insufficient memory, you need to edit the following two configuration files to change the JVM memory size

# Edit runbroker. Sh and runserver.sh to change the default JVM size vi runbrokerCopy the code

JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

Test RocketMQ

  1. Send a message
Set environment variable export NAMESRV_ADDR=localhost:9876 # 2. Use the installation package Demo messaging sh bin/view sh org. Apache. Rocketmq. Example. Quickstart. ProducerCopy the code
  1. Receives the message
Set environment variable export NAMESRV_ADDR=localhost:9876 # 2. Receive messages sh bin/tools. Sh org. Apache. Rocketmq. Example. The quickstart. ConsumerCopy the code
  1. Close the RocketMQ
NameServer sh bin/mqshutdown namesrv # 2. Close the Broker sh bin/mqshutdown BrokerCopy the code

At the end

Ok, so that’s the end of the first post. So far, we’ve taken a look at rocketMQ’s four core characters and seen how they work, and then we can start to learn the API and learn how they work. Ok

Daily for praise

Ok everyone, that’s all this article is about, can see here people ah, are really fans.

Creation is not easy, your support and recognition, is the biggest motivation for my creation, we will see the next article

Six pulse excalibur | article “original” if there are any errors in this blog, please give criticisms, be obliged!