This is the sixth day of my participation in the November Gwen Challenge. See details: The Last Gwen Challenge 2021.

I was asked to look at Pulsar. My previous messaging middleware was mostly RabbitMQ and Kafka. The project manager said Pulsar would be used. I haven’t used it before, so I’m not sure. If I’m going to use it, I’ll definitely study it.

If you want to learn Pulsar, please download it by yourself.

Link: pan.baidu.com/s/1H2qcygZO…

Extraction code: PBZJ

Now it is basically used in the project. Would you like to talk about Pulsar framework briefly? What I know at present is basically acquired on the Internet, basic knowledge points, have not gone deep into, many pits have not stepped on. Recently, I have a lot of work in the project, so I have no time to write articles.

Let’s start by understanding what Pulsar is. With all the messaging middleware out there, why build Pulsar?

What is Pulsar?

Go to the official website first, learn new technology, the official website is certainly the first choice.

Liverpoolfc.tv: pulsar.apache.org/docs/zh-CN/…

Pulsar is a server-to-server messaging system with multi-tenancy, high performance, and more.

Pulsar, which originated at Yahoo, was opened source and donated to the Apache Foundation in 2016 and upgraded to Apache Top Level project status in September 2018.

Pulsar is a messaging platform that has been very popular for the past two years. It’s being touted as the next generation messaging platform that is poised to replace Kafka.

Basic architecture of Pulsar

  • Pulsar adopts the architecture of storage and computation separation, and Pulsar uses BookKeeper to store messages. Bookkeeper ensures the reliability and high efficiency of message storage, and BookKeeper provides Pulsar with storage expansion capability.
  • Pulsar uses ZK for metadata storage.
  • Multi-tenant, Pulsar was originally designed to support multi-tenant.
  • Namespaces: A tenant can have multiple namespaces, one topic belongs to one namespace, and configurations in PulSAR are configured on a namespace basis.

  • Pulsar’s broker handles message reads and writes. There is a local cache of messages in the broker, because in most scenarios messages are consumed immediately after being written, so the cache of new messages held in the broker can greatly improve performance and overall MQ throughput.

Compared to MQ such as Kafka and RocketMQ, Pulsar’s bookKeeper-based store-and-compute separation architecture allows pulsar’s message store to scale independently of the broker.

multi-tenant

Multi-tenant is a required feature that isolates data of different services and teams in the same cluster.

persistent://core/order/create-order
Copy the code

Take this topic name as an example. Under the tenant core, there is a namespace of order, which is ultimately the topic name of create-Order.

In practice, tenants are generally divided by service team, and namespaces are different services of the current team. This makes it clear to manage the topic.

It’s usually the comparison that hurts. How does this work in message-oriented middleware without multi-tenancy:

  1. Simply not so fine, all lines of business mixed, when the team is small may not be a problem; Once the business increases, it can be very difficult to manage.
  2. You’re doing a layer of abstraction before topic, but you’re essentially implementing multi-tenancy.
  3. Each business team maintains its own cluster, which of course solves the problem, but also increases the operational complexity.

The above is very intuitive to see the importance of multi-tenancy.

application

In addition to the upper application, such as producer, consumer, such as the concept and use of everyone is similar.

For example, Pulsar supports four consumption modes:

  • Exclusive: Exclusive mode, where only one consumer can start and consume data; throughSubscriptionNameIndicate the same consumer), the scope of application is smaller.
  • FailoverFailover mode: More than one can be started simultaneously in exclusive modeconsumer, once aconsumerThe rest can be quickly replaced, but only oneconsumerCan consume; Some scenarios are available.
  • SharedShared mode: can have N consumers running at the same time, message according toround-robinPolling post to eachconsumer; When aconsumerOutage noack, the message will be delivered to other consumers. This consumption pattern can increase consumption power, but messages cannot be ordered.
  • KeySharedShared mode: Based on shared mode; Equivalent to the sametopicThe messages in the same group can only be orderly consumed by the same consumer.

The third shared consumption pattern should be the most used, and the KeyShared pattern can be used when there is an order requirement for messages.

I am not going to introduce the basic knowledge of Pulsar here, I also copied the introduction, I think it is well written by others, you might as well practice it yourself, so as to deepen your impression, pulsar can only be installed under Linux system at present. It’s easy to use, just a few lines of code. Try more, software industry technology update is so fast, insist on learning, in order not to be eliminated.

conclusion

The pulSAR community has been developing and growing. The development of PulSAR technology and the increase of the number of use cases have formed a virtuous cycle, and pulSAR ecology is also growing.

Pulsar also has many advantages that enable it to stand out in a unified messaging and event flow platform and become the choice of more people. Compared to Kafka, Pulsar is more flexible and easier to operate and scale.

If you are interested, you can do research. Compared with fishing for a day, it is better to use time to learn more knowledge that you are interested in. In fact, you can learn not only technical articles, but also astronomy and geography (I like to read knowledge about the universe).