Author: HelloGitHub- Lao Xun

Hello, here is HelloGitHub’s HelloZooKeeper series, free, open source, fun, entry-level ZooKeeper tutorials for beginners with basic programming skills.

ZooKeeper is a software project of the Apache Software Foundation that provides open source distributed configuration services, synchronization services, and naming registries for large-scale distributed computing. ZooKeeper was once a subproject of Hadoop, but is now a top independent open source project.

This series of tutorials is to explain ZooKeeper from scratch, from the most basic installation and use to the principle and source code behind the explanation, the whole series hope to let ZK knowledge “drill” into your smart brain through interesting text, humorous atmosphere. This tutorial is open: open source and collaborative, so whether you’re a novice or veteran driver, we hope you can contribute to make this tutorial even better:

  • Novice: participate in the revision of the wrong words, sentences, spelling, typesetting and other problems in the article
  • Users: Participate in content discussions, answering questions, and helping others
  • Old driver: Get involved in writing the article and put your name in the author column

Project address: github.com/HelloGitHub…

From today’s beginning, began to enter the principle of ZK explained, I will try to the principle behind the analogy of interesting points, we rest assured to look down ~

A new colleague in the office

Well, let’s see what’s going on in animal Village.

1.1 Ma Guoguo is going to be the boss

After all, Ma Guoguo was old, and it was too much for him to receive so many villagers every day. Finally, he decided to apply to the village committee to find out if he could hire more people so that he could live as a boss. After much consideration, the village committee finally agreed and decided to let Ma Guoguo go to the neighborhood to find suitable people to work. Ma Guoguo has been very focused on physical exercise, so every two days bubble in the gym, so began to look for candidates in the gym, and finally selected three strong young man, and even nickname have given them a good, a more than 80 kilograms called small P, a more than 90 kilograms called small S, the last 200 jin called small F!

When the hiring was complete, the layout of the office was changed to look like this:

Ma Guoguo is now comfortable in the rear as a manager, the work in front of the handover to the three guys, the three guys each work is relatively simple, let’s look at it one by one

1.2 Careful little P

As the first employee of the office contacted by the villagers, P would first check the villagers who would come to deal with the affairs and make some simple inquiries:

It must be mentioned here that if there is an abnormal error, P will not terminate the service for the villager, but will also continue to guide him to the counter of S

1.3 Serious little S

S’s work is also very simple. Let’s directly look at the flow chart:

S, the office’s recorder, keeps a memo handy:

This memo does not care who registers, just notes down the details of the registration (even the exceptions marked with a small P), and then files them periodically.

1.4 Capable little F

Small F as once the hercules and Ma Guoguo is played, and lost to ma Guoguo by a small advantage, but the generous small F did not put these old things in mind, still willing to help Ma Guoguo, is really a respected good comrade!

Every time S finishes filing, he will hand over those affairs to F. As the last salesman in the office, F holds two core documents handed to him by Ma Guoguo: red book and Yellow Book!

Need to seriously record the request of the villagers in the small red book, but also need to check whether there is a need to notify the villagers in the small Yellow Book, instead of Ma Guoguo to call them.

It seems that little F does very little, but in fact, it is the most, but I have simplified the logic of the little red book and the little Yellow Book, and I will explain the little red book and the little Yellow Book separately later, so that we can water another issue.

Two, the well-organized behind

This is the end of the story. Here is the translation in ape language:

I used these three names to impress you:

  • Small PIn the corresponding codePrepRequestProcessor
  • Small SIn the corresponding codeSyncRequestProcessor
  • Small FIn the corresponding codeFinalRequestProcessor

When the server is started, the three processors are chained in the order P -> S -> F, and P and S are themselves thread objects that are started with the server.

Both P and S start up using an infinite loop to process the main logic, while ZK uses a very classic pattern to process the main logic: producer and consumer! Each maintains a blocking queue, separating the logic that receives and processes requests, and by design improves throughput and performance.

In fact, this pattern can be seen everywhere in ZK, and we will talk about it when we encounter it later. Let’s dig a little deeper into the processing logic of the three processors.

2.1 PrepRequestProcessor

As can be seen from the process, PrepRequestProcessor does not involve in memory operations and file operations, and as the first processor, it is mainly responsible for checksum marking tasks.

2.2 SyncRequestProcessor

As an additional note, the two pink boxes in the flow determine whether to take a snapshot and whether to archive, respectively:

  • Snapshot or not: The number or size of transaction records is greater than a certain point, and the number of transaction records is a random number (reset after each snapshot)
  • Archived: Whether the time of the last archiving and the current time exceed the configured interval (default is 0), or the number of records in a transaction exceeds the configured interval (default is 1000).

These two judgments are used to control the frequency of snapshots and archiving:

  • If the frequency is low, more data is written to the disk at a time, which improves the performance but reduces the disaster recovery capability
  • If the frequency is high, the performance is affected, but the Dr Capability is strong

2.3 FinalRequestProcessor

It seems that there is no difference between this image and the one above (Orz), but the details are in the little red and yellow books, so it will be left to the next chapter.

Why is an exception on the side of small P not returned directly to the client, but passed backwards to small F to respond? I think in order to unify the responsibilities of each processor, the client response is handled by small F.

Here I small reveal the plot, the above scene is actually the ZK standalone version of the processing scene, if changed to the cluster version will add more processors in the chain, will involve the cluster after talk.

Third, summary

This section describes how ZK handles client requests in standalone version, and separates different logic into different objects to handle them through a chain of responsibilities. In the next chapter, we will formally enter the memory model and the implementation of notification mechanism. Let’s have a look at the two core books in The hands of Ma Guoguo

Since this chapter begins to explain the principle of ZK, it is difficult for an article and me to cover all aspects, so if you have any questions in the article, you can also make suggestions or questions about the principle of ZK, you can come to the topic I created to discuss, which is convenient for recording and answering questions:

Address: www.yuque.com/kaixin1002/…

I will create a topic for each article so that you can discuss it and clearly describe your problem.

Follow the HelloGitHub public account to receive the first updates.

There are more open source projects and treasure projects waiting to be discovered.