Author: HelloGitHub- Lao Xun
Hello, here is HelloGitHub’s HelloZooKeeper series, free, open source, fun, entry-level ZooKeeper tutorials for beginners with basic programming skills.
ZooKeeper is a software project of the Apache Software Foundation that provides open source distributed configuration services, synchronization services, and naming registries for large-scale distributed computing. ZooKeeper was once a subproject of Hadoop, but is now a top independent open source project.
ZK is often used to see in the actual development work, which is the guy to eat, that can be played through, with the right hand, or how to advance and promotion pay? Learn how to use HelloGitHub
This series of tutorials is to explain ZooKeeper from scratch, from the most basic installation and use to the principle and source code behind the explanation, the whole series hope to let ZK knowledge “drill” into your smart brain through interesting text, humorous atmosphere. This tutorial is open: open source and collaborative, so whether you’re a novice or veteran driver, we hope you can contribute to make this tutorial even better:
- Novice: participate in the revision of the wrong words, sentences, spelling, typesetting and other problems in the article
- Users: Participate in content discussions, answering questions, and helping others
- Old driver: Get involved in writing the article and put your name in the author column
Project address: github.com/HelloGitHub…
Today we will explain how to use the Java code client to manipulate ZK.
First, basic operation
1.1 New regulations of Magoguo
As always, before we get into the actual combat, I will tell you a short story (the characters in the story are completely fictional, please do not compare them, if there is any similarity, it is pure coincidence).
Since Ma Guoguo served as the head of the office, every day that is busy, the villagers have something to come to him, his little book has been densely recorded a lot of:
Especially chicken too the United States, more peremptory has become the day UP, daily updated frequently make the horse blended feeling overwhelmed, he thought, if such a hash of it, not only oneself will be more and more tired, later after their retirement, somebody else to handover will not start, that also not behind my mismanagement, pointing to me. I’m afraid I’ll have to apologize to the whole village.
Therefore, the office issued a new regulation, each time the villagers who came to register must classify their affairs to be registered, and Ma Guoguo recorded them according to these classifications, so Ma Guoguo’s notes (hereinafter referred to as: little red book) became like this:
But after implementing the rules for some time, the villagers’ representatives, led by Jitaimei and Ma Xiaoyun, proposed to Ma Guoguo: “We are old acquaintances, and every time we come to visit, we have to announce our home. Can we do something to make it easier for the people? Isn’t that the purpose of your office to serve our people?”
Ma Guo Guo listened to also feel very reasonable, so to each old acquaintance have created a label. For example, in the future, records created by Jitaimei were directly placed under the label Jitaimei, so that jitaimei only needs to care about what they want to record, so the notebook ended up like this:
It is also the same for receiving notification to villagers. Magoguo will note next to the event that needs notification. For example, Kun Kun, the number one fan of Jitaimei, is very interested in the dancing video of Jitaimei, so he will note next to the dancing event of Jitaimei:
Then take out another Yellow Book and write down who needs to be notified:
With the passage of time, ji Tai Beauty’s popularity is increasing, and now even Ma Xiaoyun, Dongdong have become her fans, have to pay attention to her updates:
So now when Ma Guoguo records the little red book, he will see if other people subscribe to the notice of the current affairs. If so, he will take out the little Yellow Book to find the corresponding villagers who need to be notified, and call them one by one.
Ma Guoguo is very satisfied with his regulations this time, when facing the reporter interview, proud of the said, this is his retirement persistence in learning the computer, from the computer file directory to get inspiration, people really want to “live and learn” ah!
This is the end of the story. Here is the translation in ape language:
ZK defines that each record must have a corresponding path, which is the classification specified by the office in the corresponding story. The structure of the whole record is indeed similar to the file tree in Linux, with a root node/and parent-child relationship between nodes. The path is divided by /, for example:
/ Chicken too beautiful/update video/dance /20201101Copy the codeThe tag in the story, which is actually the chroot specified by the client, is actually maintained by the client without the server knowing it.
1.2 Code Practice
In particular, the next step is to use the official Java client as a demonstration, create a new empty Maven project, and introduce ZK dependencies:
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.6.2</version>
</dependency>
Copy the codeTo manipulate ZK, you need to create a client object. Let’s take Chicken Taimei as an example
ZooKeeper client = new ZooKeeper("127.0.0.1:2181/ Chicken is beautiful".3000.null);
Copy the codeThe first string of ZooKeeper is the server address of the connection, followed by chroot, which is the label in the small story. After that, all operations of the client will be handled with/jicaimei as the top-level path.
Finally, when the client exits, remember to close the client
client.close();
Copy the code1.2.1 Creating a Path
Note that the official client does not have recursive creation, so when creating a multilevel path, the client needs to ensure that the parent node in the path exists!
Unsafe. zoodefs.ids. OPEN_ACL_UNSAFE is a type of ACL permission that does not need to be checked. And we’ll talk about it later, so I’ll skip it.
client.create("/ Update video/Dance /20201101"."This is Data, you can either record some business Data or you can write whatever you want.".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
Copy the codeThe final createmode. PERSISTENT indicates that the current node is of a PERSISTENT type. There are 7 types in 3.6.2, which are listed below and explained briefly:
PERSISTENT // A persistent node, once created, will not be deleted unless the client initiates a deletion request
PERSISTENT_SEQUENTIAL // A persistent sequential node, which concatenates a non-repeating prefix suffix after the user path, as above
EPHEMERAL // A temporary node that is automatically deleted when the client that created the node is disconnected
EPHEMERAL_SEQUENTIAL // Temporary sequence nodes, basically the same as above, also add a numeric suffix
CONTAINER // The container node is deleted by the server once the child node has been deleted
PERSISTENT_WITH_TTL // A persistent node with an expiration time, a node with a timeout period, will be deleted if no child nodes are created within the timeout period
PERSISTENT_SEQUENTIAL_WITH_TTL // Persistent sequential nodes with expiration time, basically the same as above, with a numeric suffix
Copy the codeYou may be familiar with the first four types, but are not familiar with the last three types, especially the last two types with TTL, these two types are not supported by default ZK configuration, need to add extendedTypesEnabled=true to zoo. CFG configuration to enable the extension function. Otherwise you will receive an Unimplemented for error.
In the example, after the path is created, it looks like this:
Chicken is beautiful | - update video dance | -- - | - 20201101Copy the code1.2.2 Deleting a Path
The official client also does not support recursive deletion, so you need to ensure that the node to be deleted is a leaf node, otherwise you will receive an error, so we delete 20201101 here:
client.delete("/ Update video/Dance /20201101", -1);
Copy the code-1 is a version field, which is equivalent to the optimistic locking mechanism provided by ZK. If it is -1, the version information of the node is ignored.
This is what happens when you delete:
Chicken is beautiful | - update video | - dancingCopy the code1.2.3 Setting Data
Each node can have its own data, which can be specified at creation time or later through Settings.
client.setData("/ Update video/Dance"."This is Data, I can write some parameters about the business.".getBytes(), -1);
Copy the codeThe meaning of -1 is the same as that in the deletion path, but the version information is ignored.
1.2.4 Checking whether a path exists
Since neither create nor delete supports recursion, the existence of the target path needs to be checked to decide whether to proceed to the next step
Stat stat = client.exists("/ Update video".false); System.out.println(stat ! =null ? "There" : "Doesn't exist."); / / there
Copy the codeFalse means do not subscribe, about subscription will be said later.
1.2.5 Obtaining Data
If you can set data, you can also get data, so ZK can play the role of data store occasionally
byte[] data = client.getData("/ Update video/Dance".false.null);
System.out.println(new String(data)); // This is Data, you can write some parameters about the business
Copy the code1.2.6 Obtaining the Child Node List
ZK is a tree structure with parent-child nodes, so you can query all the children of a node
List<String> children = client.getChildren("/ Update video".false);
System.out.println(children); // [dancing]
Copy the code1.2.7 Setting a Subscription
All three methods (including their overloaded methods) can subscribe to a path in one of two ways: check the existence of a path, get data, and get a list of child nodes.
- Pass a
booleanValue, in which case the callback object is createdZooKeeperThe third parameter ofdefaultWatcher, except in the previous examplenull - Pass one directly into the method
WatcherThe implementation class is used as a callback object after this path (recommended)
The following are demonstrated respectively:
1 / / way
ZooKeeper client = new ZooKeeper("127.0.0.1:2181/ Chicken is beautiful".3000.new Watcher() {
// This is the defaultWatcher parameter, which is the current client's default callback implementation
@Override
public void process(WatchedEvent event) {
System.out.println("This is the default callback object for this client globally."); }});// exists
client.exists("/ Update video".true);
// getData
client.getData("/ Update video/Dance".true.null);
// getChildren
client.getChildren("/ Update video".true);
Copy the code2 / / way
// exists
client.exists("/ Update video".new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("I'm the implementation of the callback object."); }});// getData
client.getData("/ Update video/Dance".new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("I'm the implementation of the callback object."); }},null);
// getChildren
client.getChildren("/ Update video".new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("I'm the implementation of the callback object."); }});Copy the codeHow the callback triggers the corresponding method each time is a mystery here and will be explained in more detail later.
About ZK client so roughly several operation, limited to the space I can’t one by one, for example, the purpose of this article series is not an official document translation, important is to inspire enthusiasm for technology, the rest of those who use it when I gave all the exercises after class ~ about ZK’s basic operation is finished.
Second, advanced operation
Once we’re done with the basics, let’s move on to something more advanced.
2.1 Chicken Beauty book signing
Let’s continue with the story of animal Village.
With the growing popularity and attention of live broadcast, Jitaimei has become a big star in animal Village, and has released an album, so I plan to hold a book signing in return for the fans.
Decided to decorate the site in front of the office of Ma Guoguo, by the strong tai Chi master Ma Guoguo as security to ensure the order of the scene
Ma Guoguo said that if you want to go in and meet one-to-one fans of Chicken Taimei, you need to take away the voucher in my hand, come out quickly after signing, and return the voucher to me.
Sorry, I put the wrong picture
Chicken too beautiful fans heard can meet one on one with the star, are crazy, are in a hurry to the door of the office, do not know who shouted in the crowd: “rob ah! First come, first served!” , like a hungry tiger pounced on the general Ma Guo Guo to the ground, the scene is quite chaotic!
Finally, the chicken is too beautiful hardcore fan Kun Kun took the lead, grabbed the only certificate in the hands of Ma Guoguo, in exchange for the opportunity to one on one with the star idol
Kun Kun holding the center of love album, satisfied to go back.
Ma Guoguo, who took back the voucher, looked at the group of hungry wolves
Suddenly I knew what I had to face next…
(After 4 hours)
Finally, all the fans took the album in hand is still warm and happy to go home. Busy a day of ma Guoguo thought next time can not be so, if it were not for my old and healthy fear is not to be carried into the hospital!
2.2 The Chicken is Too Beautiful concert
The number of fans has finally surpassed 100 million. It was time to find a reason to market themselves again, so I discussed with the agency whether I could hold a concert and sell tickets on the spot, which could not only create publicity for myself but also meet the requirements of the fans. This time the same found Ma Guogo, I hope ma Guogo can continue to help organize the order of the scene, the ma Guogo did not want to take these chores, but after hearing the price of the brokerage company…
However, he was too old to withstand the last time, so he decided to issue a new rule to deal with the crazy behavior of fans. The new venue was arranged like this:
Every fans have to pick up a the horse blended from small to large, the number of the blended each time starting from 1, hair also call, from the smallest number to call, a call to the number of fans before the ticket office to buy tickets, will focus on discharge after every fans to get the number one in front of his situation, if he bought, oneself hurriedly to prepare, Because it’s gonna be your turn next.
In this way, the whole sale scene is in perfect order, everyone praises Ma Guoguo’s superb management skills in succession.
The little story is finished again, and the following is translated in ape language:
These two stories tell the general principle of ZK distributed lock, and basically correspond to the two cases of unfair lock and fair lock, although the actual situation and the story will be different, but through the story hope to give you a perceptual understanding.
The disadvantage of not fair lock and embodied in the story, is a current and holding the lock release process, after all the process of waiting for the lock will be notified, this is often mentioned in the interview question “flock effect”, thus to compete for the lock, but because only one process can get the lock, other processes will continue to wait for again, Therefore, in the case of dealing with high concurrency scenarios, this scheme has serious performance problems and greatly increases the pressure on the server side.
In the case of fair lock, each process that does not acquire the lock usually only needs to care about the situation of the previous process, and only one process will be awakened at a time. Therefore, if ZK is used as the middleware of distributed lock, it is recommended to adopt fair lock.
2.3 Actual Code
Below is a simple (pseudo) code demonstration, how to use ZK to write distributed lock logic
2.3.1 Unfair Lock
Let’s say the object I want to lock now is the concert of Chicken Taimei
ZooKeeper client = new ZooKeeper("127.0.0.1:2181".3000.null);
try {
// It was mentioned earlier that you must ensure that the chicken too beautiful path exists
client.create("/ The Chicken is beautiful/Concert"."Data is not written casually.".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
// Create the lock successfully, then execute the business logic
System.out.println("I'm the business logic after I get the lock."); .// Delete the node after processing services to release the lock. In actual scenarios, this step should be deleted in the finally block
client.delete("/ The Chicken is beautiful/Concert", -1);
} catch (KeeperException.NodeExistsException e) {
// If NodeExistsException is reported, the lock is not obtained
System.out.println("The lock was taken by someone else.");
// Listen on this node
client.exists("/ The Chicken is beautiful/Concert".new Watcher() {
@Override
public void process(WatchedEvent event) {
if (event.getType().equals(Event.EventType.NodeDeleted)) {
// If the delete event is detected, the previous process released the lock
// This refers to the recursive process of listening for the refetch failure. In fact, it requires a wrapped method similar to the lock method, which will not be demonstrated here
client.create("/ The Chicken is beautiful/Concert"."Data is not written casually.".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL); . }}}); }...Copy the codeZK’s unfair lock uses the same path and cannot be repeatedly created, plus the feature of temporary nodes. Temporary nodes are used because if the process that acquired the lock crashes and cannot release the lock in time, it will cause a deadlock. However, the temporary node will be deleted automatically after the client is disconnected, so it avoids the risk of deadlock.
2.3.2 fair lock
Same concert, this time switch to fair lock to try
ZooKeeper client = new ZooKeeper("127.0.0.1:2181".3000.null);
String currentPath = client.create("/ The Chicken is beautiful/Concert"."Data is not written casually.".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
// The creation will be successful because the order number exists
// Get the names of all the children under the parent node
List<String> children = client.getChildren("/ The chicken is beautiful.".false);
/ / first order
Collections.sort(children);
if (children.get(0).equals(currentPath)) {
// The current path is the smallest node, obtaining the lock successfully
System.out.println("I'm the business logic after I get the lock."); .// Also remember to delete this node after processing services
client.delete(currentPath, -1);
} else {
// Not the smallest node, failed to obtain the lock
// Get the node whose ordinal number is 1 less than your own among all children based on the current node path
String preNode = getPreNode(currentPath, children);
// Listen on the node
client.exists(preNode, new Watcher() {
@Override
public void process(WatchedEvent event) {
if (event.getType().equals(Event.EventType.NodeDeleted)) {
// As with an unfair lock, try to acquire the lock again, because of the order of the nodes, so the lock should not fail this time. }}}); }...Copy the codeZK’s fair lock uses the temporary sequence node, the sequence number cannot be repeated, the current smallest node is considered to have obtained the lock successfully.
2.3.3 Curator for Recipes
You might ask, well, neither of these pieces of code works directly, so what if I want to use them in my project?
This kind of live open source community has already been done for us
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
<version>5.1.0</version>
</dependency>
Copy the codeHere’s a simple example
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000.3);
CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", retryPolicy);
client.start();
InterProcessMutex lock = new InterProcessMutex(client, "/lock");
try (Locker locker = new Locker(lock)) {
// Use the try-with-resources syntax sugar to automatically release the lock
System.out.println("Business logic after obtaining the lock");
}
client.close();
Copy the codeExhibit Curator has several built-in locks for us to use, and all work with Locker packaging
InterProcessMultiLockMultiple paths can be locked and released simultaneouslyInterProcessMutexReentrant exclusive lockInterProcessReadWriteLockRead-write lockInterProcessSemaphoreMutexDo not reenter exclusive locks
If you want to see how to write a great ZK distributed lock, check out the source code for co-recipes
Exhibit exhibit exhibit exhibit exhibit exhibit exhibit Exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit exhibit
2.3.4 Integration with Spring Boot
Is there an easier one than this? Of course!
I have a demonstration project ready for you:
- Project address: github.com/HelloGitHub…
Note that this project is only intended to demonstrate how to integrate toth into Spring Boot, everything is configured in a simple way, and only demonstrates distributed locking. Other advanced features are available if you need them!
I’m not going to show you the regular Spring Boot dependencies, I’m just going to list the ones that are associated with that:
The project uses Maven as a dependency management tool
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-zookeeper</artifactId>
</dependency>
Copy the codeAn @Configuration object to create the corresponding Bean
@Configuration
public class ZookeeperLockConfiguration {
@ Value (" ${zookeeper. Host: 127.0.0.1:2181} ")
private String zkUrl;
@Bean
public CuratorFrameworkFactoryBean curatorFrameworkFactoryBean(a) {
return new CuratorFrameworkFactoryBean(zkUrl);
}
@Bean
public ZookeeperLockRegistry zookeeperLockRegistry(CuratorFramework curatorFramework) {
return new ZookeeperLockRegistry(curatorFramework, "/HG-lock"); }}Copy the codeTwo interfaces for testing
// Inject directly into the bean where the lock is needed
@Resource
private LockRegistry lockRegistry;
@GetMapping("/lock10")
public String lock10(a) {
System.out.println("lock10 start " + System.currentTimeMillis());
final Lock lock = lockRegistry.obtain("lock");
try {
lock.lock();
System.out.println("lock10 get lock success " + System.currentTimeMillis());
TimeUnit.SECONDS.sleep(10);
} catch (Exception e) {
} finally {
lock.unlock();
}
return "OK";
}
@GetMapping("/immediate")
public String immediate(a) {
System.out.println("immediate start " + System.currentTimeMillis());
final Lock lock = lockRegistry.obtain("lock");
try {
lock.lock();
System.out.println("immediate get lock success " + System.currentTimeMillis());
} finally {
lock.unlock();
}
return "immediate return";
}
Copy the codeTo explain the logic, I call the LOCK10 interface first and then the IMMEDIATE interface. The lock10 interface will sleep for 10 seconds after acquiring the lock, and the immediate interface will attempt to acquire the lock, but will not sleep. Assuming the distributed lock is valid, the immediate interface will wait for 10 seconds. However, the time of obtaining the lock was about 10 seconds different, proving that the lock was valid. The obtrain method string parameter of lockRegistry is the corresponding business scenario, such as: order number, user ID, etc. If the string is the same, it can be considered as the same lock.
If you need to test a distributed environment, you just need to change the startup port in the configuration file to start multiple processes to simulate a distributed environment
lock10 start 1607417328823
lock10 get lock success 1607417328855
immediate start 1607417329943
immediate get lock success 1607417338872
Copy the codeWhile I was sleeping, I went to ZK and found that the framework would create two temporary nodes under the specified node to control concurrency, similar to what we demonstrated earlier, but the details are left up to you as a reader to dig.
/
|--zookeeper
|--HG-lock
|--lock
|--_c_41d75f28-2346-4cf2-89e8-accccce9ad1a-lock-0000000000
|--_c_98549447-0ee4-4c93-8194-4ed428225f75-lock-0000000001
Copy the codeFor more details on the example (which is not very detailed, it’s a very simple project), visit the project address above to see the source code.
Third, summary
This article uses the story and actual combat to explain the basic operation of ZK and part of the advanced operation. In the next article, I’ll get into the Principles, and I’ll show you how the ZK server handles each request.
Follow the HelloGitHub public account to receive the first updates.
There are more open source projects and treasure projects waiting to be discovered.