Continue to drive, in fact, ZooKeeper can be used in many scenarios, here in the introduction of several scenarios, in talking about the distributed lock, many people want to know about the distributed lock, in fact, distributed lock is not a feature of ZK, zK can do too many things. Source: github.com/limingios/n…

Scenario analysis

  • Previous Job scenarios

In many projects, there are jobs, running timed tasks, counting orders every day, sending messages regularly, lots of jobs. Original job is deployed on the server, because the code is all is consistent, lead to the job, according to how much server performs many times, in order to prevent this, will set the switch, alone in the machine that will be in setting up a text, the program to read their text, logo is open only one machine, the other is closed. So there might be five machines deployed online, but only one machine is running.

I’m sure some people ask me, why only start one machine, how nice it is to start all of them, if you want to perform a message task, and each machine receives it, if all of them start it then the user will receive the number of messages that respond to it based on the number of services. There must be something wrong with that. Don’t reinvent the wheel. Do it alone.

If the timed job service fails, the entire task system fails. There are problems.

With ZK taking over, choose one of these three

If the eldest brother dies, the following two younger brothers choose the eldest brother according to the rules

  • Distributed configuration Center

Publish and subscribe model, also known as the so-called configuration center, as the name implies, publishers publish data to ZK nodes for subscribers to obtain data, centralized management and dynamic update of configuration information.

Summary: Inductive change in pull mode (C>S)/push mode (S>C)

Apply to:

Global configuration Information Service address list of the service framework

Note:

Scenarios where the amount of data is small, but the data may be updated relatively quickly

On behalf of:

Baidu disconf Github: github.com/knightliao/…

(2) distributed lock

  • origin

If you want to control a piece of code, such as i++, on three machines with a JVM on each machine, the JVM is relatively independent, so using the JVM lock is completely impossible. This is where distributed locks are created. You can also do this with a database. Most business scenarios do not require distributed locking due to the exclusive locking feature of databases. But there are also businesses that need distributed locks. But for large Internet architecture, database is the bottleneck, try to read less write, if the use of the database this exclusive lock feature may give the database more pressure. Database may produce deadlock, database competition is too fierce, the lock is not released in time, the database will also have problems, direct lock table. This is where distributed locks must be used.

  • introduce

Distributed lock, this is mainly easy to zooKeeper for us to ensure data consistency gun. Locking services can be divided into two categories, one for holding exclusivity and the other for controlling timing. 1. Exclusiveness means that only one client who tries to acquire the lock succeeds in obtaining the lock. The usual practice is to treat a Znode on zK as a lock, and implement this by creating zNode. All clients create the /distribute_lock node, and the client that is successfully created owns lock 2. Control timing, that is, all clients trying to acquire the lock will eventually be scheduled to execute, but with global timing. This is similar, except that the /distribute_lock is already present, and the client creates a temporary ordered node below it (this can be specified by the node’s attribute control: createmode.ephemeral_sequential). The parent node of ZK (/distribute_lock) maintains a sequence to ensure that the child nodes are created sequentially, thus forming a global sequence for each client.

  • Exclusive lock

Exclusive lock (Exclusive Lovks, X lock), also known as the write lock If the transaction T1 to data objects 01 and the exclusive lock, so during the whole lock, only allow transaction T1 to read and update operations of O1, any other affairs can’t on the data object for any type of operation (lock) can no longer the object, Until T1 releases the exclusive lock. He who creates gets the lock

But this one up here, if 10,000 threads are waiting for a lock, then when the lock is released, 10,000 threads are competing at the same time, and there’s a herding effect. Herding is when every node fails and all nodes listen and react, which puts a lot of pressure on the server. The principle of ZooKeeper distributed lock is actually very simple. Firstly, ZooKeeper creates a PERSISTENT node, and then each thread that wants to obtain the lock will create a temporary sequential node under this node, and then specify the smallest node to obtain the lock. Therefore, each thread will first determine whether it is the node with the smallest serial number, and if it is, it will acquire the lock. If it is not, Watcher will listen to the previous node that is smaller than it. If the previous node does not exist, watcher will again determine whether it is the node with the smallest serial number, and if it is, it will obtain the lock. When a node fails, only those listening on it respond.

  • A Shared lock

Shared Locks are also known as read Locks.

If transaction T1 adds a shared lock to O1, then T1 can only read O1. Other transactions can also add an exclusive lock to O1 until all shared locks are released.

The zookeeper client

Very few people directly use native, just like we parse JSON, in fact, with strings according to the rules can also parse, later with a new JAR, Ali Fastjson, who is not using strings to parse it. Standing on the shoulders of giants can see higher and farther.

  • ZkClient

ZkClient is an open source client developed by Datameer engineers. It wraps the native API of Zookeeper and implements functions such as timeout reconnection and Watcher repeated registration.

  • Curator

Curator is an open source Zookeeper client of Netflix. Compared with the native client provided by Zookeeper, Curator has a higher level of abstraction, which simplifies the development of Zookeeper client.

zkClient

When using the Java client of ZooKeeper, several problems need to be solved: repeated registration of watcher, session failure and reconnection, and exception handling. It has been used in many projects, including Dubbo, Kafka, and Helix.

Maven rely on

<dependency>
 <groupId>com.101tec</groupId>
 <artifactId>zkclient</artifactId>
 <version>0.10</version>
</dependency>
Copy the code

or

 <dependency>
 <groupId>com.github.sgroschupf</groupId>
 <artifactId>zkclient</artifactId>
 <version>0.1</version>
 </dependency>

Copy the code

Github:github.com/sgroschupf/…

  • ZKClient design

  • This section describes components of ZkClient

From the above structure, IZKConnection is an adapter between ZkClient and ZooKeeper. In the code directly using ZKClient, its essence or entrusted zooKeeper to deal with. There are several ways to register watcher using the ZooKeeper client:

GetData (); exists(); getChildren (); getChildren (); getChildren (); Getdata, EXISTS registers the watcher of a node, and getChildren registers the Watcher of a child node. In ZKClient, events are divided into node events (data events) and child node events according to event types. The corresponding event handlers are IZKDataListener and IZKChildListener. Session-related events and event handlers have also been added. ZkEventThread is a thread dedicated to handling events.

  • Start the ZKClient

When the ZKClient object is created, the connection to the ZooKeeper server is established. The specific process is as follows:

1. At startup, specify connection string, connection timeout, serialization tool, etc. Create and start eventThread to receive events and schedule the execution of event listeners. 3. Connect to the ZooKeeper server and use ZKClient itself as the default Watcher.

  • Register Watcher for the node

The three ZooKeeper methods: getData, getChildren, exists, and ZKClient provide corresponding proxy methods. Take exists:

The watcher process is determined by hasListeners’ path.

HasListeners are listeners bound to the data node.

So, by default, watcher is automatically registered for the specified path, and it is the default watcher (ZKClient). How do hasListeners determine true for path listeners?

  • ZKClient provides subscription functions:

A new session simply calls subscribteXxx to subscribe to the corresponding event after getting the responding data node.

  • This operation changed ZooKeeper

Zookeeper provides the following change operations: Creating and deleting nodes and modifying node data. Create operation, data nodes are divided into four types, and ZKClient provides corresponding agents for each of them:

Deleting a node:

Modifying node data:

WriteDataReturnStat () : Writes data and returns the status of the data. UpdateDataSerialized () : Modifies serialized data. Data is first read, then modified using DataUpdater, and finally sent to the server by calling writeData.

  • The client handles the changes

As you already know, ZKClient is the default Watcher, and the Watcher registered for each data node is the default Watcher. So how do you notify the Listener of various events?

The processing process can be summarized as follows:

1. Determine the change type: the change type includes State change, ChildNode change (create ChildNode, delete ChildNode, modify ChildNode data), and NodeData change (create specified node, delete node, change node data).

2. Collect the Listeners associated with the Path, create a ZKEvent for each Listener, and send the ZkEventThread to process the ZKEvent.

3, ZkEventThread, after getting ZkEvent, only need to call ZkEvent run method for processing.

How to call the Listener depends on the run() implementation of ZkEvent.

  • Serialization processing

ZooKeeper involves two types of serialization and deserialization operations: getData and setData. In ZKClient, readData and writeData are used instead.

For readData, getData of ZooKeeper is called and ZKSerializer is used for deserialization.

For writeData: Use ZKSerializer to serialize objects, and then call setData of ZooKeeper.

  • Register to monitor

In ZkClient, the client can subscribe to the Zookeeper server time by registering the related event listener. ZkClient provides the following listening event interfaces:

Interface class Register listening methods Unlistening method
IZkChildListener The subscribeChildChanges method of ZkClient The unsubscribeChildChanges method of ZkClient
IZkDataListener SubscribeDataChanges method of ZkClient The subscribeChildChanges method of ZkClient
IZkStateListener SubscribeStateChanges method of ZkClient The unsubscribeStateChanges method of ZkClient

ZkClient also provides an unsubscribeAll method to unsubscribeAll listening.

  • How does ZkClient solve the problems encountered when using the ZooKeeper client?

Watcher automatic re-registrations: These are dependent on hasListeners to determine whether to re-register. If you are not clear about this, you can refer to the following procedure for handling Session failure and reconnection: If the Session expires, close the existing connection and re-establish the connection. Exception handling: Comparing ZooKeeper and ZKClient, you can find that all ZooKeeper operations are thrown exceptions, but all ZKClient operations are not thrown exceptions. When an exception occurs, it either logs, returns null, or makes a corresponding Listener call.

Compared with the official ZooKeeper client, ZKClient only needs to pay attention to the actual Listener implementation. So this client is still recommended to use.

PS: zkClient should be a tool class for the operation of ZK, convenient to use ZK, understand the API, in fact, not difficult. But the principle of ZK must be understood. Temporary, permanent, exclusive, shared locks.