Zookeeper is a distributed coordination service system, which provides a solution to the problem of consistency in distributed environment. This section describes basic knowledge about Zookeeper.

The data model

A data model is a logical structure for storing and processing data. The data model in Zookeeper is a tree structure, similar to a computer’s file system. We can create child nodes under the root node and continue to create next-level nodes under the child node. Each layer of the Zookeeper tree is separated by a slash (/), and only the Zookeeper node can be queried using an absolute path.

The node type

Data nodes in Zookeeper are also classified into persistent nodes, temporary nodes, and ordered nodes.

1. Persistent nodes

Persistent node: Once a node is created as a persistent node, the data node is always stored on the Zookeeper server. The node is not deleted even if the session between the client and server that created it is closed.

2. Temporary nodes

Temporary nodes are not permanently stored on the Zookeeper server. When the client session that created the temporary node is closed, the node is deleted from the Zookeeper server.

We usually use temporary nodes to do statistics on the performance of machines in a server cluster. For example, set the cluster to the “/ Servers “node and create a temporary node for each server in the cluster. The node is deleted when the server goes offline. Finally, we can know the running condition of the cluster by counting the number of temporary nodes.

3. Ordered nodes

The ordered node is based on the permanent node and temporary node, adding the node ordered feature. Node order means that Zookeeper automatically uses a monotonically increasing number as the suffix of the node.

Each Zookeeper node maintains a storage node data, ACL access control information, a binary array of node data, and a stat field that records its own status information.

Node state structure

Run stat/node name on the client to see the node status information output by the console.

The ZooKeeper data node introduces the concept of versioning. Each data node has three versions: the number of changes to node data, child node information, and ACL information.

Use Zookeeper to implement locks

Situation: How to solve the problem of oversold? There are two types of lock: pessimistic lock and optimistic lock.

Pessimistic locking

Pessimistic locking assumes that processes compete for resources in a critical region all the time, so the data will remain locked. For pessimistic locks, we usually use temporary nodes to avoid the lock persisting due to abnormal process interruption, and add listening events on the server side to get other processes to acquire the lock again.

Optimistic locking

Optimistic lock: When data is submitted for update, it checks whether data conflicts. If conflicts are found, the operation is rejected. Optimistic lock can be divided into three steps: read, check and write. CAS, for example, is an implementation of optimistic locking. In Zookeeper, the Version attribute is used to verify optimistic locks. In the underlying implementation of ZooKeeper, when the server processes the setDataRequest request, the checkAndIncVersion method will be called first to verify the data version. Zookeeper obtains the version of the currentversion and nodeRecord using getRecordForPath to obtain the version information currentversion of the current server. If version is -1, optimistic locking is not applicable. If the value is not -1, compare version with currentVersion.

For optimistic and pessimistic locks, see: juejin.cn/post/694028…

They can only use absolute paths, not use relative paths, is due to its underlying hashtableConcurrentHashMap was used to realize.

Watch mechanism

We can use the Watch monitoring mechanism to implement a publish and subscribe function. To realize the publish and subscribe function, there are several core nodes: the client registers the service, the server processes the request, and the client performs the corresponding operation after receiving the callback. Using the Watch mechanism, you can pass either the Watch argument or the getData () method to the constructor on the client side

The underlying principle of Watch mechanism

The structure is similar to the “observer mode” in that an object or data node may be monitored by multiple clients and notified when a corresponding event is triggered.

ZooKeeper implements two watch lists on the client side and the server side, namely, ZKWatchManager and WatchManager.

How to implement Watch mechanism

Client Watch registration implementation process

The Watcher mechanism on the client side is one-time and is deleted when triggered. When the ZooKeeper client sends a session request for a Watch monitoring event, it does two things:

  • Mark the session as a request with a Watch event
  • Store the Watch event to ZKWatchManager

Service end Watch registration implementation process

The Zookeeper server processes the Watch event as follows:

  • Resolves whether the received request has a Watch registration event
  • Store the corresponding Watch event to the WatchManager

Process of triggering the Watch event on the server

After the client and server register watch, the Watch event can be triggered. After the change of the content of the node data, will be called WatchManager. TriggerWatch method trigger data changes. TriggerWatch does the following:

  1. Encapsulates WatchedEvent objects with session state, event type, and data node
  2. Query the Watch event registered with the controller. If there is a Watch event, add it to the defined Watchers set and delete it in WatchManager
  3. Call the process method to send notifications to the client.

Procedure for handling client callback

Use the sendThread.readResponse () method to unify the responses on the server.

  1. The deserialization server sends the request header and determines that the value of the phase attribute field XID is -1, indicating that the request response is of notification type.
  2. Deserialize the received byte stream into a WatcherEvent object
  3. Check whether the client is configured with the chrootPath attribute. If the value is true, process the chrootPath of the received node path.
  4. The eventThread.queueEvent () method hands the received event to the EventThread thread for processing

The execution logic inside the eventThread.queueEvent () method:

  1. Query the registered client Watch information from ZKWatchManager and remove it from ZKWatchManager.
  2. The queried Watcher is stored in the waitingEvents queue, and the run method in the EventThread class is called. The loop pulls out the Watcher event waiting in the waitingEvents queue for processing.
  3. The processEvent(event) method is called to execute the Process () method that implements the Watcher interface.

Watch is disposable, so we need to add the Watch event again when we get the notification from the server.