Mind mapping

preface

In many cases, ZooKeeper can be found in various framework applications, such as Kafka middleware, Dubbo framework, Hadoop, etc. Why do you see ZooKeeper everywhere?

What is ZooKeeper

ZooKeeper is a distributed service coordination framework, providing distributed data consistency solutions, based on ZooKeeper data structure, Watcher, election mechanism and other features, can achieve data publish/subscribe, soft load balancing, naming service, unified configuration management, distributed lock, cluster management and so on.

Why use ZooKeeper

ZooKeeper ensures that:

Update requests are made in sequence. Update requests from the same client are executed in the order in which they are sent
Data update atomicity. A data update either succeeds or fails
Globally unique data view. Client Data views are consistent regardless of which server the client is connected to
Real time. Within a certain time range, the data read by the client is the latest

Data structure

The data structure of ZooKeeper is similar to that of the Unix file system. ZooKeeper is generally regarded as a tree. Each node is called a ZNode, and each ZNode can store 1 MB of data by default. Each ZNode can be identified by a unique path. As shown below:

When creating a ZNode, you can specify the following four types:

PERSISTENT ZNode. The creation is not deleted even if the client is disconnected from the server. The creation disappears only when the client is deleted.
PERSISTENT_SEQUENTIAL, serial number ZNode. As with persistent nodes, they are not deleted after disconnection, and the number of ZNodes is automatically increased.
EPHEMERAL is the temporary ZNode. When the client disconnects from the server, the ZNode will be deleted.
EPEMERAL_SEQUENTIAL, temporary sequential numbered ZNode. As with temporary nodes, disconnections are removed and the ZNode number is automatically increased.

4. Monitoring notification mechanism

Watcher is a mechanism implemented based on the observer pattern. If we need to implement notification when a ZNode changes, we can use the Watcher listener.

The client registers zNodes that need to receive notifications with ZooKeeper by setting monitoring points (watcher). ZooKeeper sends messages to the client when zNodes change.

This notification mechanism is one-off. Once Watcher is triggered, ZooKeeper is removed from the corresponding store. If you need to constantly monitor ZNode changes, you can set up a new Watcher to register with ZooKeeper after receiving notifications.

There are many types of monitoring points, such as monitoring ZNode data changes, monitoring ZNode child node changes, and monitoring ZNode creation or deletion.

5. Election mechanism

ZooKeeper is a highly available application framework because ZooKeeper supports clustering. When the ZooKeeper server is in cluster state, the configuration file does not specify Master and Slave. Instead, the ZooKeeper server conducts internal elections when it is initialized. One ZooKeeper server serves as the Leader and multiple ZooKeeper servers serve as followers, and the half availability rule is followed.

In order to save cost, the number of servers in a cluster is usually set to an odd number.

If the connection with the Leader cannot be maintained for a long time at runtime, a new Leader will be elected again to ensure the availability of the service.

Six, the beginning of experience

Download ZooKeeper from the official website. I’m using version 3.3.6.

Decompress the zoo_sample. CFG file in the /conf directory and rename it zoo.cfg.

CFG dataDir in zoo. CFG and create the corresponding directory:

Zkserver. CMD: /bin/zkserver. CMD /bin/zkserver. CMD

If the startup is successful, you should see this dialog box:

For visual interface, I recommend using ZooInspector, which is easy to operate:

6.1 Connecting to ZooKeeper Using Java

First we introduce Maven dependencies:

<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.6</version>
</dependency>
Copy the code

Next we write a Main method to do this:

    // Connection address and port number
    private static final String SERVER_HOST = "127.0.0.1:2181";

    // Session timeout time
    private static final int SESSION_TIME_OUT = 2000;

    public static void main(String[] args) throws Exception {
        // Parameter 1: server address and port number
        // Parameter two: timeout period
        // Parameter three: listener
        ZooKeeper zooKeeper = new ZooKeeper(SERVER_HOST, SESSION_TIME_OUT, new Watcher() {
            @Override
            public void process(WatchedEvent watchedEvent) {
                // Get the status of the event
                Event.KeeperState state = watchedEvent.getState();
                // Check whether it is a connection event
                if (Event.KeeperState.SyncConnected == state) {
                    Event.EventType type = watchedEvent.getType();
                    if (Event.EventType.None == type) {
                        System.out.println("Zk client connected..."); }}}}); zooKeeper.create("/java"."Hello World".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        System.out.println("ZNode added successfully");
        zooKeeper.close();
    }
Copy the code

Create a persistent ZNode with path/Java and value “Hello World” :

7. Overview of API

7.1 create

public String create(final String path, byte data[], List<ACL> acl, CreateMode createMode)
Copy the code

Parameter Description:

The path ZNode path
Data Data stored in a ZNode
Acl Acl permission control
CreateMode ZNode type

ZooKeeper defines three common ACL permissions in the zoodefs. Ids class:

/** * This is a completely open ACL. * This is a completely open ACL. Any connected client can operate This property
public final ArrayList<ACL> OPEN_ACL_UNSAFE = new ArrayList<ACL>(Collections.singletonList(new ACL(Perms.ALL, ANYONE_ID_UNSAFE)));

/** * This ACL gives the creators authentication id's all permissions. */
public final ArrayList<ACL> CREATOR_ALL_ACL = new ArrayList<ACL>(Collections.singletonList(new ACL(Perms.ALL, AUTH_IDS)));

/** * This ACL gives the world the ability to read
public final ArrayList<ACL> READ_ACL_UNSAFE = new ArrayList<ACL>(Collections.singletonList(new ACL(Perms.READ, ANYONE_ID_UNSAFE)));
Copy the code

CreateMode is one of the four ZNode types mentioned earlier:

public enum CreateMode {
    /** * persistent ZNode */
    PERSISTENT (0.false.false),
    /** * persistence automatically adds sequence number ZNode */
    PERSISTENT_SEQUENTIAL (2.false.true),
    /** * temporary ZNode */
    EPHEMERAL (1.true.false),
    /** * temporarily add sequence number ZNode */
    EPHEMERAL_SEQUENTIAL (3.true.true);
}
Copy the code

7.2 the query

// Synchronously obtain node data
public byte[] getData(String path, boolean watch, Stat stat){
    ...
}

// Get node data asynchronously
public void getData(final String path, Watcher watcher, DataCallback cb, Object ctx){... }Copy the code

The stat argument in the getData() method is used to receive the returned node description:

public byte[] getData(final String path, Watcher watcher, Stat stat){
    / / to omit...
    GetDataResponse response = new GetDataResponse();
    // Sends a request to the ZooKeeper server and obtains a response
    ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
    if(stat ! =null) {
        // Assign the response Stat to the incoming StatDataTree.copyStat(response.getStat(), stat); }}Copy the code

GetData using synchronous getData() :

    // Data description, including version number, ACL permission, child node information, and so on
    Stat stat = new Stat();
    The result is byte[] data, and the getData() method copies the description to the stat object underneath
    byte[] bytes = zooKeeper.getData("/java".false, stat);
    // Print the result
    System.out.println("ZNode data:" + new String(bytes));//Hello World
    System.out.println("Got datSpanning version number :" + stat.getVersion());// The default data version is 0
Copy the code

The 7.3 update

public Stat setData(final String path, byte data[], int version){... }Copy the code

Note that the third parameter version uses the CAS mechanism to prevent multiple clients from updating node data at the same time, so the version number needs to be passed in the update. Each update will increase the version number by 1. If the server receives the version number and finds that it is inconsistent, it will throw an exception.

Therefore, you need to query the version number before updating, otherwise you do not know the current version number, you can not update:

    // Obtain node description
    Stat stat = new Stat();
    zooKeeper.getData("/java".false, stat);
    System.out.println("Update ZNode data...");
    // Update operation, pass path, update value, version number three parameters, return the new description
    Stat setData = zooKeeper.setData("/java"."fly!!!".getBytes(), stat.getVersion());
    System.out.println(The updated version number is: + setData.getVersion());// The updated version is: 1
Copy the code

After the update, the version number has been increased:

If the version parameter is “-1”, it tells the ZooKeeper server that the client needs to update the data based on the latest version. But -1 is not a legal version number, but an identifier.

7.4 delete

public void delete(final String path, int version){... }Copy the code

Path Path of the node to be deleted
Version version number

Here we also need to pass in the version number, which can be obtained by calling getData(), as simple as this:

Stat stat = new Stat();
zooKeeper.getData("/java".false, stat);
/ / delete the ZNode
zooKeeper.delete("/java", stat.getVersion());
Copy the code

7.5 watcher mechanism

As mentioned in the third point above, ZooKeeper can use notification monitoring mechanism. When ZNode changes, it will receive notification messages and process them. With the Watcher mechanic, ZooKeeper can do a lot of things. How to use it?

Generally speaking, ZooKeeper’s notification monitoring mechanism can be divided into three processes:

① The client registers the Watcher ② the server processes the Watcher ③ The client calls back the Watcher client.

There are four ways to register watcher: new ZooKeeper(), getData(), exists(), and getChildren(). Here’s how to register watcher using the exists() method:

First we need to implement the Watcher interface and create a new listener:

public class MyWatcher implements Watcher {
    @Override
    public void process(WatchedEvent event) {
        // Get the event type
        Event.EventType eventType = event.getType();
        // Notification status
        Event.KeeperState eventState = event.getState();
        // Node path
        String eventPath = event.getPath();
        System.out.println("Type of event monitored :" + eventType.name());
        System.out.println("Monitored notification status :" + eventState.name());
        System.out.println("ZNode listened path :"+ eventPath); }}Copy the code

Then call the exists() method to register a listener:

zooKeeper.exists("/java".new MyWatcher());
// The listener is triggered to update the ZNode data
zooKeeper.setData("/java"."fly".getBytes(), -1);
Copy the code

You can then see the printed message on the console:

EventType is the Event type. We can determine the Event type and perform related business processing according to the notification status of Event.KeeperState.

EventType, KeeperState, EventType, KeeperState

The event type is an enumeration
public enum EventType {
    None (-1),/ / no
    NodeCreated (1),// The data node Watcher is listening to is created
    NodeDeleted (2),// The data node monitored by Watcher is deleted
    NodeDataChanged (3),// The contents of the data node monitored by Watcher are changed
    NodeChildrenChanged (4);// The list of children of the data node monitored by Watcher has changed
}

// Notification status is also an enumeration
public enum KeeperState {
    Unknown (-1),// Attributes expire
    Disconnected (0),// The client is disconnected from the server
    NoSyncConnected (1),// Attributes expire
    SyncConnected (3),// The client is properly connected to the server
    AuthFailed (4),// Authentication failed
    ConnectedReadOnly (5),// Return this state to the client, which can only handle read requests
    SaslAuthenticated(6),// When the server uses SASL for verification
    Expired (-112);// The session session is invalid
}
Copy the code

7.5.1 Features of Watcher

Once and for all. Once watcher is triggered, the ZK is removed from the corresponding store.

    zooKeeper.exists("/java".new Watcher() {
        @Override
        public void process(WatchedEvent event) {
            System.out.println("I am a listener for the exists() method"); }});// The listener is triggered to update the ZNode data
    zooKeeper.setData("/java"."fly".getBytes(), -1);
    // Attempts to trigger the listener a second time
    zooKeeper.setData("/java"."spring".getBytes(), -1);
Copy the code

Serial execution. The process of the client Watcher callback is a serial synchronization process to ensure order.

    zooKeeper.exists("/java".new Watcher() {
        @Override
        public void process(WatchedEvent event) {
            System.out.println("I am a listener for the exists() method"); }}); Stat stat =new Stat();
    zooKeeper.getData("/java".new Watcher() {
        @Override
        public void process(WatchedEvent event) {
            System.out.println("I'm a listener for the getData() method.");
        }
    }, stat);
    // The listener is triggered to update the ZNode data
    zooKeeper.setData("/java"."fly".getBytes(), stat.getVersion());
Copy the code

If the result is printed, the listener of the exists() method is called first, and then the listener of the getData() method is called. Because the exists() method is registered first.

Lightweight. WatchedEvent is the smallest notification unit of ZK’s entire Watcher notification mechanism. WatchedEvent consists of three parts: notification status, event type, and node path. Watcher notifications simply tell clients what happened, not what happened.

Write in the last

I remember Li Yongle, the UP host of STATION B, said that only when you make more people’s lives better, can your own life become better.

This quote was one of the reasons I started writing tech Share this year. I hope you find this article useful

Famous aviator Ma teacher said: return to the city is charged, and like is free ~

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

ZooKeeper introduction, read this article is enough!

Mind mapping

preface

What is ZooKeeper

Why use ZooKeeper

Data structure

4. Monitoring notification mechanism

5. Election mechanism

Six, the beginning of experience

6.1 Connecting to ZooKeeper Using Java

7. Overview of API

7.1 create

7.2 the query

The 7.3 update

7.4 delete

7.5 watcher mechanism

7.5.1 Features of Watcher

Write in the last

ZooKeeper introduction, read this article is enough!

Mind mapping

preface

What is ZooKeeper

Why use ZooKeeper

Data structure

4. Monitoring notification mechanism

5. Election mechanism

Six, the beginning of experience

6.1 Connecting to ZooKeeper Using Java

7. Overview of API

7.1 create

7.2 the query

The 7.3 update

7.4 delete

7.5 watcher mechanism

7.5.1 Features of Watcher

Write in the last

Related Posts

The most complete Explain summary, SQL optimization is no longer difficult!

Government Service both | | new Service discovery, configuration management implementation

MySQL: How to design a database with high concurrency?