ZooKeeper is a distributed, open source distributed application coordination service. It provides consistency services for distributed applications, including data release/publication, load balancing, configuration maintenance, domain name service, distributed synchronization, and group service. Why did we choose ZooKeeper? Zookeeper has the following features:
| The characteristics of | instructions |
|---|---|
| Final consistency | Displaying the same view for the client is a very important feature in ZooKeeper |
| reliability | If a message is accepted by one server, it is accepted by all servers. |
| The real time | Zookeeper cannot guarantee that both clients will get the newly updated data at the same time. If you need the latest data, you should call the sync() interface before reading the data |
| independence | Clients do not interfere with each other |
| atomic | Updates can only succeed or fail, there is no intermediate state. |
| sequential | All servers publish the same message in the same order. |
1. Design objectives of ZooKeeper
Zookeeper is dedicated to providing a distributed coordination service with high performance, high availability, and strict sequential access control (mainly for write operations). Its design objectives are as follows:
- Simple data model, Zookeeper enables distributed programs to coordinate with each other through a shared tree structure namespace. That is, the data model in the memory of Zookeeper server is composed of a series of data nodes called ZNodes. Zookeeper stores the full amount of data in the memory. To improve server throughput and reduce latency.
- You can build a cluster. A Zookeeper cluster usually consists of a group of machines, each of which maintains the current server state in memory and communicates with each other.
- Sequential access. For each update request from the client, Zookeeper assigns a globally unique increment number that reflects the order in which all transactions are performed.
- High performance: Zookeeper stores all data in memory and directly serves all non-transaction requests from clients. Therefore, Zookeeper is especially suitable for read operations.
2. The zookeeper architecture diagram
Zookeeper:
| role | describe |
|---|---|
| leader | Responsible for initiating and deciding votes and updating system status |
| learner | There are followers and observers. Followers are used to receive client requests, return results to the client, and vote in the master election process. The Observer can accept client connections and forward write requests to the Leader. However, the Observer does not participate in the voting process and only synchronizes the status of the Leader. The Purpose of the Observer is to expand the system and improve the read speed |
| client | Request Initiator |
Each Server has three states during the working process:
- LOOKING: The current Server does not know who the leader is and is searching for it
- LEADING: The current Server is the elected leader
- FOLLOWING: The leader has been elected and the current Server synchronizes with it
How do you elect a Server leader? Half pass, odd number election – 3 machines hang 1 2>3/2-4 machines hang 2 2! >4/2 Specific election process: » Each Server starts asking other servers for whom it wants to vote. » The server responds with the leader ID and the zxID of the last transaction (each server will recommend itself when the system is started). And set the Server information to the Server to vote for next time. » Sever with the most votes is counted as the winner. If the winner has more than half of the votes, server is elected as the leader. Otherwise, continue the process until the leader is elected » The leader starts to wait for the server to connect » followers connect to the leader, Send the maximum zxID to the leader » The leader determines the synchronization point based on the zxID of the followers » Notify the followers that they have become uptoDate after the synchronization completes » The client can accept the request for service again
3. Working principles of ZooKeeper
The core of Zookeeper is atomic broadcast. This mechanism ensures the synchronization between servers. The Zab protocol implements this mechanism. Zab protocol has two modes: recovery mode (master selection) and broadcast mode (synchronization). Zab goes into recovery mode when the service starts or after the leader crashes, and the recovery mode ends when the leader is elected and most of the servers are in sync with the leader state. State synchronization ensures that the leader and Server have the same system state. Once the leader has synchronized the status of most followers, he can start to broadcast messages. When a server is added to the ZooKeeper service, it starts in recovery mode, discovers the Leader, and synchronizes the status with the Leader. When the synchronization ends, it also participates in the message broadcast. The Zookeeper service remains Broadcast until the Leader crashes or the leader loses most of the followers. The broadcast mode needs to ensure that proposals are processed sequentially, so ZK uses an incremented transaction ID number (ZXID) to ensure that. All proposals had zxIDS added to them when they were put forward. In the implementation, the ZXID is a 64 digit number, whose 32 bits higher is the epoch, which is used to identify whether the leader relationship changes. Each time a leader is elected, it will have a new epoch. The lower 32 bits are an increasing count. When the leader crashes or loses most of the followers, zK enters recovery mode. In recovery mode, a new leader needs to be elected to restore all the servers to a correct state.
4.Zookeeper data model
Each node is called znode in ZooKeeper and has a unique path identifier » The node znode can contain data and child nodes, But EPHEMERAL type of node cannot have children » Znode can have multiple versions of data, such as under a certain path has multiple data version, so, we should take version of data query this path » client applications can be set on a node monitor » node does not support part, speaking, reading and writing, but one-time complete, speaking, reading and writing
5. The Zookeeper node
Znodes come in two types, ephemeral and persistent.
The type of Znode is determined at creation time and cannot be changed later
Zookeeper deletes the temporary ZNode when the Zookeeper guarantee is terminated. The temporary ZNode cannot have child nodes
Persistent ZNodes do not depend on client sessions and are deleted only when the client explicitly wants to delete the persistent ZNode
Znode has four types of directory nodes
- PERSISTENT,
- EPHEMERAL
- PERSISTENT_SEQUENTIAL,
- EPHEMERAL_SEQUENTIAL
6. They are guaranteed
* Atomicity of data updates: A single update either succeeds or fails » Globally unique data views: Data views are consistent regardless of which server the client is connected to » Real-time, within certain event ranges, The client can read the latest data
7.ACL
Zookeeper uses Access Control Lists (ACLs) to Control permissions. The ACL defines the following permissions:
· CREATE: Permission to CREATE child nodes.
· READ: Permission to obtain node data and child node list.
· WRITE: permission to update node data.
· DELETE: permission to DELETE child nodes.
· ADMIN: Sets the ACL permission of a node.
Is everyone still ok? If you like, move your hands to show 💗, point a concern!! Thanks for your support!
Welcome to pay attention to the public number [Ccww technology blog], original technical articles launched at the first time
