1. Zk theory knowledge

    1. Zk is introduced

    2. Three characters in ZK

    3. Leader election in the cluster

    4. The CAP theorem

    5. Data synchronization

  2. The ZK cluster is set up

    1. Vm Settings

    2. Download zK configuration environment variables

    3. Configuration myid

    4. Modifying a Configuration File

    5. Start the cluster

    6. Shut down the cluster

  3. Zk basic commands

    1. Accessing the Client

    2. Look at the node

    3. Create a node

      1. Permanent node

      2. Temporary node

      3. The difference between temporary and permanent nodes

      4. Order node

      5. Temporary sequence node

    4. Viewing Node Content

    5. Object Description

    6. Remove nodes

    7. Modifying node Content

Zk theory knowledge

Zk is introduced

ZooKeeper was developed by Yahoo Research and later donated to Apache. ZooKeeper is an open source distributed application coordination server that provides consistency services for distributed systems. Its consistency is achieved through ZAB protocol based on Paxos algorithm. Its main functions include configuration and maintenance, domain name service, distributed synchronization, and cluster management. Zookeeper’s official website is zookeeper.apache.org

Three characters in ZK

To avoid the single point of Zookeeper, ZK also comes in the form of a cluster. Roles in a ZK cluster fall into the following three categories:

Leader: The sole handler of transaction requests (write requests) and can also handle read requests

Follower: processes read requests from the client and responds to them. However, it does not process transaction requests. It only forwards client transaction requests to the Leader for processing. Have the right to vote on the transaction proposal initiated by the Leader; Synchronize the results of transactions in Leader; Leader A participant in an election has the right to vote and stand for election. (Like a regular job)

Observer: can be understood as followers who do not participate in the Leader election do not have the right to vote and stand for election in the Leader election process. Meanwhile, there is no voting right for Leader’s proposal. Used to help followers process more client read requests. The addition of Observer will improve the throughput of cluster read request processing, but will not increase the pressure of transaction request passing or Leader election. (Like a temp)

Leader election in the cluster

Each ZK host will send its myID and ZXID to other ZK hosts, where myID is the number of the host and ZXID is the epoch(the year number can be understood as the unique identifier that each Leader has. This ID is not available during the cluster restart phase)+ xID (transaction ID is the increasing ID in the cluster for each write request). After receiving messages from other ZK hosts, each ZK host compares the ZXids of all hosts, including itself, and selects the leader with the largest ZXID. If the ZXids are the same, select the leader with the largest myID.

The CAP theorem

CAP theorem refers to the principle that Consistency, Availability and Partition tolerance are incompatible in a distributed system.

Consistency (C) : Indicates whether data consistency can be maintained among multiple hosts in a distributed system. That is, after system data is updated, data on all hosts remains in the same state.

Availability (A) : The services provided by the system must always be available, that is, for each user request, the system can always respond to the user within A limited time.

Fault tolerance of partitions (P) : The distributed system can still provide services that meet the requirements of consistency and availability when encountering any network partition failure.

Zookeeper meets CP consistency and partition fault tolerance, while Eureka meets AP availability and partition fault tolerance

Data synchronization

When there is a new write request in the cluster, the Leader encapsulates the request as a Proposal and sends it to the followers. After receiving the message, the followers return an ACK. If the number of ACKS received by the Leader exceeds half, the Leader notifies all the followers to execute the commit update transaction. The proposal is sent to all observers, and the Observer updates the transaction. After the synchronization of the Follower and observer, the Follower and observer send an ACK to the Leader. After receiving the ACK, the Leader will add the Follower and observer to the usable queues respectively. Only the Follower and observer in the queues can provide services externally. In other words, when ZK synchronizes data, it does not provide services externally, and the host that fails to synchronize data does not provide services externally. This is how ZK maintains consistency. This also loses usability, as the user may not get a response from the ZK cluster within a very short period of time. \

The ZK cluster is set up

1 THE VMware VM is used here. After creating the VM, disable the firewall first; otherwise, ZKS cannot communicate with each other.

2 download the package from zookeeper.apache.org and decompress it. Please note that ZK requires JDK support so make sure you have the JDK on your virtual machine. Then configure the environment variables. After the configuration, run the source /etc/profile command to refresh the environment variables.

 

 

3 Perform the same operations for the four VMS in the preceding two steps. The myID must be configured for each of the four VMS. Note that the directory, file name, and content are the same.

4 Copy the zoo_sample. CFG configuration file or rename it zoo.cfg

Modify the configuration file. The first three configuration files are identical. The configuration inside is noted.

TickTime =2000 milliseconds of each tick. TickTime =2000 milliseconds of each tick Synchronization phase can take up to 10 units initLimit=10 # The number of ticks that can pass between # sending a SyncLimit =5 # The directory where the snapshot is stored. # do not use /tmp for storage, DataDir =/home/zk/zookeeper-3.4.14/data # the port at which the clients will The first port is the cluster communication port, 1=192.168.31.51:2888:3888 server.2=192.168.31.39:2888:3888 server.3=192.168.31.148:2888:3888 Server. 4 = 192.168.31.27:2888-3888: the observer # the maximum number of client connections. # happens this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1Copy the code

View Code

The configuration file for the fourth Observer is as follows, which is really just one more line of configuration.

TickTime =2000 milliseconds of each tick. TickTime =2000 milliseconds of each tick Synchronization phase can take up to 10 units initLimit=10 # The number of ticks that can pass between # sending a SyncLimit =5 # The directory where the snapshot is stored. # do not use /tmp for storage, DataDir =/home/zk/zookeeper-3.4.14/data # the port at which the clients will The first port is the cluster communication port, 1=192.168.31.51:2888:3888 server.2=192.168.31.39:2888:3888 server.3=192.168.31.148:2888:3888 Server. 4 = 192.168.31.27:2888-3888: the observer # this node for the observer peerType = # observer the maximum number of client connections. #  increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1Copy the code

View Code

5 Starting a Cluster

 

 

 

 

6 Shutting down the Cluster

Of the above four hosts, host 2 is the leader. We shut down host 2 to see if the cluster will choose the leader by itself. Host 2 has four observers so it does not participate in the election

 

 

Zk basic commands

Accessing the Client

Zkcli. sh Goes to the local client. Note that the data is still in the connected cluster even though the local ZK is connected.

 

Look at the node

ls /path/path

Zk’s nodes are similar to our file directories that iterate down one level at a time. The current ZooKeeper node comes with zK.

Create a node

Permanent node

create /path value 

Create a permanent node and set the node information

Temporary node

create -e /path value

Create a temporary node and set the node information

The difference between temporary and permanent nodes

The temporary node is bound to the client. Once the client is shut down, the node is automatically deleted and the temporary node cannot create child nodes

 

Order node

create -s /path value

The nodes created are automatically numbered, which means that we can use ZK to get unique numbers that are also unique in a distributed environment.

Temporary sequence node

create -e -s /path value

 

Viewing Node Content

get /path/path/…

Gets the content of the node and a detailed description of the node

Object Description

Remove nodes

delete /path/path…

Notice If a node contains child nodes, you cannot delete them. You must delete all child nodes before deleting them.

 

Modifying node Content

set /path/path… value

Lock a node and modify its contents