This is the 15th day of my participation in the August More Text Challenge

1. Cluster Installation

Here, three machines are prepared in advance, named HadoOP101, Hadoop102 and Hadoop103.

We need to install ZooKeeper on all three machines.

1. Upload the installation package to Linux, decompress it, and rename it zooKeeper

Tar -zxvf zookeeper-3.5.7.tar.gz -c /opt/module/Copy the code

2. Set the server number.

For the numbers of the three servers, you need to create a myID file and point to the folder of this file in the zoo. CFG directory.

Create a zkData folder in the ZooKeeper directory and add the myID file with the number 1

 mkdir zkData
 touch myid
Copy the code

Add the number corresponding to server to the file:

 1
Copy the code

Two and three for the other two.

CFG file. Rename conf/zoo_sample. CFG to zoo.cfg

 vim conf/zoo.cfg
Copy the code

Modify the dataDir

 dataDir=/opt/sofeware/zookeeper/zkData
Copy the code

Add cluster configuration at the end of the file:

 #######################cluster##########################
 server.1=hadoop101:2888:3888
 server.2=hadoop102:2888:3888
 server.3=hadoop103:2888:3888
Copy the code

The configuration parameters here need to be explained:

Server. A = B: C: D.Copy the code

A is A number that indicates the server number;

In cluster mode, A file myID is configured, which is in the dataDir directory. There is A data in this file that is the value of A. When Zookeeper starts up, it reads this file and compares the data with the configuration information in zoo.cfg to determine which server it is.

B is the address of the server;

C is the port through which the followers of the server exchange information with the Leader server in the cluster.

In case the Leader server in the cluster fails, a port is needed to re-elect a new Leader, and this port is used to communicate with each other during the election.

4. Cluster operations

You can directly start bin/ zkserver. sh start. But it needs to be on every machine

Cluster below to write a group script for convenience

#! /bin/bash for i in hadoop101 hadoop102 hadoop103 do echo "===== $i zookeeper======" case $1 in "start") echo "==================== START $i ZOOKEEPER ==================== " ssh $i /opt/sofeware/zookeeper/bin/zkServer.sh start ;; "status") echo "==================== STATUS $i ZOOKEEPER ==================== " ssh $i /opt/sofeware/zookeeper/bin/zkServer.sh status ;; "stop") echo "==================== STOP $i ZOOKEEPER ==================== " ssh $i /opt/sofeware/zookeeper/bin/zkServer.sh stop ;; *) echo "Input error" exit ;; esac doneCopy the code

Group directly:

 [root@hadoop101 bin]# zkServer.sh start
 ​
Copy the code

View status:

[root@hadoop101 bin]# zkServer.sh status ===== hadoop101 zookeeper====== ==================== STATUS hadoop101 ZOOKEEPER  ==================== /usr/bin/java ZooKeeper JMX enabled by default Using config: /opt/sofeware/zookeeper/bin/.. /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: follower ===== hadoop102 zookeeper====== ==================== STATUS hadoop102 ZOOKEEPER ==================== /usr/bin/java ZooKeeper JMX enabled by default Using config: /opt/sofeware/zookeeper/bin/.. /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: leader ===== hadoop103 zookeeper====== ==================== STATUS hadoop103 ZOOKEEPER ==================== /usr/bin/java ZooKeeper JMX enabled by default Using config: /opt/sofeware/zookeeper/bin/.. /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: followerCopy the code

You can see which is the host and which is the slave information.

2. Basic client commands

Basic command syntax Functional description
help Displays all operation commands
ls path Use the ls command to view the children of the current ZNode. -w To listen for child changes. -s to add secondary information
create Normal create -s contains sequence -e temporary (restart or timeout disappears)
get path Get the value of the node. -w listens for changes in the node content. -s adds secondary information
set Set the value of a node
stat Viewing Node Status
delete Remove nodes
deleteall Delete nodes recursively
2.1 Starting the Client
 bin/zkCli.sh
Copy the code
2.2 Displaying all Operation commands
 [zk: localhost:2181(CONNECTED) 1] help
Copy the code
2.3 Viewing the Content of the Current node
 [zk: localhost:2181(CONNECTED) 0] ls /
 [zookeeper]
Copy the code
2.4 Viewing detailed data about the current node
[zk: localhost:2181(CONNECTED) 2] ls2 / 'ls2' has been deprecated. Please use 'ls [-s] path' instead. [zookeeper] cZxid = 0x0  ctime = Thu Jan 01 08:00:00 CST 1970 mZxid = 0x0 mtime = Thu Jan 01 08:00:00 CST 1970 pZxid = 0x0 cversion = -1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 1Copy the code
2.5 Creating a Common Node
 [zk: localhost:2181(CONNECTED) 3] create /jiangxi "nanchang"
 Created /jiangxi
 ​
 [zk: localhost:2181(CONNECTED) 4] create /jiangxi/nc "hl"
 Created /jiangxi/nc
Copy the code
2.6 Obtaining the node value
 [zk: localhost:2181(CONNECTED) 5] get /jiangxi
 nanchang
 [zk: localhost:2181(CONNECTED) 6] get /jiangxi/nc
 hl
Copy the code
2.7 Creating temporary Nodes
 create -e /jiangxi/yichun "ll"
Copy the code

The temporary node is not visible after the client restarts.

2.8 Creating nodes with Serial Numbers
 [zk: localhost:2181(CONNECTED) 7] create -s /jiangxi "jinxian"
 Created /jiangxi0000000001
 [zk: localhost:2181(CONNECTED) 8] create -s /jiangxi "yushan"
 Created /jiangxi0000000002
Copy the code
2.9 Changing the Node Value
 [zk: localhost:2181(CONNECTED) 9] set /jiangxi "xiazhen"
 [zk: localhost:2181(CONNECTED) 10] get /jiangxi
 xiazhen
Copy the code
2.10 Value change monitoring of nodes

Hadoop102 Machine start monitoring:

 [zk: localhost:2181(CONNECTED) 8] get -w /jiangxi 
Copy the code

After the hadoop101 machine changes the value, Hadoop102 can observe the value change

 [zk: localhost:2181(CONNECTED) 1] 
 WATCHER::
 ​
 WatchedEvent state:SyncConnected type:NodeDataChanged path:/jiangxi
Copy the code

If the child node of a node changes, the information is as follows:

 WATCHER::
 WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/jiangxi
Copy the code
2.11 Deleting a Node
 delete /jiangxi/yushan
Copy the code
2.12 Checking node Status
 [zk: localhost:2181(CONNECTED) 1] stat /jiangxi
 cZxid = 0x100000002
 ctime = Fri Aug 13 14:34:44 CST 2021
 mZxid = 0x100000008
 mtime = Fri Aug 13 14:39:31 CST 2021
 pZxid = 0x100000003
 cversion = 1
 dataVersion = 2
 aclVersion = 0
 ephemeralOwner = 0x0
 dataLength = 7
 numChildren = 1
Copy the code

Third, API application

1. Add the JAR package

<dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>RELEASE</version> </dependency> <dependency> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> The < version > 3.5.7 < / version > < / dependency > < / dependencies >Copy the code

2. APi testing

public class ZookeeperTest { public static String connectString = "hadoop101:2181,hadoop102:2181,hadoop103:2181"; private static int sessionTimeout = 2000; private ZooKeeper zkClient = null; /** * Create a ZooKeeper client * @throws IOException */ @before public void init() throws IOException {zkClient = new ZooKeeper(connectString, sessionTimeout, New Watcher() {@override public void process(WatchedEvent WatchedEvent) { System.out.println(watchedEvent.getType()+"--"+watchedEvent.getPath()); Zkclient.getchildren ("/",true); } catch (Exception e) { e.printStackTrace(); }}}); } /** * Create child nodes * @throws InterruptedException * @throws KeeperException */ @test public void create() throws InterruptedException, KeeperException {// Parameter 1: Path to the node to be created; // Parameter 2: node data; // Parameter 3: node permission; // Parameter 4: The node's type String node = zkClient. Create ("/xiaolei ", "leilei". GetBytes (), ZooDefs. Ids. OPEN_ACL_UNSAFE, CreateMode. PERSISTENT); @test public void getChildren() throws InterruptedException, KeeperException { List<String> children = zkClient.getChildren("/", true); for (String child : children) { System.out.println(child); Thread.sleep(long.max_value); } /** * Check whether ZooKeeper exists * @throws InterruptedException * @throws KeeperException */ @test public void exist() throws InterruptedException, KeeperException { Stat exists = zkClient.exists("/xiaolei", false); if(exists==null){ System.out.println("not exist"); }else{ System.out.println("exist"); }}}Copy the code

4. Summary of ZooKeeper

4.1 What is ZooKeeper

Overview + functions + internal structure + application scenarios + advantages and disadvantages.

Zookeeper is a common middleware in the big data or distributed domain. It is mainly used as a registry for service registration and service discovery. Its internal structure is very similar to Linux file system, as a whole as a tree, each node is called zNode, can store 1MB of data by default, this node is also divided into temporary nodes and persistent two types, to sum up, it is a distributed coordination service framework.

4.2 What Functions does ZooKeeper provide

The services it provides include:

  • Unified naming management: Client applications can obtain information such as the address and provider of a resource or service by a specified name.
  • Unified configuration management: Extract common configuration files and distribute them to other systems.
  • Unified cluster management: Monitors node survival status and running requests.
  • Soft load balancing: Let the least visited servers handle the latest client requests.
  • Distributed lock: temporary sequential node implementation. Exclusive lock, shared lock. An exclusive lock means that only one thread can use a resource at a time. A shared lock means that read locks are shared. Read and write locks are mutually exclusive, that is, multiple threads can read the same resource at the same time. Zookeeper controls distributed locks.

4.3 Do you understand THE ZAB agreement

At the core of Zookeeper is the atomic broadcast mechanism, which ensures synchronization between servers. The protocol that implements this is called the Zab protocol. Zab protocol has two modes: recovery mode and broadcast mode.

Recovery mode

Zab goes into recovery mode when the service starts or after the leader crashes, and the recovery mode ends when the leader is elected and most servers have completed state synchronization with the Leader. State synchronization ensures that the leader and Server have the same system state.

Broadcasting mode

Once the leader has synchronized the status of most followers, it can start to broadcast messages. When a server is added to the ZooKeeper service, it starts in recovery mode, discovers the Leader, and synchronizes the status with the Leader. When the synchronization ends, it also participates in the message broadcast. The ZooKeeper service remains Broadcast until the Leader crashes or the leader loses most of the followers.

4.4 Znode Node type

(1) Persistent-persistent node

The node exists on Zookeeper unless manually deleted

(2) EPHEMERAL- The temporary node

The life cycle of the temporary node is bound to the client session. Once the client session fails (the client disconnects from ZooKeeper and the session does not necessarily fail), all the temporary nodes created by the client are removed.

PERSISTENT_SEQUENTIAL node

The basic features are the same as the persistent node, but with the addition of sequential attributes. The node name is followed by an increment integer maintained by the parent node.

(4) EPHEMERAL_SEQUENTIAL- temporary sequential node

Basic features are the same as temporary nodes, with sequential attributes added and a self-incrementing integer number maintained by the parent node appended to the node name.

4.5 Notification Mechanism

The client will set up a Watcher event for a ZNode. When the Znode changes, these clients will receive notification from ZK, and then the client can make business changes according to the zNode changes.

More interview questions: blog.csdn.net/ThinkWon/ar…