preface
1. Background of ZooKeeper
1.1 Distributed development
Distributed this concept I think we are not strange, but the real combat starts from Google, distributed was put forward in the laboratory a long time ago, but it is more complex to learn more difficult technology in the computer, and the market is not mature, so large-scale commercial applications have not emerged, However, after The publication of MapReduce, DFS and Bigtable papers by Google, the pattern of distribution in the computer industry has changed, and the distributed problem has been realized from the architecture. Moreover, the mature application has been applied in massive data storage and computing, and the scale of its cluster is the largest in the world at present. ZooKeeper: ZooKeeper: ZooKeeper: ZooKeeper: ZooKeeper
On the basis of DFS distributed computing framework and the key and the value data efficiently solve the bottleneck of computation, and developers don’t have to write complex distributed applications, as long as the underlying framework developers complete with less code can complete the development of distributed application, which makes the developers only need to pay attention to business logic. Google’s leading position in the technology of the industry and its unmatched technical strength make IT highly respected for the technology that Google has withdrawn. In recent years, distribution has become a solution for massive data storage and computing, high concurrency, high reliability, and high availability.
1.2 Generating ZooKeeper
It is well known that distributed architecture is usually a centralized design, that is, a master machine connected to multiple processing nodes. The problem can be considered from this point of view, when the master machine fails, the whole system is inaccessible, so ensuring high availability of the system is very critical, that is, to ensure high availability of the master machine. Distributed lock is a better solution to this problem. Multiple master machines grab a lock. This brings us to our focus on Zookeeper.
Chubby is a distributed lock implementation of Google, using the PaxOS algorithm to solve a distributed transaction management system. Zookeeper is yahoo’s distributed lock management system modeled after the mighty Google Chubby. Meanwhile, Zookeeper distributed service framework is a sub-project of Apache Hadoop. It is a reliable coordination system for large distributed systems. It is mainly used to solve some data management problems frequently encountered in distributed applications and can maintain metadata with high reliability. It provides configuration and maintenance, name service, distributed synchronization, and group service. ZooKeeper is designed to encapsulate complex and error-prone key services and provide users with easy-to-use interfaces and a system with high performance and stable functions.
1.3 ZooKeeper Usage
As a distributed service framework, Zookeeper is mainly used to solve the consistency problem of application systems in distributed clusters. Zookeeper provides data storage based on the directory node tree similar to file system. However, Zookeeper is not specifically used to store data. Its main purpose is to maintain and monitor changes in the state of your stored data. By monitoring these changes in data state, you can achieve data-based cluster management, and some of the typical problems that Zookeeper solves are described in more detail below.
Note that the “data” here is limited:
(1) From the data size: We know that ZooKeeper data is stored in a database called ReplicatedDataBase, which is an in-memory database. Since it is in memory, I should know that the amount of data should not be too large, which is very different from HDFS of Hadoop. HDFS data is mainly stored on disks, so data storage is mainly HDFS. ZooKeeper is mainly used for coordination, not data storage.
(2) From the perspective of data type: As mentioned above, ZooKeeper data is stored in memory. Due to the limitation of memory space, we cannot store data as we want. Therefore, ZooKeeper stores data that we care about and the amount of data is not too large, and the corresponding data will be selected according to the function we want to achieve. To put it simply, what to do what to save what data, all functions of ZooKeeper are realized by the nature of the ZK node and the data associated with the node. As for what data to associate, it depends on what you do.
Such as:
(1) Cluster management: Using the temporary node feature, the node is associated with the host name, IP address and other related information of the machine. The single point of failure of the cluster also belongs to this category.
② Unified naming: mainly use the uniqueness of nodes and directory node tree structure.
3. Configuration management: A node is associated with configuration information.
(4) Distributed lock: nodes are associated with competing resources.
2. ZooKeeper application Scenarios
ZooKeeper is a highly available distributed data management and system coordination framework. Based on the implementation of Paxos algorithm, the framework ensures the strong consistency of data in the distributed environment. Based on such characteristics, ZooKeeper can be applied to many scenarios. It is important to note that ZK was not designed for these scenarios in nature, but is a typical way to use them that many developers have explored since, based on the characteristics of the framework. Therefore, we can also design the corresponding scenario implementation according to our own needs. As mentioned above, any function of ZooKeeper is inseparable from the ZooKeeper data structure, and any function is realized by using the “Znode structure features + node associated data “. Ok, so let’s take a look at the ZooKeeper data structure features. The ZooKeeper data structure is as follows:
The Zookeeper data structure has the following characteristics:
① Each subdirectory entry, such as NameService, is called znode, which is uniquely identified by its path. For example, Server1, a Znode is identified by /NameService/Server1.
Znodes can have child directories, and each Znode can store data. Note that EPHEMERAL directory nodes cannot have child directories.
(3) ZNodes have versions. Each Znode can store multiple versions of data, that is, one access path can store multiple copies of data.
4. Znode can be a temporary node. Once the client that created the ZNode loses contact with the server, the ZNode will be automatically deleted. This connection state is called a session. If a ZNode is a temporary node, the session expires and the ZNode is deleted.
⑤ The zNode directory name can be automatically numbered. If App1 already exists, it will be automatically named App2.
(6) ZNode can be monitored, including the change of the data stored in the directory node, the change of the sub-node directory, etc. Once the change can be notified to the monitoring client. This is the core feature of Zookeeper, and many functions of Zookeeper are based on this feature.
2.1 Data Publishing and subscription
(1) Description of typical scenarios
Publish and subscribe is the so-called configuration management, as the name implies, is to publish data to the ZK node, for subscribers to dynamically obtain data, centralized management and dynamic update of configuration information. For example, global configuration information, address lists, etc. Centralized configuration management is common in application clusters. Commercial companies usually implement a centralized configuration management center to meet the requirements of different application clusters for sharing configurations and notify each machine in the cluster of configuration changes.
(2) applications
① The index information and the state of the machine nodes in the cluster are stored in some designated nodes of ZK for each client to subscribe.
② System logs (after processing) are stored. These logs are usually cleared after 2-3 days.
(3) Centralized management of some configuration information used in the application, take the initiative to obtain a time when the application is started, and register a Watcher on the node, after each configuration update, real-time notification to the application, obtain the latest configuration information.
(4) Some global variables needed in the business logic, such as the message queue of some message middleware usually has an offset, which is stored in ZK, so that each sender in the cluster can know the current sending progress.
⑤ Some information in the system needs to be obtained dynamically, and there will be manual modification of this information. Previously, it was usually exposed interfaces, such as JMX interfaces, but with ZK, you just put this information on the ZK node.
(3) Application examples
For example, an application system needs to be run by multiple PC servers, but some configuration items of the application systems run by them are the same. To modify the configuration items, you must modify each PC Server running the application system at the same time, which is very troublesome and error-prone. The configuration information is saved in a directory node of Zookeeper, and the status of all application machines whose configuration information needs to be modified is monitored. Once the configuration information changes, each application machine receives a notification from Zookeeper and applies the new configuration information to the system. The Following figure shows the ZooKeeper configuration management service:
Zookeeper makes it easy to implement centralized Configuration management, such as placing the required Configuration information on the /Configuration node. Zk.exist (“/Configuration “,true) and implement the Watcher callback method process(). When data changes on ZooKeeper /Configuration, each machine will be notified and the Watcher callback will be executed. Zk. getData(“/Configuration “,false,null).
2.2 Name Service
(1) Scene description
Distributed applications usually require a complete set of naming rules, which can not only generate unique names but also facilitate people to identify and remember. In general, tree name structure is an ideal choice. Tree name structure is a hierarchical directory structure, which is friendly to people and will not repeat. Zookeeper’s Name Service is similar to what JNDI can do, which is to associate a hierarchical directory structure to a resource. But Zookeeper’s Name Service is more general. Maybe you don’t need to associate a Name with a particular resource, you just need a Name that doesn’t repeat itself, like creating a unique numeric primary key in a database.
(2) applications
In distributed system, by using naming service, client application can obtain the address, provider and other information of resource service according to the specified name. A named entity can be a machine in a cluster, a service address, a process object, etc., all of which can be called a Name. A common one is a list of service addresses in some distributed service frameworks. By calling the node creation API provided by ZK, it is easy to create a globally unique PATH that can be used as a name. Name Service is a built-in function of Zookeeper. You can implement it by calling the Zookeeper API. You can easily create a directory node by calling the CREATE interface.
(3) Application examples
Alibaba’s open source distributed service framework Dubbo uses ZooKeeper as its naming service to maintain the global service address list. In the Dubbo implementation, the service provider writes its URL address to the specified node/Dubbo/serviceName/providers directory on the ZK at startup to publish the service. When the service consumer starts, it writes its URL address to the subscribe /dubbo/{serviceName}/providers directory and publishes the service. Service consumers start, subscription/dubbo/serviceName/will write your own URL directory, this operation completes the service. Service consumers start by subscribing to provider urls in /dubbo/{serviceName}/providers and writing their own urls to /dubbo/ serviceName/consumers. Note that all addresses registered with ZK are temporary nodes, ensuring that service providers and consumers are automatically aware of resource changes. Dubbo also monitors service granularity by subscribing to the/Dubbo /{serviceName} /consumers directory and writing its own URLS. Note that all addresses registered with ZK are temporary nodes, ensuring that service providers and consumers are automatically aware of resource changes. In addition, as for monitoring service granularity, Dubbo method is a subscription/Dubbo/serviceName/consumers directory into your URL address. Note that all addresses registered with ZK are temporary nodes, ensuring that service providers and consumers are automatically aware of resource changes. In addition, Dubbo also monitors service granularity by subscribing to all provider and consumer information under the/Dubbo /{serviceName} directory.
2.3 Distribution notice/coordination (Distribution of notification/coordination)
(1) Description of typical scenarios
ZooKeeper has a unique watcher registration and asynchronous notification mechanism, which can well realize notification and coordination between different systems in a distributed environment, and realize real-time processing of data changes. The use method is usually that different systems register the same ZNode on ZK, monitor the changes of ZNode (including the content of zNode itself and its children), one of the systems update ZNode, then the other system can receive notification, and make corresponding processing.
(2) applications
① Another heartbeat detection mechanism: the detection system and the detected system are not directly related, but through a node on the ZK association, greatly reducing the system coupling.
(2) Another system scheduling mode: a system is composed of two parts: the console and the push system. The responsibility of the console is to control the push system to carry out the corresponding push work. Some of the actions that administrators make on the console actually change the state of some nodes on ZK, and ZK notifies them of the changes to the client that signed up for Watcher, the push system, and then makes push tasks accordingly.
③ Another work report mode: some similar to the task distribution system, after the sub-task starts, to ZK to register a temporary node, and regularly report their progress (write the progress back to the temporary node), so that the task manager can know the task progress in real time.
In summary, using ZooKeeper for distributed notification and coordination can greatly reduce coupling between systems.
2.4 Distribute Lock
(1) Scene description
Distributed lock, which mainly benefits from ZooKeeper, ensures the strong consistency of data for us. That is, as long as users fully believe that the data of the same Znode on any node (a ZK server) in the ZK cluster must be the same at any time. Locking services can be divided into two categories, one for holding exclusivity and the other for controlling timing.
Hold exclusivity means that only one client that attempts to acquire the lock succeeds in acquiring it. A common approach is to treat a ZNode on a ZK as a lock and implement this by creating a zNode. All clients create the /distribute_lock node, and the client that is successfully created owns the lock.
Control timing, that is, all clients that attempt to acquire the lock will eventually be scheduled to execute, but there is a global timing. This is basically the same as above, except that the /distribute_lock already exists and the client creates temporary ordered nodes under it. The parent node of Zk (/distribute_lock) maintains a sequence, which ensures that the child nodes are created sequentially, and thus the global sequence of each client.
(2) applications
Shared locks are easy to implement within the same process, but difficult to implement across processes or between different servers. Zookeeper makes this easy by creating an EPHEMERAL_SEQUENTIAL directory node that requires the Server to acquire the lock. Then we call getChildren to see if the smallest directory node in the current directory node list is the one we created. If it is, then it has the lock. If not, it calls the exists(String Path, Boolean watch) method and monitors the directory node list on Zookeeper until it creates the smallest directory node in the list to obtain the lock. Releasing the lock is simple. Just delete the directory node that it created itself earlier.
Listing 1 The TestMainClient code
package org.zk.leader.election; import org.apache.log4j.xml.DOMConfigurator; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; import org.apache.zookeeper.ZooKeeper; import java.io.IOException; /** * TestMainClient * <p/> * Author By: Sunddenly studio * Created Date: 2014-11-13 */ public class TestMainClient implements Watcher { protected static ZooKeeper zk = null; protected static Integer mutex; int sessionTimeout = 10000; protected String root; public TestMainClient(String connectString) { if(zk == null){ try { String configFile = this.getClass().getResource("/").getPath()+"org/zk/leader/election/log4j.xml"; DOMConfigurator.configure(configFile); System.out.println(" Create a new connection :"); zk = new ZooKeeper(connectString, sessionTimeout, this); mutex = new Integer(-1); } catch (IOException e) { zk = null; } } } synchronized public void process(WatchedEvent event) { synchronized (mutex) { mutex.notify(); }}}Copy the codeListing 2Locks code
package org.zk.locks; import org.apache.log4j.Logger; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.ZooDefs; import org.apache.zookeeper.data.Stat; import org.zk.leader.election.TestMainClient; import java.util.Arrays; import java.util.List; /** * locks * <p/> * Author By: Sunddenly * Created Date: 2014-11-13 16:49:40 */ public class Locks extends TestMainClient { public static final Logger logger = Logger.getLogger(Locks.class); String myZnode; public Locks(String connectString, String root) { super(connectString); this.root = root; if (zk ! = null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { logger.error(e); } catch (InterruptedException e) { logger.error(e); } } } void getLock() throws KeeperException, InterruptedException{ List<String> list = zk.getChildren(root, false); String[] nodes = list.toArray(new String[list.size()]); Arrays.sort(nodes); if(myZnode.equals(root+"/"+nodes[0])){ doAction(); } else{ waitForLock(nodes[0]); } } void check() throws InterruptedException, KeeperException { myZnode = zk.create(root + "/lock_" , new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL); getLock(); } void waitForLock(String lower) throws InterruptedException, KeeperException { Stat stat = zk.exists(root + "/" + lower,true); if(stat ! = null){ mutex.wait(); } else{ getLock(); } } @Override public void process(WatchedEvent event) { if(event.getType() == Event.EventType.NodeDeleted){ System.out.println(" get notification "); super.process(event); doAction(); Private void doAction(){system.out.println (); } public static void main(String[] args) { String connectString = "localhost:2181"; Locks lk = new Locks(connectString, "/locks"); try { lk.check(); } catch (InterruptedException e) { logger.error(e); } catch (KeeperException e) { logger.error(e); }}}Copy the code2.5 Cluster Management
(1) Description of typical scenarios
Cluster machine monitoring:
This is typically used in scenarios where there are high requirements on the state of the machines in the cluster, the machine on-line rate, and the ability to quickly respond to changes in the machines in the cluster. In such scenarios, there is often a monitoring system to check whether the cluster machine is alive in real time. It used to be that the monitoring system periodically checked each machine by some means (such as ping), or that each machine periodically reported to the monitoring system that it was alive. This works, but there are two obvious problems:
① When there is a change in the cluster machine, there are many things involved in the modification.
② There is a certain delay.
Another cluster machine activity monitoring system can be implemented using two features in ZooKeeper:
(1) The client registers a Watcher on node X and notifies the client if x’s child nodes change.
② Create an EPHEMERAL node that disappears once the session between the client and server ends or expires.
Master election:
In a distributed environment, the same service application is distributed on different machines. Some service logic, such as some time-consuming calculations and network I/O, usually needs to be executed by only one machine in the whole cluster, and other machines can share the result. This greatly reduces rework and improves performance, so this master election is the main problem encountered in this scenario.
Another kind of cluster-master election can be implemented using two features in ZooKeeper:
(1) The strong consistency of ZooKeeper ensures the global uniqueness of node creation in the case of distributed and high concurrency. That is, when multiple clients request to create /Master nodes at the same time, only one client request can be successfully created. With this feature, you can easily conduct clustered elections in a distributed environment.
② In addition, this scenario evolves into a dynamic Master election. This takes advantage of the EPHEMERAL_SEQUENTIAL node feature, where each node is automatically numbered. Allow all requests to be created successfully, but only in a sequence where the machine with the smallest serial number is selected as the Master.
(2) applications
In a search system, if each machine in the cluster generates a full index, it is not only time-consuming, but also cannot guarantee the consistency of index data between each other. So let the Master in the cluster do a seat to generate a full row index and then synchronize it to the other machines in the cluster. In addition, the Master oyakodon has a disaster prevention measure that does not depend on a business to manually seat the Master at 挃 at any time. That is to say, the zK application can obtain the Master from a place such as HTTP if it does not have access to the Master information. In Hbase, ZooKeeper is also used to implement dynamic HMaster voting. The Hbase implementation stores ROOT table addresses and HMaster addresses on the ZK, and HRegionServer registers itself as an Ephemeral node into Zookeeper. HMaster can sense the survivability state of each HRegionServer at any time. In the event of HMaster failure, there is an HMaster to avoid the survivability state of HMaster single point problem. Well is re-selected to run by an HMaster, avoiding the HMaster’s single point problem.
(3) Application examples
Cluster monitoring:
Application cluster, we often need to get every machine in the cluster or rely on the other one in the cluster which machines is alive, and in the cluster machines because of downtime, network chain scission and other reasons can not prompt notice under the condition of artificial intervention to each machine, they can easily realize the function of the cluster management, If multiple servers form a service cluster, a “manager” must know the service status of each machine in the current cluster. If a machine fails to provide service, the other clusters in the cluster must know so that they can adjust the service redistribution strategy. In addition, when the service capacity of the cluster is increased, one or more servers are added. This is the cluster monitoring function of ZooKeeper.
For example, if I have a ZNode named /Configuration on the ZooKeeper server, then each machine in the cluster will go through this node to create an EPHEMERAL node, such as server1 create /Configuration/server1. Server2 creates /Configuration /Server1, and then both Server1 and server2 watch /Configuration, so that changes of data or child nodes under the parent node will be notified to the client that performs watch on this node. Because the EPHEMERAL node has the important feature of losing nodes when the client-server connection breaks or the session expires, the corresponding node is lost when a machine is down or disconnected. Then all clients in the cluster that watch /Configuration will be notified and get the latest list.
Master election:
Zookeeper can not only maintain the service status of machines in the current cluster, but also select a “manager” to manage the cluster. This is another function of Zookeeper, Leader Election. Zookeeper uses a Leader Election to select a Master Server. As before, each Server creates an EPHEMERAL directory node, except that it is also an EPHEMERAL_SEQUENTIAL directory node, so it is an EPHEMERAL_SEQUENTIAL directory node. It is EPHEMERAL_SEQUENTIAL directory node because we can number each Server. We can select the Server with the smallest number as Master, and if the smallest number dies, Since it is the EPHEMERAL node, the node corresponding to the deceased Server was also removed, so the current list of nodes has a minimum number of nodes, we selected this node as the current Master. In this way, the dynamic selection of the Master is realized, avoiding the traditional single Master prone to single point of failure.
Listing 3Leader Election code
package org.zk.leader.election; import org.apache.log4j.Logger; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.ZooDefs; import org.apache.zookeeper.data.Stat; import java.net.InetAddress; import java.net.UnknownHostException; /** * LeaderElection * <p/> * Author By: Sunddenly studio * Created Date: 2014-11-13 */ public class LeaderElection extends TestMainClient { public static final Logger logger = Logger.getLogger(LeaderElection.class); public LeaderElection(String connectString, String root) { super(connectString); this.root = root; if (zk ! = null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { logger.error(e); } catch (InterruptedException e) { logger.error(e); } } } void findLeader() throws InterruptedException, UnknownHostException, KeeperException { byte[] leader = null; try { leader = zk.getData(root + "/leader", true, null); } catch (KeeperException e) { if (e instanceof KeeperException.NoNodeException) { logger.error(e); } else { throw e; } } if (leader ! = null) { following(); } else { String newLeader = null; byte[] localhost = InetAddress.getLocalHost().getAddress(); try { newLeader = zk.create(root + "/leader", localhost, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL); } catch (KeeperException e) { if (e instanceof KeeperException.NodeExistsException) { logger.error(e); } else { throw e; } } if (newLeader ! = null) { leading(); } else { mutex.wait(); } } } @Override public void process(WatchedEvent event) { if (event.getPath().equals(root + "/leader") && Event. GetType () = = event. The EventType. NodeCreated) {System. Out. Println (" informed "); super.process(event); following(); }} void leading() {system.out.println (); } void following() {system.out.println (" become a member of the group "); } public static void main(String[] args) { String connectString = "localhost:2181"; LeaderElection le = new LeaderElection(connectString, "/GroupMembers"); try { le.findLeader(); } catch (Exception e) { logger.error(e); }}}Copy the code2.6 Queue Management
Zookeeper can handle two types of queues:
(1) A queue can only be used when all the members of the queue are together. Otherwise, it is a synchronous queue that waits for all the members to arrive.
② Queue entry and exit operations are carried out in FIFO mode, such as producer and consumer model.
(1) The implementation idea of synchronous queue with Zookeeper is as follows:
Creating a parent directory /synchronizing/start, each member monitors the existence of the Set Watch bit directory /synchronizing/start, and each member joins this queue. This is done by creating a temporary directory node of the /synchronizing/ member_I directory, and then each member obtains all the directory nodes of the /synchronizing/ member_I directory. Synchronizing /start Determines whether the value of I is already the number of members. If it is smaller than the number of members, wait for /synchronizing/start to occur, and if it is equal, create /synchronizing/start.
It’s easier to understand with the following flow chart:
Listing 4 Synchronizing code
package org.zk.queue; import java.net.InetAddress; import java.net.UnknownHostException; import java.util.List; import org.apache.log4j.Logger; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.ZooDefs.Ids; import org.apache.zookeeper.data.Stat; import org.zk.leader.election.TestMainClient; Synchronizing * <p/> * Author By: sunddenly Synchronizing * <p/> * Created Date: 2014-11-13 */ public class Synchronizing extends TestMainClient { int size; String name; public static final Logger logger = Logger.getLogger(Synchronizing.class); /** * constructor ** @param connectString Server connection * @param root directory * @param size queue size */ Synchronizing(String connectString, cascading) String root, int size) { super(connectString); this.root = root; this.size = size; if (zk ! = null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], Ids.OPEN_ACL_UNSAFE,CreateMode.PERSISTENT); } } catch (KeeperException e) { logger.error(e); } catch (InterruptedException e) { logger.error(e); } } try { name = new String(InetAddress.getLocalHost().getCanonicalHostName().toString()); } catch (UnknownHostException e) { logger.error(e); }} /** * Join queue ** @return * @throws KeeperException * @throws InterruptedException */ void addQueue() throws KeeperException, InterruptedException{ zk.exists(root + "/start",true); zk.create(root + "/" + name, new byte[0], Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL); synchronized (mutex) { List<String> list = zk.getChildren(root, false); if (list.size() < size) { mutex.wait(); } else { zk.create(root + "/start", new byte[0], Ids.OPEN_ACL_UNSAFE,CreateMode.PERSISTENT); } } } @Override public void process(WatchedEvent event) { if(event.getPath().equals(root + "/start") && event.getType() = = Event. EventType. NodeCreated) {System. Out. Println (" informed "); super.process(event); doAction(); Private void doAction(){system.out.println (); } public static void main(String args[]) {// Start Server String connectString = "localhost:2181"; int size = 1; Synchronizing b = new Synchronizing(connectString, "/synchronizing", size); try{ b.addQueue(); } catch (KeeperException e){ logger.error(e); } catch (InterruptedException e){ logger.error(e); }}}Copy the code(2)FIFO queue with Zookeeper implementation ideas as follows:
Create a SEQUENTIAL subdirectory /queue_i in a specific directory to ensure that all members are numbered when they join the queue, and return the elements of the SEQUENTIAL queue using the getChildren() method. Then consume the smallest of them, so that the FIFO is guaranteed.
Here is sample code for the queue form of producer and consumer
Listing 5FIFOQueue code
import org.apache.log4j.Logger; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.ZooDefs; import org.apache.zookeeper.data.Stat; import java.nio.ByteBuffer; import java.util.List; /** * FIFOQueue * <p/> * Author By: Sunddenly * Created Date: 2014-11-13 */ public class FIFOQueue extends TestMainClient{ public static final Logger logger = Logger.getLogger(FIFOQueue.class); /** * Constructor * * @param connectString * @param root */ FIFOQueue(String connectString, String root) { super(connectString); this.root = root; if (zk ! = null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.PERSISTENT); } } catch (KeeperException e) { logger.error(e); } catch (InterruptedException e) { logger.error(e); }} /** * @param I * @return */ Boolean produce(int I) throws KeeperException, InterruptedException{ ByteBuffer b = ByteBuffer.allocate(4); byte[] value; b.putInt(i); value = b.array(); zk.create(root + "/element", value, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL); return true; } /** * consumer ** @return * @throws KeeperException * @throws InterruptedException */ int Consume () throws KeeperException, InterruptedException{ int retvalue = -1; Stat stat = null; while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() == 0) { mutex.wait(); } else { Integer min = new Integer(list.get(0).substring(7)); for(String s : list){ Integer tempValue = new Integer(s.substring(7)); if(tempValue < min) min = tempValue; } byte[] b = zk.getData(root + "/element" + min,false, stat); zk.delete(root + "/element" + min, 0); ByteBuffer buffer = ByteBuffer.wrap(b); retvalue = buffer.getInt(); return retvalue; } } } } @Override public void process(WatchedEvent event) { super.process(event); } public static void main(String args[]) {// Start Server testMainServer.start (); String connectString = "localhost:"+TestMainServer.CLIENT_PORT; FIFOQueue q = new FIFOQueue(connectString, "/app1"); int i; Integer max = new Integer(5); System.out.println("Producer"); for (i = 0; i < max; i++) try{ q.produce(10 + i); } catch (KeeperException e){ logger.error(e); } catch (InterruptedException e){ logger.error(e); } for (i = 0; i < max; i++) { try{ int r = q.consume(); System.out.println("Item: " + r); } catch (KeeperException e){ i--; logger.error(e); } catch (InterruptedException e){ logger.error(e); }}}}Copy the code3. ZooKeeper application
Suppose our cluster has:
(1) 20 search engine servers: each is responsible for a portion of the search tasks in the total index.
① Fifteen of the search engine’s servers now offer search services.
② Five servers are generating indexes.
The servers of these 20 search engines often have to tell the server that is providing the search service to stop providing the service and start generating the index, or the server that is generating the index has finished generating the index and is ready to provide the search service.
(2) A master server: responsible for sending search requests to the servers of these 20 search engines and merging result sets.
(3) An alternate master server: Responsible for replacing the master server when the master server goes down.
(4) A Web CGI: send a search request to the master server.
Using Zookeeper ensures that:
(1) Total server: automatically perceive how many servers provide search engines, and send search requests to these servers.
(2) Standby master server: The standby master server is automatically enabled when downtime occurs.
(3) Web CGI: can automatically know the total server network address changes.
(4) The implementation is as follows:
① All servers that provide search engines create ZNodes in Zookeeper. zk.create(“/search/nodes/node1”, “hostname”.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateFlags.EPHEMERAL);
GetChildren (“/search/nodes”, true); zk.getChildren(“/search/nodes”, true);
③ The master server traverses these sub-nodes and obtains the data of the sub-nodes to generate a list of servers that provide search engines;
④ When the master server receives the event information that the child node changes, it returns to the second step.
⑤ The main server creates a node in Zookeeper, zk.create(“/search/master”, “hostname”.getBytes(), ids.open_acl_unsafe, createFlags.ephemeral);
⑥ The standby master server monitors the /search/master node in Zookeeper. When the node data of this ZNode changes, start itself as the master server and put its network address data into this node.
⑦ Web CGI from Zookeeper “/search/master” node to get the total server network address data, and send search requests to it.
The web CGI monitors the “/search/master” node in Zookeeper. When the node data of this ZNode changes, it obtains the network address data of the total server from this node and changes the network address of the current total server.
The last
I here organized a ZooKeeper information document, Spring series of family keg, Java systematic information (including Java core knowledge, interview topics and 20 years of the latest Internet real questions, e-books, etc.) friends who need to pay attention to the public number [procedures yuan small wan] can obtain.