Author: HelloGitHub- Lao Xun

Hello, here is HelloGitHub’s HelloZooKeeper series, free, open source, fun, entry-level ZooKeeper tutorials for beginners with basic programming skills.

Project address: github.com/HelloGitHub…

Today we will take you into the source code of ZooKeeper!

First, source code debugging

It is better to teach a man to fish than to give him fish

I always believe that “paper come zhongjae shallow”, the final readers want to really understand the internal principle of ZK, read the source code is essential, if you and I also have the ability to naked eye Debug, it actually can not bother to build source code debugging environment, directly positive hard just.

But if not, download the ZK source code, use the IDE to run directly, and then interrupt the point where you need to learn, isn’t it delightful

1.1 Download Source Code

Download the ZooKeeper 3.6.2 source code

Click on any of the links above to download the package quickly. After downloading the package, unzip it and you will get the following directory structure

.Flag school ── Zookeeper-Server Flag school ── Zookeeper-Metrics Flag School ── Zookeeper-Metrics Flag School ── Zookeeper - docs ├ ─ ─ zookeeper - contrib ├ ─ ─ zookeeper - compatibility - tests ├ ─ ─ zookeeper - client ├ ─ ─ zookeeper - assembly ├ ─ ─ Zk - merge - pr. Py ├ ─ ─ pom. The XML ├ ─ ─ owaspSuppressions. XML ├ ─ ─ excludeFindBugsFilter. XML ├ ─ ─ dev ├ ─ ─ the conf ├ ─ ─ CheckstyleSuppressions. XML ├ ─ ─ checkstyle - strict. XML ├ ─ ─ checkstyle - simple. XML ├ ─ ─ bin ├ ─ ─ README_packaging. Md ├ ─ ─ └─ ├─ ├─ Download.txt ├─ Download.txt ├─ Download.txt ├─ Download.txt ├─ Download.txt ├─ Download.txt ├─ Download.txt ├─ download.txt ├─ download.txt ├─ download.txt ├─ download.txt ├─ download.txt ├─ download.txt ├─ download.txt ├─ download.txtCopy the code

There is pom.xml in the directory, so ZK needs to compile the entire project through Maven, making sure its own Maven is installed first

$MVN - version Apache Maven 3.5.4 (1 edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00) Maven Home: /your/ Maven /home/apache-maven-3.5.4 Java version: 1.8.0_181, vendor: Oracle Corporation, the runtime: / Library/Java/JavaVirtualMachines jdk1.8.0 _181. JDK/Contents/Home/jre Default locale: Zh_CN, platform Encoding: UTF-8 OS name: "MAC OS X ", version: "10.16", ARCH: "x86_64", family:" MAC"Copy the code

If this output indicates that Maven was successfully installed, I will skip over the installation process here. If you have any difficulties, please leave a message to us

1.2 Compiling a Project

Go to the same directory as pom.xml and enter

$ mvn install -DskipTests=true
Copy the code

You will see that the project is compiling, and the final output, BUILD SUCCESS, indicates that the project is compiled

[the INFO] Reactor Summary: [INFO] [INFO] Apache ZooKeeper 3.6.2... SUCCESS [s] 3.621 [INFO] Apache ZooKeeper - Documentation... SUCCESS [s] 2.086 [INFO] Apache ZooKeeper - Jute... SUCCESS [s] 10.633 [INFO] Apache ZooKeeper - Server... SUCCESS [19.246s] [INFO] Apache Zookeeper-metrics Providers............... SUCCESS [0.108s] [INFO] Apache ZooKeeper - Prometheus. IO Metrics Provider.. SUCCESS [s] 1.286 [INFO] Apache ZooKeeper - Client... SUCCESS [s] 0.083 [INFO] Apache ZooKeeper - Recipes... SUCCESS [0.092s] [INFO] Apache Zookeeper-recipes-election.............. SUCCESS [s] 0.244 [INFO] Apache ZooKeeper - Recipes - Lock... SUCCESS [s] 0.259 [INFO] Apache ZooKeeper - Recipes - Queue... SUCCESS [s] 0.295 [INFO] Apache ZooKeeper - an Assembly... SUCCESS [5.425s] [INFO] Apache Zookeeper-Compatibility Tests............. SUCCESS [0.075s] [INFO] Apache Zookeeper-Compatibility tests-co-curator 3.6.2 SUCCESS [0.432s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 44.263s [INFO] Finished at: 2021-01-22T13:49:30+08:00 [INFO] ------------------------------------------------------------------------Copy the code

1.3 Open and configure the project

Then you can open the directory from your IDE. I’m using IDEA here

Then start configuring Run/Debug Configurations

Click + to add a new configuration

Select the Application

1.3.1 PC Startup Configuration

Then the configuration is filled in or selected according to the following figure

  1. Give this configuration a cool name
  2. chooseModify optionsOpen submenu
  3. Make sure all three suboptions in the menu are checked (preceded by √)

Then we look at the configuration

In my computer over compressed the project path to/Users/junjiexun/Desktop/apache – they are – 3.6.2 reader please modify according to their own situation

  1. Select your local JDK (I am local 1.8 other versions of the DO not know whether to do, the lower version is certainly not, because the source code used 1.8 some of the writing method)
  2. choosezookeeper
  3. configurationVM options, the content of- Dlog4j. = the configuration file: / Users/junjiexun/Desktop/apache - they are - 3.6.2 / conf/log4j propertiesIf this parameter is not configured, logs cannot be generated
  4. Specify the startup classorg.apache.zookeeper.server.ZooKeeperServerMain
  5. A command line parameter is required for standalone startup2181 / Users/junjiexun/Desktop/apache - they are - 3.6.2 / data
  6. This should be automatically filled in without modification, but the content is/ Users/junjiexun/Desktop/apache - they are - 3.6.2
  7. Click in the middle+Add the package path with the content asorg.apache.zookeeper.server.*

Then click Apply and OK to complete the save.

Then click on the bug to start it

The 2021-01-22 15:12:16, 319 [myid:] - INFO [main: NIOServerCnxnFactory @ 674] - binding to the port 0.0.0.0/0.0.0.0:2181 2021-01-22 15:12:16, 413 [myid:] - INFO [42] main: WatchManagerFactory @ - Using org. Apache. The zookeeper. Server. Watch. WatchManager as WatchManager 2021-01-22 15:12:16.413 [myID :] -info [main:WatchManagerFactory@42] -using . Org. Apache. Zookeeper server. Watch. WatchManager as watch manager 15:12:16 2021-01-22, 413 [myid:] - INFO [main: ZKDatabase @ 132] - zookeeper. SnapshotSizeFactor = 0.33 2021-01-22 15:12:16, 413 [myid:] - INFO [main: ZKDatabase @ 152] -zookeeper.com mitLogCount=500 2021-01-22 15:12:16.429 [myID :] -info [main:SnapStream@61] - Zookeeper.snapshot.com pression. Method = CHECKED the 15:12:16 2021-01-22, 432 [myid:] - INFO [main: FileSnap @ 85] - Reading The snapshot/Users/junjiexun/Desktop/apache - they are - 3.6.2 / data/version - 2 / snapshot. 2 2021-01-22 15:12:16, [myid:] - 444 INFO [main:DataTree@1737] - The digest value is empty in snapshot 2021-01-22 15:12:16480 [myID :] -info [main:ZKDatabase@289] - Snapshot loaded in 67 ms, highest zxid is 0x2, Digest IS 1371985504 2021-01-22 15:12:16.481 [myID :] -info [main:FileTxnSnapLog@470] -snapshotting: digest is 1371985504 2021-01-22 15:12:16.481 [myID :] -info [main:FileTxnSnapLog@470] -snapshotting: 0 x2 to/Users/junjiexun/Desktop/apache - they are - 3.6.2 / data/version - 2 / snapshot. 2 2021-01-22 15:12:16, 488 [myid:] - INFO [main:ZooKeeperServer@529] -Snapshot Taken in 6 ms 2021-01-22 15:12:16,544 [myID :] -info [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@136] - PrepRequestProcessor (sid:0) started, ReconfigEnabled =false 2021-01-22 15:12:16.546 [myID :] -info [main:RequestThrottler@74] - Zookeeper. Request_throttler. ShutdownTimeout 15:12:16 = 10000, 2021-01-22, 623 [myid:] - INFO - [main: ContainerManager @ 83] Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0 2021-01-22 15:12:16628 [myID :] -info [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.Copy the code

See the log output, if there is no error, it is successful!

Then we can test it on the client side

ZooKeeper client = new ZooKeeper("127.0.0.1:2181".3000.null);
List<String> children = client.getChildren("/".false);
System.out.println(children);
client.close();
Copy the code

The output is

[zookeeper]
Copy the code

The standalone version is done! Let’s try the cluster version

1.3.2 Cluster edition startup configuration

We sometimes need to debug the cluster version of ZK only logic, then the standalone version of the previous is not enough, and HERE I recommend decompressing the previous source package into two different directories, and then open the two directories through the IDE, to fully simulate two different nodes. The configuration of the cluster edition is similar to that of the standalone edition. Let’s take a look at the differences. What I’m going to do here is start two nodes with myID 1 and myID 2.

  1. First will be the defaultzoo_sample.cfgCopy and rename it tozoo.cfgOr you can rename it directly
  2. newdataDirectory (if none exists) and create a new text file under it myID text content 1

Then edit zoo. CFG:

# modifiedDataDir = / Users/junjiexun/Desktop/apache - they are - 3.6.2 / dataAdd the following two linesServer. 1 = 127.0.0.1:2888-3888 for server 2 = 127.0.0.1:2887-3887Copy the code

The configuration is as follows:

  1. Startup class is different, cluster isorg.apache.zookeeper.server.quorum.QuorumPeerMain
  2. The command line argument is differentzoo.cfgThe path, my path is/ Users/junjiexun/Desktop/apache - they are - 3.6.2 / conf/zoo. The CFG

Then is to configure the second node, I it is assumed that the second node project directory is/Users/junjiexun/Desktop/apache – they are – 3.6.2 – bak

The second node changes the contents of the myID file to 2

The contents of zoo.cfg are

# modifiedDataDir = / Users/junjiexun/Desktop/apache - they are - 3.6.2 visiting/data# change, because my two nodes are in the same machine, so the port cannot be repeated
clientPort=2182
Add the following two lines as wellServer. 1 = 127.0.0.1:2888-3888 for server 2 = 127.0.0.1:2887-3887Copy the code

Command line parameter is/Users/junjiexun/Desktop/apache – they are – 3.6.2 visiting/conf/zoo. The CFG

Everything else I didn’t mention is the same as node 1.

Let’s try starting two nodes

2021-01-22 15:44:08.461 [myID :1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):WatchManagerFactory@42] - Using . Org. Apache. Zookeeper server. Watch. WatchManager as watch manager 15:44:08 2021-01-22, 461 (myid: 1) - the INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):WatchManagerFactory@42] - Using . Org. Apache. Zookeeper server. Watch. WatchManager as watch manager 15:44:08 2021-01-22, 471 (myid: 1) - the INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):Learner@677] - Learner received NEWLEADER message 2021-01-22 15:44:08.471 [myID :1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@1811] - Dynamic reconfig is disabled, We don't store the last seen config. 2021-01-22 15:44:08471 [myID :1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):FileTxnSnapLog@470] - Snapshotting: 0 x28100000001 to/Users/junjiexun/Desktop/apache - they are - 3.6.2 / data/version - 2 / snapshot. 28100000001 2021-01-22 15:44:08.472 [myID :1] -info [QuorumPeer[myID =1](plain=[0:0:0:0:0:0:0:0:0]:2181)(Secure =disabled):ZooKeeperServer@529] - Snapshot Taken in 1 MS 2021-01-22 15:44:08/525 [myID :1] -info [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):Learner@661] - Learner received UPTODATE message 2021-01-22 15:44:08/525 [myID :1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@868] - Peer state changed: Following - synchronization 2021-01-22 15:44:08.537 [myID :1] -info [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):CommitProcessor@476] - Configuring CommitProcessor With readBatchSize -1 commitBatchSize 1 2021-01-22 15:44:08.537 [myID :1] -info [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):CommitProcessor@438] - Configuring CommitProcessor With 4 Worker threads. 2021-01-22 15:44:088,544 [myID :1] -info [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):RequestThrottler@74] - Zookeeper. Request_throttler. ShutdownTimeout 15:44:08 = 10000, 2021-01-22, 567 (myid: 1) - the INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@863] - Peer state changed: following - broadcastCopy the code

Finally, the Peer state changed indicates that the election has been completed. The node 1 posted is Follower, so it is done!

Later when you want to learn the source code process, directly local start server can, is not happy ~

1.4 Source code reading refers to north

  • Server start, clusterQuorumPeerMain#mainThat single machineZooKeeperServerMain#main
  • The clientZooKeeper
  • Parsing configuration dependencies,QuorumPeerConfig#parse
  • Memory model (small red book)DataTree
  • Callback Notification (minibar)IWatchManagerView the interface implementation
    • The default implementationWatchManager
    • Optimization schemeWatchManagerOptimized
  • The electionFastLeaderElection#lookForLeader
  • Server instance, set pipelinesetupRequestProcessorsmethods
    • The Leader nodeLeaderZooKeeperServer
    • Followers nodeFollowerZooKeeperServer
    • The Observer nodeObserverZooKeeperServer
  • Each assembly line employeeRequestProcessorView the implementation of the interface
  • The persistence logFileTxnLogThe snapshot,FileSnap
  • Session managementSessionTrackerImpl#run
  • agreementRecordView the implementation of the interface

1.5 Source code reading experience

Reading the source code of a large project must be a time-consuming and laborious work, I also talk about my reading ZK source code experience:

  • Don’t be obsessed with details! Large projects often have a large amount of source code, and if you stare at every detail of logic, you can get lost in a sea of source code.
  • Source code is usually read with a purpose in mind. For example: how ZK performs protocol conversion, how ZK is elected, etc. After having a purpose, looking at the relevant source code is to selectively ignore some other irrelevant details, through method names or comments, to get a sense of the specific code block first.
  • Encounter read not to understand the place, you can go to the net to see if anyone has written a similar blog, standing on the shoulders of giants, it is likely that others a little you pass.
  • Indirect or direct inheritance is common in ZKZooKeeperThreadAre thread objects where the main logic can be viewedrunMethods.
  • The most important attributes of any class must be found in the member fields, and the data structure behind the class can be roughly inferred by looking at the member fields.
  • If a member attribute has a blocking queue field, it will most likely represent the producer-consumer pattern, focusing on the use of the blocking queue and when to put and remove elements.

1.6 summary

I use some graphic length to introduce how to debug ZK source code locally, and how to read the source code scientifically. My local environment is Mac, the IDE is IDEA, if your environment or tools are different from mine, you can also give us a message oh ~

Second, the design pattern applied in ZK

ZK itself is a distributed application and an excellent open source project, so I’ll briefly talk about some of the design patterns I’ve seen in ZK while reading the source code

2.1 Producer and Consumer

This application is the design pattern of ZK china-africa very representative, ZK itself is the design of C/S architecture, the request is sent to the client server data, the response is a server-side data sent to the client, and the order of the ZK to realize some function is not by linear to call a different method to complete, usually by the producer thread, The producer thread will put some request objects received upstream into the blocking queue, and the current method will return. After that, the consumer thread will loop continuously from the blocking queue to complete the subsequent business logic. For example:

  • PrepRequestProcessor, the blocking queue issubmittedRequests
  • SyncRequestProcessor, the blocking queue isqueuedRequests

2.2 Factory Mode

There are some implementations of the interface, and ZK itself provides a default choice, but if the user has configured other implementations in the configuration, the ZK factory automatically creates those other implementations. For example:

  • When creating aClientCnxnSocketWill be based onzookeeper.clientCnxnSocketTo select the IO implementation of the client
  • When creating aIWatchManagerWill be based onzookeeper.watchManagerNameTo select the watch management implementation on the server side
  • When creating aServerCnxnFactoryWill be based onzookeeper.serverCnxnFactoryTo select the IO factory implementation on the server

2.3 Chain of responsibility model

As I have learned before, ZK server business logic processing is realized by connecting one XxxProcessor to another. The processors do not care about the call order, but only associate with each other through nextProcessor. Different server roles can also greatly reuse code in this way

  • In single-machine mode:PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
  • Leader in cluster mode:LeaderRequestProcessor -> PrepRequestProcessor -> ProposalRequestProcessor -> CommitProcessor -> Leader.ToBeAppliedRequestProcessor -> FinalRequestProcessor
  • Followers in cluster mode:FollowerRequestProcessor -> CommitProcessor -> FinalRequestProcessor
  • Observer in cluster mode:ObserverRequestProcessor -> CommitProcessor -> FinalRequestProcessor

2.4 Policy Mode

Zookeeper.snapshot.com pression. The method can be configured to different snapshot compression algorithm, when the need to generate the snapshot files, depending on the different compression algorithms to perform:

  • gz:GZIPInputStream
  • snappy:SnappyInputStream
  • The default:BufferedInputStream

2.5 Decorator mode

The same compression algorithm provides a CheckedInputStream unified processing object that wraps the above three compression implementations around InputStream, all of which are subclasses of InputStream

switch(depending on the configuration) {// The policy pattern is embodied
  case GZIP:
    is = new GZIPInputStream(fis);
    break;
  case SNAPPY:
    is = new SnappyInputStream(fis);
    break;
  case CHECKED:
  default:
    is = new BufferedInputStream(fis);
}
// all wrapped into CheckedInputStream
// Display the decorator pattern
return new CheckedInputStream(is, new Adler32()); 
Copy the code

Third, summary

Today I talked about how to directly from ZK source DEBUG, introduced some design patterns used in ZK, if you have read the source question, welcome to leave a message to me oh. This article was first published on the public account HelloGitHub

The next issue will introduce ZK’s advanced use of pure combat, look forward to it

As always, if you have any questions about this article, it can also be suggestions or questions about the principles of ZK, please come to the warehouse to make an issue to us, or come to the topic of language sparrow discussion.

Address: www.yuque.com/kaixin1002/…