>>>> 😜😜😜 Github: 👉 github.com/black-ant CASE Backup: 👉 gitee.com/antblack/ca…

A preface.

Curious about the cluster election process, I took a look at the Zookeeper source code, and this document provides a brief account of the process.

To understand any source code, the first step is to run. This is the main introduction, if the fast run source code, at the same time a brief introduction of the key points, easy to deal with

2. The operation of the source code

2.1 Main startup process

// Step 1: Git fetch
https://github.com/apache/zookeeper.git

// Step 2: Run locally (Intellij)
1.Locate the Zookeeper-server submodule2.Find the corresponding startup class3.Start the project by command// PS: Check the contents of the Zookeeper package (zkserver.cmd) and use the same method to start >>>>
setlocal
call "%~dp0zkEnv.cmd"

set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

Copy the code

2.2 Other startup classes

Enter the source code to see many other startup classes, here refer to the API Doc, a general understanding of the

2.3 Main logic of source code

The main entry class of the source code is QuorumPeerMain, while other classes are called by QuorumPeerMain (e.g.). Here is a simple call flow chart >>>

2. Startup process

Take a quick look at what QuorumPeerMain does when it launches:

3.1 QuorumPeerMain profile

When the class’s main() method is used to start the program, the first parameter is used as the path to the configuration file, which can contain the following information:

  • DataDir: directory where ZooKeeper data is stored.
  • DataLogDir: directory for storing ZooKeeper transaction logs.
  • ClientPort: Port used to communicate with clients.
  • TickTime: The duration of a tick in milliseconds. This is the basic unit of time in ZooKeeper.
  • InitLimit: maximum number of beats for the tracker to wait for initial synchronization with the leader.
  • SyncLimit: Maximum number of beats at which the tracker waits for a message (including heartbeat) from the leader.
  • Server. id: This is the host that the server with the given ID will use for the arbitration protocol :port[:port].
# here is my configuration for tickTime=2000
initLimit=10
syncLimit=5
dataDir=D:\\java\\workspace\\git\\zookeeper\\temp
clientPort=2181
Copy the code

3.2 the IDEA of configuration

VM Options

"-Dzookeeper.root.logger=INFO,CONSOLE" -cp "D:\java\workspace\git\zookeeper\zookeeper\zookeeper-server\target\classes; D:\java\workspace\git\zookeeper\zookeeper\zookeeper-server\target\lib\*; D:\java\workspace\git\zookeeper\zookeeper\bin\.. A \ *; D:\java\workspace\git\zookeeper\zookeeper\bin\.. \lib\*; D:\java\workspace\git\zookeeper\zookeeper\bin\.. \conf"
Copy the code

Program arguments

D:\java\workspace\git\zookeeper\zoo.cfg
Copy the code

One of the most important things is the log level and your source path. After configuration, the project should run normally

Request and reception

The core objects of Zk are ClientCnxn and ServerCnxn respectively

As you can see from log, the default is to walk NIO

[main:NIOServerCnxnFactory@89] - binding to port 0.0. 0. 0/0.0. 0. 0:2181
[NIOServerCxn.Factory:0.0. 0. 0/0.0. 0. 0:2181:NIOServerCnxnFactory@222] - Accepted socket connection from /127.0. 01.:53152
[NIOServerCxn.Factory:0.0. 0. 0/0.0. 0. 0:2181:ZooKeeperServer@903] - Connection request from old client /127.0. 01.:53152; will be dropped if server is in r-o mode
[NIOServerCxn.Factory:0.0. 0. 0/0.0. 0. 0:2181:ZooKeeperServer@942] - Client attempting to renew session 0x100001d6f1f0009 at /127.0. 01.:53152
[NIOServerCxn.Factory:0.0. 0. 0/0.0. 0. 0:2181:ZooKeeperServer@687] - Invalid session 0x100001d6f1f0009 for client /127.0. 01.:53152, probably expired
[NIOServerCxn.Factory:0.0. 0. 0/0.0. 0. 0:2181:NIOServerCnxn@1056] - Closed socket connection for client /127.0. 01.:53152 which had sessionid 0x100001d6f1f0009
Copy the code

3.1 The Client Initiates a Request

Taking SetData as an example, the process is as follows:

  • C- ZooKeeper #
  • C- ClientCnxn # submitRequest: Constructs the Packet and adds it to the Queue
  • C-clientcnxnsocketnio: initiates processing requests
public Stat setData(final String path, byte[] data, int version) throws KeeperException, InterruptedException {

    final String clientPath = path;
    PathUtils.validatePath(clientPath);
    
    // serverPath -> /testWatch
    final String serverPath = prependChroot(clientPath);

    RequestHeader h = new RequestHeader();
    h.setType(ZooDefs.OpCode.setData);
    
    / / build request
    SetDataRequest request = new SetDataRequest();
    request.setPath(serverPath);
    request.setData(data);
    request.setVersion(version);
    
    SetDataResponse response = new SetDataResponse();
    
    // Initiate a request through ClientCnxn
    ReplyHeader r = cnxn.submitRequest(h, request, response, null);
    if(r.getErr() ! =0) {
        throw KeeperException.create(KeeperException.Code.get(r.getErr()), clientPath);
    }
    return response.getStat();
}
Copy the code

The underlying NIO calls are still used, as we’ll see later

3.2 The server processes the request

The core management class of Zookeeper is ZooKeeperServer, which includes the following methods:

  • ProcessConnectRequest: Processes connection requests

Core one: ProcessConnect quest

Creating a connection is the starting point for everything, and is invoked to this class primarily through the following process

C-zookeeperserver: Only the core code is reservedpublic void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
    BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
    
    // Set parameters
    ConnectRequest connReq = new ConnectRequest();
    connReq.deserialize(bia, "connect");
    boolean readOnly = false;
    
    readOnly = bia.readBool("readOnly");
    cnxn.isOldClient = false;
    
    // Timeout time: 4000
    int sessionTimeout = connReq.getTimeOut();
    // Connect password, if not, an empty array
    byte passwd[] = connReq.getPasswd();
    
    // Minimum maximum session timeout
    int minSessionTimeout = getMinSessionTimeout();
    int maxSessionTimeout = getMaxSessionTimeout();

    // Set the timeout period
    cnxn.setSessionTimeout(sessionTimeout);
    cnxn.disableRecv();
    
    // If the session exists
    long sessionId = connReq.getSessionId();
    if(sessionId ! =0) {
    
        long clientSessionId = connReq.getSessionId();
        // Close and then reopen the session
        serverCnxnFactory.closeSession(sessionId);
        cnxn.setSessionId(sessionId);
        reopenSession(cnxn, sessionId, passwd, sessionTimeout);
    } else {
        // Create a sessioncreateSession(cnxn, passwd, sessionTimeout); }}Copy the code

Core 2: Request Processing

There is a slightly convoluted multi-threading process that will be covered in more detail later, but let’s look at the main call flow first

  • C- PrepRequestProcessor # pRequest: A first request processor initiated by the configuration class
  • C- SyncRequestProcessor # processRequest: Add request to Queue (core)
  • C- SyncRequestProcessor # run: This loop will process the Request
  • C- FinalRequestProcessor # processRequest: Initiates a Process call

PS: The processing of the cycle is very good, it is worth studying deeply

4. Data structure stored by Zookeeper

Another important point is to understand the structure of data stored in Zookeeper. Zookeeper has the following core data storage objects:

  • ZKDatabase: data center
  • DataTree: Number of data, core data objects

4.1 Data acquisition

Taking data acquisition as an example, the following process was experienced:

  • C- SyncRequestProcessor # run: Note that this is a continuous process of getting data from the queue
  • C- FinalRequestProcessor # processRequest
  • C- ZKDatabase # getNode
  • C- DataTree # getNode
public class DataTree {
    // Core storage object
    private final ConcurrentHashMap<String, DataNode> nodes = new ConcurrentHashMap<String, DataNode>();

    // watches object collection
    private final WatchManager dataWatches = new WatchManager();
    private final WatchManager childWatches = new WatchManager();

    /** the root of zookeeper tree */
    private static final String rootZookeeper = "/";

    /** the zookeeper nodes that acts as the management and status node **/
    private static final String procZookeeper = Quotas.procZookeeper;

    /** this will be the string thats stored as a child of root */
    private static final String procChildZookeeper = procZookeeper.substring(1);

    /** * the zookeeper quota node that acts as the quota management node for * zookeeper */
    private static final String quotaZookeeper = Quotas.quotaZookeeper;

    /** this will be the string thats stored as a child of /zookeeper */
    private static final String quotaChildZookeeper = quotaZookeeper
            .substring(procZookeeper.length() + 1);

    /** * the path trie that keeps track fo the quota nodes in this datatree */
    private final PathTrie pTrie = new PathTrie();
    
}
Copy the code

conclusion

Content is not much, but more important, after these several links clear, the back around the whole link on the line

The subsequent article has been sorted out almost, and will be sent out later with some modifications. The article was written earlier and the version is older, but the core is almost the same

Appendix:

Zookeeper project structure

/ / ─ zookeeper - assembly
/ / ─ zookeeper - client│ └ ─ zookeeper client - c/ / ─ zookeeper - compatibility - tests│ └ ─ zookeeper - compatibility tests -- curator/ / ─ zookeeper - contrib│ ├─ Heavy Exercises - Heavy Exercises - Heavy Exercises - Heavy Exercises - Heavy Exercises - Heavy Exercises - Heavy Exercises │ ├─ Heavy Exercise │ ├─ Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy Exercise - Heavy exercise ├─ Heavy Exercises - ├─ heavy Exercises - │ ├─ heavy Exercises - heavy exercises - heavy exercises - heavy exercises/ / ─ zookeeper - docs
/ / ─ zookeeper - it
/ / ─ zookeeper - jute
/ / ─ zookeeper - metrics - will│ └ ─ zookeeper - Prometheus - metrics/ / ─ zookeeper - recipes│ ├─ Bass Exercises - Zookeeper-recipes-Election │ ├─ Zookeeper-recipes-lock, ├─ zookeeper-recipes-queue/ / ─ zookeeper server. -

Copy the code

Reference documentation

www.cnblogs.com/jing99/p/12…