preface

Welcome to our GitHub repository Star: github.com/bin39232820… The best time to plant a tree was ten years ago, followed by now

Tips

The interview guide series, which in many cases does not delve into the details, is a way for students to review their knowledge in the role of the interviewee, so I assume that most of the things you, as the interviewer, know.

www.processon.com/view/link/6…

This is the brain map address

Where t

Distributed system development, of course, we need to understand some of the theoretical knowledge of distributed system pull, today to take a look at these theories

Then below is a summary of previous articles

  • 2021-Java Backend Engineer Interview Guide (Introduction)
  • 2021-Java Backend Engineer Interview Guide
  • 2021-Java Backend Engineer Interview Guide -(Concurrency – Multithreading)
  • 2021-Java Backend Engineer Interview Guide -(JVM)
  • 2021-Java Backend Engineer Interview Guide -(MySQL)
  • 2021-Java Backend Engineer Interview Guide -(Redis)
  • Java Backend Engineer Interview Guide -(Elasticsearch)
  • 2021-Java Backend Engineer Interview Guide -(Message Queue)
  • 2021-Java Backend Engineer Interview Guide -(SSM)
  • 2021-Java Backend Engineer Interview Guide (SpringBoot+SpringCloud)

And then let’s look at ZK.

Understand distributed CAP theory? Talk about bai

CAP is an acronym for Consistency, Availability and Partition Tolerance.

In theoretical computer science, CAP theorem states that for a distributed system, read and write operations can only be designed to satisfy two of the following three points:

  • Consistence: All nodes access the same latest copy of data
  • Availability: Non-failed nodes return reasonable responses (not error or timeout responses) in a reasonable amount of time.
  • Partition tolerance: Distributed systems can still provide services against network partitions.

And the network is unreliable, so we must meet the fault tolerance of partition, so in distributed theory, it is impossible to choose CA architecture, only CP or AP architecture.

Take for example the practical application case of CAP

  • ZooKeeper guarantees CP. Read requests to ZooKeeper are consistent at any time. However, ZooKeeper does not guarantee the availability of each request. For example, the service is unavailable during the Leader election process or when more than half of the machines are unavailable.
  • Eureka guarantees AP. Eureka is designed to ensure A (availability) first. There are no Leader nodes in Eureka; each node is equal and equal. So Eureka won’t be unavailable during elections or when more than half of the machines are unavailable, as ZooKeeper is. Eureka guarantees that the failure of most nodes will not affect normal service delivery, as long as only one node is available. It’s just that the data on this node may not be up to date.
  • Nacos supports both CP and AP.

You know Base theory. Tell me about Base theory

BASE is an acronym for Basically Available, soft-state, and Eventually Consistent. BASE theory is the result of balancing consistency C and availability A in CAP. It comes from the summary of distributed practice of large-scale Internet system and is gradually evolved based on CAP theorem, which greatly reduces our requirements on the system.

The core idea of BASE theory

Even if strong consistency cannot be achieved, each application can adopt appropriate methods to achieve final consistency according to its own service characteristics.

BASE theory is essentially an extension and supplement to CAP, and more specifically, a supplement to AP scheme in CAP.

Let’s talk about the Paxos consistency protocol

Paxos problem refers to the problem of Consensus in a distributed system where there are faults but no maliciously corrupt nodes (messages may be lost but not faked).

Paxos is the first proven consensus algorithm based on two-phase commit and extended. Nodes are divided into three types in the algorithm:

  • Proposer: A proposer that submits a proposal and waits for approval to close the case, usually with the client.
  • Acceptor: The acceptor is responsible for voting on proposals, usually the server. More than half of the proposed recipients voted and were selected.
  • Learner learner: Is informed of the result of the proposal and agrees with it, and does not participate in the voting process. Both client and server can be used.

Each node can play multiple roles in the protocol.

Paxos features:

  • One or more nodes can make proposals
  • The system must agree on one of the proposals
  • At best, agreement can be reached on a firm proposal
  • As long as more than half of the nodes are alive and can communicate with each other, the entire system must reach a consistent state

Summary of Paxos two-phase commit

The two phases are prepare and commit. The preparation phase deals with which proposal to vote on, and the submission phase deals with confirming the final value.

To put it simply, after the proposer sends out a proposal and receives some feedback, there are two results: one is that his proposal is accepted by most nodes; the other is that his proposal is not accepted, and he will try again if it is not accepted. The proposer received a majority of acceptance feedback and this should not be considered final confirmation. Because the recipients don’t know that the proposal they just responded to is the absolute majority. Therefore, it is necessary to introduce a new round of reconfirmation, in which the proposer initiates a new round of reconfirmation after judging that the proposal is likely to be accepted by a majority. This is the commit phase. The proposal in the submission stage is sent out, and the proposal value in the other stages is compared, and the maximum value is returned. Therefore, the proposer receives the return message without new proposal, indicating that the lock is successful. If there is new proposal content, the proposal value is compared with the maximum value, and then the larger value is replaced. If you do not receive enough responses, you need to make another request.

Once the majority has accepted the common proposal value, a resolution is formed, called the final confirmed proposal.

The two phases are prepare and commit. The preparation phase deals with which proposal to vote on, and the submission phase deals with confirming the final value.

  • To put it simply, after the proposer sends out a proposal and receives some feedback, there are two results: one is that his proposal is accepted by most nodes; the other is that his proposal is not accepted, and he will try again if it is not accepted.
  • The proposer received a majority of acceptance feedback and this should not be considered final confirmation. Because the recipients don’t know that the proposal they just responded to is the absolute majority. Therefore, it is necessary to introduce a new round of reconfirmation, in which the proposer initiates a new round of reconfirmation after judging that the proposal is likely to be accepted by a majority. This is the commit phase. The proposal in the submission stage is sent out, and the proposal value in the other stages is compared, and the maximum value is returned. Therefore, the proposer receives the return message without new proposal, indicating that the lock is successful. If there is new proposal content, the proposal value is compared with the maximum value, and then the larger value is replaced. If you do not receive enough responses, you need to make another request.
  • Once the majority has accepted the common proposal value, a resolution is formed, called the final confirmed proposal.

So let’s talk about the Raft algorithm

The Raft algorithm is also a majority rule algorithm where a server can play one of the following roles at any one time:

  • Leader: is responsible for Client interaction and log replication. A maximum of one Leader exists in the system at a time
  • Follower: passively responds to request RPCS but never initiates request RPCS
  • Candidate: Intermediate state of transition from Follower to Leader

Term

In Raft, we use a concept that can be understood as a cycle (Term, Term), using Term as a cycle, each Term is a continuously increasing number, each round of election is a Term cycle, only one Leader can be generated in a Term; When Raft starts, all the followers have Term 1, and one of the logical clocks of the followers expires and changes to Candidate, Term is increased by 1, and Term becomes Term 2 (Term), and then the election begins. There are several situations where Term changes:

  • If no Leader is elected or an exception occurs during the current Term 2, the Term increases and the election for a new Term begins
  • In this cycle of Term 2, the Leader is elected, and then the Leader breaks down. Then other followers become candidates, and the Term increases, and the election of a new Term begins
  • When the Leader or Candidate finds that his or her Term is smaller than that of other followers, the Leader or Candidate becomes a Follower and the Term increases
  • If the Term of a Follower is smaller than another Term, the Follower will update the Term to keep it the same as the other followers.
  • Raft ensures that there is only one Leader in each Term. During the normal operation of Raft, the Term of all nodes is consistent. If the node does not fail, the Term will remain forever. If a node receives a request whose Term is smaller than the current Term, it rejects the request.

The election

Raft elections are triggered by a timer. Each node has a different election timer time. The initial state of each node is Follower, and the Term increases after a node’s timer triggers the election. There are three possible scenarios:

  • The RequestVote request receives votes from n/2+1 (majority) nodes, switches from Candidate to Leader, and sends heartBeat to the other nodes to keep the Leader running
  • During this period, if an AppendEntries RPC request is received from another node (for example, the node’s Term is large), the current node becomes Follower; otherwise, keep Candidate rejecting the request
  • If Election timeout occurs, the Term increases and the Election is restarted

During a Term, each node can only vote once, so when there are multiple candidates, there will be a problem that the election initiated by each Candidate has received less than half of the votes. At this time, each Candidate will increment the Term, restart the timer and re-launch the election. Since the timer time in each node is random, there will not be multiple candidates voting at the same time for many times.

Talk about the differences between Paxos and Raft

The same

  • With majority approval, the entries will be fixed and eventually all the nodes will agree

The difference between

  • Raft emphasizes being the only protocol for the leader, the leader is supreme: the newly elected leader owns all the committed logs, whereas PaxOS requires an extra process to get the committed logs from other nodes, which allows the logs to have holes

Talk about Zookeeper

Zookeeper is an open source distributed Apache project that provides coordination services for distributed applications. Zookeeper understands this from a design pattern perspective: Zookeeper is a distributed service management framework designed based on the observer mode. It is responsible for storing and managing the data that everyone cares about, and then accepting the registration of observers. Once the status of these data changes, Zookeeper will be responsible for notifying those observers who have registered on Zookeeper to respond accordingly. In this way, the management mode of the cluster is similar to Master/Slave

  • Zookeeper: A cluster consisting of a leader and multiple followers.
  • The Leader is responsible for initiating and deciding votes and updating system status
  • Followers receive customer requests, return results to the client, and vote for the Leader
  • The Zookeeper cluster can run normally as long as more than half of the nodes in the cluster are alive.
  • Global data consistency: Each server stores the same data copy. The data is consistent regardless of which server the client connects to.
  • Update requests are ordered, and update requests from the same client are executed in the order in which they are sent.
  • Data update atomicity, a data update either succeeds or fails.
  • Real-time: The client can read the latest data within a certain period of time.

The Structure of the ZooKeeper data model is similar to that of the Unix file system, and can be viewed as a tree, with each node called a ZNode. Clearly, the ZooKeeper cluster maintains a data structure of its own. The storage structure is a tree structure in which each node is called a “znode”. Each Znode can store 1MB of data by default, and each Znode can be uniquely identified by its path

Talk about Zookeeper’s event notification mechanism

We can think of a Watch as a trigger registered on a particular Znode. When the Znode changes, that is, when the create, delete, setData methods are called, the corresponding event registered on Znode will be triggered, and the client requesting Watch will receive an asynchronous notification.

The specific interaction process is as follows:

  • The client calls the getData method with the watch argument true. The server receives the request, returns the node data, and inserts the Watch Znode path and Watcher list into the corresponding hash table.

  • When the Znode of the Watch has been deleted, the server will search the hash table, find all Watcher corresponding to the Znode, asynchronously notify the client, and delete the corresponding key-value in the hash table.

Talk about Zookeeper’s ZAB protocol

Zookeeper Atomic Broadcast, which effectively solves the problems of Zookeeper cluster crash recovery and data synchronization between the primary and secondary nodes.

Zookeeper cluster node status

  • Looking: Election status.
  • Following: Follower Indicates the status of the slave node.
  • Leading: Status of the Leader node (primary node).

Maximum transaction ID The maximum ZXID is the latest transaction ID of a node, including the epoch and count. Epoch means epoch, which is equivalent to term when the Raft algorithm chooses the primary.

Talk about ZAB protocol crash recovery, aka master elections

If the current primary Zookeeper node fails, the cluster will perform crash recovery. ZAB’s crash recovery is divided into three phases:

  • During the election phase, nodes in the cluster are in the Looking state. They vote each other with their server ID and latest transaction ID (ZXID). Then, the node will compare its ZXID with the ZXID received from other nodes. If it finds that the ZXID of others is larger than its own, that is, the data is newer than its own, the node will re-vote and vote for the node with the largest ZXID currently known. After each vote, the server counts the votes and determines whether any node has received more than half of the votes. If such a node exists, the node becomes quasi-leader and its state changes to Leading. The status of other nodes changes to Following.
  • The discovery phase is used to discover the latest ZXID and transaction logs from the node. One might ask: since the Leader is chosen as the primary node and already has the latest data in the cluster, why do we need to look for the latest transactions from the node? This is to prevent unexpected situations, such as having multiple leaders in the previous phase due to network reasons. Therefore, at this stage, the Leader gathers ideas and receives the latest epoch values from all the followers. The Leader selects the largest epoch and then adds 1 to generate a new epoch and distributes it to each Follower.
  • Synchronization Phase During the synchronization phase, the latest historical transaction logs collected by the Leader are synchronized to all followers in the cluster. Only if half of the followers have synchronized successfully can the quasi Leader become the official Leader.

Zookeeper data synchronization

ZAB data writing involves the Broadcast stage. In simple terms, when Zookeeper updates data under normal circumstances, the Leader broadcasts data to all followers. The process is as follows:

  • The client issues a write data request to any Follower.
  • The Follower forwards write requests to the Leader.
  • The Leader adopts the two-stage submission mode and sends the Propose broadcast to the followers first.
  • The Follower receives a Propose message, writes the LOG successfully, and returns an ACK message to the Leader.
  • The Leader receives more than half of the ACK messages, returns a success message to the client, and broadcasts a Commit request to the Follower

How do you use ZooKeeper

This section describes how to use the Zookeeper client Curator

  • Co-curator – Framework: Some encapsulation of zooKeeper’s underlying API
  • Toco-client: provides client operations, such as retry policies
  • Toco-recipes: Encapsulates advanced features such as Cache event listening, elections, distributed locks, distributed counters, distributed barriers, and more

I use it all the time and it’s still very useful, so I recommend it to you

The end of the

Next, let’s review the network foundation we often use, is also very important knowledge point oh.

Daily for praise

Ok, everybody, that’s all for this article, you can see people here, they are real fans.

Creation is not easy, your support and recognition, is the biggest motivation for my creation, we will see in the next article

Wechat search “six pulse Excalibur program life” reply 888 I find a lot of information to you