Teaching learning – Notes sharing “deep” distributed principle

— All brain maps are made by myself, please do not abuse without permission — the distributed architecture in-depth analysis

---

6, With the circle, as retreat webs

---

A, distributed theory

Part 1 – Distributed Architecture Systems Review

“Distributed System Definition”

A system in which hardware or software components are distributed across different network computers, communicating and coordinating with each other only through messaging

• The business is split into sub-businesses
• Sub-business divisions in differentServer nodeIn the
• Server nodeThe value can be: different cabinets, rooms, or cities……

Distributed system characteristics – distribution, reciprocity, concurrency, lack of global clock, failures are always sent

“History of Distributed Systems”

Alibaba’s “de-IOE” campaign (referring to IBM minicomputers, Oracle databases, EMC’s high-end storage). Alibaba in 2009 “to IOE” strategy technology director revealed that as of May 17, 2013 alibaba’s last IBM minicomputer in Alipay offline

The significance of IOE removal is as follows: 1) The cost performance of upgrading the single-node processing capacity is getting lower and lower; 2) the single-node processing capacity has bottlenecks; 3) Stability and availability are difficult to achieve

“Evolution of Distributed Architecture”

Mind maps in here

Part 2 – Issues facing distributed systems

Communication exception

Due to the unreliability of the network itself, every network communication is accompanied by the risk of network unavailability (such as the unavailability of hardware devices or systems such as optical fibers, routing and DNS), which results in the failure of the final distributed system to carry out a network communication smoothly

The network communication between nodes of distributed system can be carried out normally, and its delay is longer than that of single operation. The huge delay difference also affects the process of sending and receiving messages, so message loss and message delay become very common

Network Partition

Between the network disconnected, but each network of internal network is normal Resulting in the network environment of the whole system been split into several isolated area – – > distributed system will appear local small cluster In extreme cases, these small cluster can independently originally need to complete the whole distributed system functions, including data of the transaction, This presents a very big challenge for distributed consistency

Node Failure

Downtime or “rigor mortis” of the server nodes that make up a distributed system. As a rule of thumb, every node can fail and it happens often

“Three state”

Success Failure Timeout

Because the network is unreliable, although in most cases, network communication can receive a successful or failed response, when the network is abnormal, the timeout phenomenon will occur

• The message was lost when it was sent
• Loss of information during feedback

Part 3 – Distributed theory: consistency

When data is stored in multiple copies, the data in each copy is the same

“Copy Consistency”

In distributed systems, there are often multiple copies of data. If a single database processes all data requests, data consistency can be basically guaranteed by the FOUR ACID Principles.

But involves many copies copies, we almost have no way to guarantee can be updated at the same time of all the machines including backup all data network delay, even if I send all the machines at the same time the update data request, there is no guarantee that these requests are consistent response time lag, there will be some machine data inconsistency between the situation

In general, we cannot find a distributed consistent solution that satisfies all the system attributes of distributed systems. Therefore, how to ensure data consistency without affecting system performance is the key consideration and tradeoff of every distributed system.

. Thus, consistency levels are born

“Consistent classification”

=> Strong consistency

This level of consistency is the most intuitive for users. It requires the system to write and read what it writes. The user experience is good but the implementation often has a great impact on the performance of the system, which is difficult to achieve.

=> Weak consistency

This consistency level constrains that the system does not guarantee that the written value can be read immediately after the data is written successfully, nor does it promise how long it will take for the data to reach the consistency state. However, the system tries to ensure that the data can reach the consistency state after a certain time level (such as second level)

---

–> Read-write consistency

[Solution] :

• Option 1: One option is to go to the main library every time for certain items. (The main library is under great pressure)
• Option 2: We set up an update window, in the latest update period, we default to read from the main library, after this window, we will select the slave library that has been updated recently to read
• Scheme 3: We directly record the updated time stamp of the user, and carry this time stamp with the request. Any slave library whose last update time is less than this time stamp will not respond.

---

–> Monotonic consistency

[Solution] :

• A hash value is calculated based on the user ID and mapped to the machine using the hash value. The same user will only be mapped to the same machine no matter how much it is refreshed. This ensures that you don’t read content from other libraries and have a bad user experience.

---

–> Cause and effect consistency

---

–> Final consistency √

The final consistency is the weakest of all distributed consistency models. Can be considered the “weakest” consistency without any optimization

Part 4 – Distributed theory: CAP theorem

A distributed system cannot simultaneously satisfy C:Consistency, A: Availability and P: Partition tolerance at most two of the three basic requirements

• C consistency Consistency in a distributed system refers to the consistency of data on all nodes, or data on all copies

• A Availability Reads and writes always succeed. This means that the system is always available and the service is always normal

• P-zone fault tolerance The system can provide consistent and availability services even when some nodes or network partitions fail

“C-consistency”

【 Achieve goals 】

1 Database write success –> database check success 2 Database write failure –> database check failure

[Implementation]

1 Data is synchronized to the secondary database after data is written to the primary database. 2 After data is written to the primary database, the secondary database is locked during data synchronization to the secondary database. After the synchronization is complete, the lock is released.

“A-availability”

【 Achieve goals 】

1 Respond to the query result immediately. 2 Do not allow timeout or error responses

[Implementation]

1 After data is written to the master database, data must be synchronized to the slave database. 2 Do not lock the resources in the database. 3 The slave database must return the query data, even if the data is old, but no error or timeout is returned

P-partition tolerance √

【 Achieve goals 】

1 Write operations are not affected if synchronization to the secondary database fails. 2 The failure of one node does not affect other nodes

[Implementation]

1. Replace synchronous operation with asynchronous operation as far as possible. For example, synchronize data from the master database to the slave database in asynchronous mode, so that loose coupling between nodes can be effectively realized; 2 Add database nodes. One secondary node fails and other secondary nodes provide services

C/A/P can only choose 2 from 3

= = >

• A user sent a request to N1 to change the data, to update the database from V0 to V1. Due to the network disconnection, N2 database is still V0. What if there is a request sent to N2, but N2 cannot directly give the latest result V1?

• At this time, there are no more than two methods: 1. Return the wrong V0 data to the user. 2 Block wait until the network communication is restored and data in N2 is updated. Obviously the former sacrifices consistency and the latter usability.

This example is simple, but the point is important. In distributed system, we cannot satisfy the three characteristics of CAP at the same time, so we must abandon one of them. There are obviously three possible permutations without one of them.

Allow the system to be inaccessible. In this case, the user experience is often sacrificed and the user waits until the system data is consistent before resuming service

This is the design of most distributed systems, guaranteeing high availability and fault tolerance of partitions, but sacrificing consistency

If P is to be discarded, then the distributed system is to be discarded, and CAP is out of the question. It can be said that P is a prerequisite for distributed systems, so this case does not exist.

Part 5 – Distributed theory: BASE theory

“Core Ideas”

Even if strong consistency cannot be achieved, each application can adopt appropriate methods to achieve final consistency according to its own service characteristics.

“Basically Available”

Distributed system allows partial loss of availability in case of unpredictable failure (≠ system unavailability)

e.g.

• Loss in response time: Normally, an online search engine needs to return the search results to users within 0.5 seconds. However, due to faults (such as power failure or network disconnection in some equipment rooms of the system), the response time of the search results increases to 1 to 2 seconds.
• Loss of function:
  
  Under normal circumstances, when shopping on an e-commerce site such as Taobao, consumers can complete almost every order smoothly.
  
  However, some consumers may be directed to a degraded page in order to protect the stability of the system (or to ensure consistency) due to the surge of consumers’ shopping behaviors during the shopping peak periods of some festivals (such as Singles’ Day and Singles’ Day)

“Soft state”

Compared with consistency, data copies of multiple nodes are required to be consistent, which is a “hard state”.

Soft state refers to allowing data in a system to exist in an intermediate state without affecting the overall availability of the system -> allowing delays in synchronizing data between copies of data on different nodes.

“Eventually consistent”

After a period of synchronization, all data copies in the system can finally reach a consistent state

The essence of consistency is that the system needs to ensure the consistency of the final data, but does not need to ensure the strong consistency of the system data in real time

Part 6 – Distributed theory: Distributed transactions

Database Transaction Review

Atomicity

Consistency

Isolation

Durablity

“Distributed Transaction Definition”

In fact, distributed transaction is essentially consistent with the concept of database transaction. Since it is a transaction, it needs to meet the basic characteristics of transaction (ACID). However, distributed transaction has a very different form of expression compared with local transaction

Part 7 – Distributed theory: Consistency protocol 2PC

“Prepare Phase”

Undo logs record data before modification and are used for database rollback. Redo logs record data after modification and are used for writing data files after transaction submission

“Commit Phase”

According to the instructions of the transaction manager, participants perform commit or rollback operations and release lock resources used during transaction processing. Note: Lock resources must be released at the last stage.

“Execution Process”

“Shortcomings”

[Core issue] : Synchronous blocking — > During the execution of phase 2 commit, all logic involved in the transaction operation is blocked

• Single point of issue: The coordinator is important throughout the two-phase commit process, and if the coordinator fails during the commit phase, the whole process will not work, and more importantly

Yes: Other participants will remain locked in the transaction resource and will not be able to complete the transaction

• Inconsistent data: Suppose that after the coordinator sends commit requests to all participants, a local network exception occurs or the coordinator crashes itself before all commit requests are sent, resulting in only some participants receiving commit requests. This leads to serious data inconsistencies.

• Too conservative: If in the submission query stage of two-stage submission, the coordinator cannot obtain the response information of all participants due to the failure of the participants, then the coordinator can only rely on its own timeout mechanism to determine whether the transaction needs to be interrupted. Obviously, this strategy is too conservative. In other words, the two-phase commit protocol has no well-designed fault tolerance mechanism, and the failure of any node will lead to the failure of the whole transaction.

Part 8 – Distributed theory: Conformance protocol 3PC

The process of “submitting transaction request” of 2PC is divided into two parts, and a transaction processing protocol consisting of canCommit, preCommit and doCommit phases is formed

“CanCommit”

1 Transaction query The coordinator sends a canCommit request containing transaction contents to all participants, asks if the transaction commit operation can be performed, and waits for the response from each participant

After receiving the canCommit request containing transaction content from the coordinator, the participant will feedback Yes response and enter the preparatory state if it thinks the transaction can be successfully executed –> otherwise feedback No response

“PreCommit”

1 If all participants feedback is Yes (1) send preCommit request ==> Coordinator sends preCommit request to all participants and enters prepared stage. (3) Each participant reports the transaction execution result to the coordinator ==> The participant successfully performs the transaction operation, then feedback Ack

2. After any participant feedback No response, or after waiting timeout, the coordinator cannot receive feedback from all participants, Then, interrupting transaction (1) sends abort request ==> The coordinator issues abort request to all participants. (2) Interrupting transaction ==> Participants will interrupt the transaction whether they receive abort request from the coordinator or wait for timeout during the coordinator’s request

“DoCommit”

At this stage, assuming that the coordinator is in a normal working state and it has received Ack responses from all participants, it will transition from pre-commit state to commit state. (2) Transaction submission ==> After receiving the doCommit request, the participants will officially execute the transaction submission operation and release the transaction resources occupied during the whole transaction execution. (3) Feedback the transaction submission result ==> After completing the transaction submission, Send Ack Response to coordinator (4) Complete transaction ==> The coordinator completes the transaction after receiving Ack messages from all participants

2 Interrupt transaction (1) Send interrupt request ==> After receiving abort request, participants will roll back the transaction according to the recorded Undo information, and release the resource state occupied during the whole transaction execution to commit state after the rollback. (2) Transaction rollback ==> After receiving the doCommit request, the participant will formally execute the transaction commit operation. (3) Feedback transaction rollback result ==> Participants send Ack messages to the coordinator after completing transaction rollback. (4) Interrupt transaction ==> The coordinator interrupts the transaction after receiving Ack messages from all participants

[Comparison of 2PC & 3PC] 1. The timeout mechanism is set up for both the coordinator and the participant. 2. 3 PreCommit is a buffer that ensures the state of all participating nodes is consistent until the final commit phase —–P.S. 3PC protocol does not fully solve the data inconsistency problem ———

Part 9 – Distributed theory: Consistency algorithm Paxos

Since the advent of Paxos has continued to monopolize distributed consistency algorithms, Paxos is almost synonymous with distributed consistency. Many of Google’s large distributed systems, such as Chubby, Megastore, and Spanner, use the Paxos algorithm to solve the distributed consistency problem. The open-source ZooKeeper and MySQL Group Replication, introduced in MySQL 5.7 to replace the traditional master-slave Replication, have adopted the Paxos algorithm to solve the distributed consistency problem.

“Solved what?”

Distributed system consistency problem Solved In a distributed system where the above exception may occur, the value of a data can be quickly and correctly agreed within the cluster

Here the value of a data is not just a narrow sense of a number, it can be a log, it can be a command… The value of a certain data has different meanings according to application scenarios

Distributed systems use multiple copies to store data. If the execution sequence of multiple copies is not controlled, the update operation is performed on multiple copies. Data in each copy is inconsistent due to faults such as network delay and timeout

We hope that the execution sequence of each copy is [OP1 op2 op3…. opn] and Paxos determines the value of immutable variable OPI in turn. After determining OPI each time, each copy performs OPI operation, and so on

“Generating background”

In A cluster environment, the states on all machines are required to be consistent. Two machines want to change A state, machine A wants to change the state to A, and machine B wants to change the state to B

↓ As 2PC/3PC said before, we have to bring in a coordinator.

↓ But what if the coordinator goes down? — You need to make a backup of the coordinator as well as the cluster…….

Who are you going to listen to?

The Paxos algorithm is designed to solve this problem

“Related Concepts”

☆ Proposal: The Proposal information includes Proposal ID and Value.

Role 1 – Client Indicates the Client

Role 2-proposer Proposer

Role 3 – Acceptor decision maker

Role 4 – Learners of Learners who make final decisions

A consistency algorithm needs to ensure the following:

• Of these proposed proposals, only one will be chosen
• If no proposal is put forward, there should be no proposal chosen.
• Once a proposal is selected, all processes should be able to learn to the selected value

“Derivation process”

(To be followed up — later introduction of special topics)

“Algorithm Description”

• Phase 1:

(a) a Proposer selects a proposal number N and sends a Prepare request numbered N to more than half of the acceptors. (b) If an Acceptor receives a Prepare request numbered N that is greater than the number of Prepare requests it has responded to, it sends the highest-numbered proposal (if any) it has accepted as a response. The Acceptor also promises not to accept any proposal numbered less than N.

• Stage 2:

(a) If the Proposer receives a response to a Prepare request numbered N from more than half of its acceptors, then it sends an Accept request to the proposals numbered N to more than half of the acceptors. Note: V is the value of the highest-numbered proposal in the response received. If the response does not contain any proposals, then V is decided by the Proposer itself. (b) If an Acceptor receives an Accept request for a proposal numbered N, it accepts the proposal as long as the Acceptor has not responded to a Prepare request numbered greater than N.

Of course, each Proposer may produce more than one proposal when it is executed, but if each Proposer follows the algorithm described above, it is a guarantee that the algorithm is executed correctly

“Learner learns the selected value”

(To be followed up — later introduction of special topics)

“Keep the Paxos algorithm alive”

(To be followed up — later introduction of special topics)

Part 10 – Distributed theory: Consistency algorithm Raft

Rax provides the same functionality and performance as Paxos, but its algorithm structure is different from Paxos. Rax algorithms are easier to understand and easier to build real-world systems

---

Rax decomposed the consistency algorithm into 3 modules +2 stages:

1 Leader election 2 Log replication 3 Security

A Election process B Normal operation

“Leader Election”

In Rax, at any time a server can play one of the following roles:

• Leader: Handles client interactions, log replication, and other actions. Generally, there is only one leader at a time
• Candidate: A candidate is an entity that nominates itself during the election process and, once elected, becomes the leader
• Followers: Similar to voters, completely passive roles, such servers wait to be notified to vote

The change in their identity was influenced by the election.

Rax uses a heartbeat mechanism to trigger an election. When the server is started, the initial state is follower. Each server has a timer with an Election timeout (typically 150-300ms). If a server receives any message from the leader or candidate without a timeout, the timer restarts. If a timeout occurs, the server starts an election.

Animation detail description

Node Exception

-> The leader is unavailable

-> follower is unavailable

A node is unavailable –> (after a period of time) —-> The node is recovered and the logs of the leader are synchronized

-> Multiple candidates or leaders

Multiple candidates or leaders in a cluster are usually the result of poor data transfer

The situation of multiple leaders is relatively rare, but multiple candidates are more likely to appear in the “chaotic” period when the leader has not been selected at the initial stage of cluster node startup

Log Replication (For Data Consistency)

After the Leader is selected, the client requests are received. The Leader adds the request to its Log as Log entries and then initiates a parallel AppendEntries RPC copy of the Log entry to the other server. When the log is copied to most servers, the Leader applies the log to its state machine and returns the execution results to the client.

• Each request from the client contains instructions to be executed by the replicated state machine.
• The leader adds this instruction to the log as a new log entry, and then initiates an RPC in parallel to the other servers to copy this information.
• The follower responds to an ACK, and if the follower crashes or is slow or loses packets, the leader retries until all the followers have finally copied all the log entries.
• All followers are told to submit the log, and the leader submits the log to his state machine and returns it to the client.

Second, distributed system design strategy

Part 1 – Heartbeat detection

In distributed environments, there are many concepts of nodes. How to detect a node failure or even failure?

Heartbeat detection is usually used to solve this problem

If a node receives a heartbeat, it is normal. However, if the node does not receive a heartbeat, it is not declared dead. In this case, you can help the Server make a decision by checking the heartbeat mechanism periodically and the cumulative failure detection mechanism.

Periodic Heartbeat Detection

The Server sends monitoring requests to the Node cluster every t seconds and sets the timeout period. If the timeout period is exceeded, the Server is judged as “dead”.

“Cumulative Failure Detection Machine”

Based on the periodic detection of heartbeat mechanism, the node return status (including timeout and correct return) within a certain period is counted to calculate the node death probability. In addition, a limited number of retries can be initiated for nodes declared “near death” for further judgment.

Part 2 – High availability design

Master-slave

In the database part, it is commonly called MS mode, namely Master/Slave mode, which is commonly used in database high availability solutions. For example, MySQL and Redis use MS mode to implement Master/Slave replication to ensure high availability

The basis for data replication between MySQL is binary log files.

Once binary logging is enabled for a MySQL database, all operations in its database are recorded as “events” in the binary log as the master database. Other databases communicate with the master server as slaves through an I/O thread. If the master binary log file changes, it copies the changes to its own relay log and then one of the SLAVE SQL threads executes the related “events” into its own database to achieve consistency between the slave and master databases. Master slave replication is also implemented

Active-active

In database mode, it is commonly called MM mode and MS mode, namely multi-master mode. Multiple masters exist in a system. Each Master has the Read-write capability and merges versions based on the timestamp or service logic

“Cluster”

In cluster mode, the high availability of the main control node must be resolved.

Part 3 – Fault tolerance

Classic example — “Cache penetration”

In the project using the cache is often to check whether there is in the cache If we query a data in the cache didn’t exist, can cause every request to query the DB, so cache will lost its meaning, in the large flow, or someone malicious attacks: such as frequent launched for the id for the condition of “1” to query, may hang up the DB

A more subtle approach is to preassign a value to the non-existent key. For example, key= “null”. When the null value is returned, our application can assume that the key does not exist, and then decide whether to continue to wait for access, or to abandon the operation: ↓ 1 If you continue to wait for the access and request the key again after a polling point in time, 2 If the obtained value is no longer null, the key is considered to have a value ====>, thus avoiding transparent transmission to the database and blocking a large number of similar requests in the cache

Part 4 – Load Balancing

There are hardware and software solutions for load balancers. Hardware solutions are famous F5 // software LVS, HAProxy, Nginx

Nginx project

Using Nginx as an example, load balancing has the following policies:

• Round Robin Distributes Web requests from clients to different backend servers according to the order specified in the Nginx configuration file
• Minimum connections Who currently has the least connections and to whom they are distributed
• IP address hashing ensures that the same IP request can be forwarded to the same back-end node for processing to facilitate session persistence
• Weight-based load balancing configuration Nginx distributes more requests to high-configured back-end servers and fewer requests to low-configured servers

Distributed architecture network communication

In the distributed services framework, one of the most basic issues is how remote services communicate. There are many technologies for remote communication in the Java domain: RMI, Hessian, SOAP, ESB, JMS, etc. What are the principles behind them?

Part 1 – Fundamentals

The basic principle of computer system network communication: at the bottom level, network communication needs to do is to transfer the stream from a computer to another computer, based on the transmission protocol and network IO to achieve, among which the transmission protocol is more famous TCP, UDP and so on

TCP and UDP are transmission protocols extended for certain application scenarios based on Socket concepts. Network IO mainly includes BIO, NIO and AIO. All distributed application communication is realized based on this principle, just for easy application. Languages often provide application-layer protocols that are closer to the ease of application

Part 2 – RPC

With RPC, remote services can be called just like local calls, which is a way of communication between processes

RPC is not a specific technique, but refers to the entire network remote call process

“RPC Architecture”

Four core components:

• Client Is the caller of a service
• The Client Stub stores the address message of the server, and then packages the request parameters of the Client into network messages, which are sent to the server remotely over the network
• Server The real service provider
• The Server Stub receives the message sent by the client, unpacks the message, and invokes local methods

(1) The client invokes the service in the way of local invocation (i.e. in the way of interface); (2) After receiving the call, the client stub is responsible for assembling methods and parameters into a message body capable of network transmission (serializing the message body object into binary); (3) The client sends the message to the server through sockets; (4) The server stub decodes the message after receiving it (deserialize the message object); (5) The server stub invokes the local service according to the decoding result; (6) The local service executes and returns the result to the server stub; (7) The server stub packages the returned result into a message (serializes the result message object); (8) The server sends the message to the client through sockets; (9) The client stub receives the result message and decodes it (serializes the result message); (10) The client gets the final result. The goal of RPC is to encapsulate steps 2, 3, 4, 7, 8, and 9.

Regardless of the type of data, the data must be converted into binary streams for transmission over the network. The data sender needs to convert objects into binary streams, and the data receiver needs to restore binary streams into objects

Part 3 – Java RMI

Using JRMP (Java Remote Message Ageing Protocol) as a communication protocol, RMI, a pure Java version of the distributed Remote invocation solution, is primarily used for communication between different virtual machines on different or the same host. The communication here can be understood as an object on one virtual machine calling a method on another virtual machine object

“PMI Architecture”

Three core components:

The client

1) Stub/Stub: proxy for the remote object on the client; 2) Remote Reference Layer: parsing and implementing Remote Reference protocol; 3) Transport layer: send calls, pass remote method parameters, receive remote method execution results.

The service side

1) Skeleton: read the method parameters passed by the client, call the actual object method of the server, and receive the return value after the method execution; 2) Remote Reference Layer: send Remote method call to skeleton after processing Remote Reference; 3) Transport layer: listen for inbound connections of clients, receive and forward calls to the remote reference layer.

The registry

Register a remote object as a URL and reply a reference to the remote object to the client

“Remote Call Procedure”

1) The client queries and obtains the remote object reference from the registry of the remote server; 2) The pile object has the same interface and method list as the remote object, and when the client calls the remote object, it is actually done by the corresponding pile object proxy. 3) After the remote reference layer converts the local reference of the pile into the remote reference of the object on the server, it transfers the call to Transport, which sends the call through TCP protocol. 4) On the server side, the transport layer listens for inbound connections, and once it receives a remote call from the client, it forwards the reference to its upper remote reference layer; 5) The remote reference layer on the server side converts the remote application sent by the client into the reference of the local VIRTUAL machine, and then passes the request to the Skeleton; 6) The skeleton reads the parameters and passes the request to the server, which finally invokes the actual method.

“Result Return Process”

1) If the remote method calls return values, the server passes those results down the skeleton -> remote reference layer -> transport layer; 2) The client transport layer receives the return values and passes them up the transport layer -> Remote reference layer -> pile, which deserializes the return values and passes the final result to the client program.

Code Implementation

(To follow up)

Part 4 – BIO, NIO, AIO

“Synchronous and asynchronous”

Synchronization and asychronize refer to the interaction between an application and the kernel

Synchronization: user processes wait for I/O operations or check whether I/O operations are ready by taking turns The bank withdraws the money, I go to withdraw the money myself, while waiting

Asynchronous: When an asynchronous process call is made, the caller does not get the result immediately. Instead, after the call is made, the called notifies the caller through the status, notification, or callback function to handle the call I ask my friend to withdraw the money for me, and he returns the money to me (delegate to the operating system OS, which needs to support IO asynchronous API).

“Blocking and non-blocking”

Blocking and non-blocking are different ways for a process to access data, depending on the ready state of the IO operation

Blocking mode ==> Read and write will wait for the ATM to queue for the withdrawal, you can only wait for the withdrawal queue (when using blocking IO, The Java call will block until the read and write is finished and then return).

Non-blocking ==> read and write methods return a status value e.g You can’t go until your number is announced, but you can keep asking the lobby manager if the line is ready. (With non-blocking IO, the Java call returns immediately if it cannot be read or written, and continues reading or writing when the IO event dispatcher tells it to be read or written. The loop continues until the read or write is complete.)

3. exemplify

Lao Zhang put the kettle on the fire and stood waiting for it to boil. (Synchronous blocking)

Lao Zhang put the kettle on the fire, went to the living room to watch TV, and from time to time went to the kitchen to check whether the water was boiling. (Synchronous non-blocking)

Lao Zhang put the kettle on the fire and stood waiting for the water to boil. (Asynchronous blocking)

Lao Zhang put the kettle on the fire and went to the living room to watch TV. He stopped looking at the kettle before it rang and went to get the kettle after it rang. (Asynchronous non-blocking)

“BIO” fostered fostered

One connection per thread, that is, when the client has a connection request, the server has to start a thread to process it. If the connection does not do anything, it will cause unnecessary thread overhead, which can be improved by thread pooling

BIO to use

(To follow up)

“NIO” fostered fostered fostered

NIO is introduced

One request per channel, that is, all connection requests sent by the client are registered with the multiplexer, and the multiplexer polls the connection for IO requests before starting a thread for processing.

The most important abstraction that NIO introduces is the concept of channels. Channel Indicates the Channel for data connection. Data can be read from a Channel to Buffer and written from Buffer to a Channel

A buffer channel, a channel, can write data to or store data in a buffer, Buwer

With selectors, a large number of active I/O channels can be monitored and maintained in real time with a single thread

The characteristics of

When a connection is created, it does not need to correspond to a single thread. The connection is registered with the multiplexer, so a connection only needs one thread. All connections need one thread to operate.

Example analysis

1. Each child has a teacher. Every teacher asks the child whether he or she needs to go to the toilet at intervals, and if he or she needs to go to the toilet, he or she will be taken to the toilet. 100 children need 100 teachers to ask, and every child needs a teacher to lead him or her to go to the toilet. This is the IO model, one connection corresponds to one thread.

2. All the children have the same teacher. The teacher asks all the children whether they want to go to the toilet at intervals, and then sends the number of children who want to go to the toilet to the toilet in batches every moment. This is the NIO model. All the children register with the same teacher, corresponding to which all connections are registered in a thread, and then polling in batches.

NIO use

(To follow up)

“AIO” fostered fostered fostered

When a stream is available for reading, the operating system transfers it to the buffer of the read method and notifies the application program. For write operations, the operating system notifies the application program when the write stream is complete. Therefore, both read and write are asynchronous, and the callback function is called upon completion.

Usage scenario: The architecture with a large number of connections and long connections (heavy operation), such as album server. Focus on the call OS to participate in concurrent operations, programming is more complex. Java7 support is available

Part 5 – Netty

Netty helps you develop a web application quickly and easily, simplifying and streamlining NIO’s development process. As the most popular NIO framework, Netty has been widely used in the Field of Internet, big data distributed computing, game industry, communication industry and so on. Well-known Elasticsearch and Dubbo frameworks all use Netty inside

“Advantages”

• Provides a uniform API for various transport protocols
• Highly customizable threading model — single thread, one or more thread pools
• Better throughput, lower wait latency
• Less resource consumption
• Minimize unnecessary memory copies

“Shortcomings”

• NIO’s libraries and apis are cumbersome to use. You need to be familiar with Selector, ServerSocketChannel, SocketChannel, ByteBuffer, etc.
• Reliability is not strong, development workload and difficulty are very large
• NIO Bug. An example is the Epoll Bug, which causes the Selector to poll empty, eventually causing the CPU to be 100%.

“Thread model”

① Single-threaded model

② Thread pool model

(3) Netty model

Netty abstracts two groups of thread pools, BossGroup for receiving client connections and WorkerGroup for network read and write operations. A NioEventLoop represents a thread that executes processing tasks in a continuous loop, and each NioEventLoop has a selector that listens for the socket network channel bound to it. NioEventLoop adopts serial design inside, from the message read -> decode -> processing -> encoding -> send, always in the IO thread NioEventLoop responsible.

Netty Core Components

① ChannelHandler and its implementation classes

Defines a number of event handling methods that we can override to implement specific business logic

We often need to define a Handler class to inherit ChannelInboundHandlerAdapter, then rewrite the corresponding method to implement the business logic

- public void channelActive(ChannelHandlerContext ctx), channel ready event -public void channelRead(ChannelHandlerContext ctx, Object msg), channel read data event -public void channelReadComplete(ChannelHandlerContext ctx), data read completion event -public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause), an abnormal event occurs on the channelCopy the code

(2) the ChannelPipeline

A collection of handlers that handle and intercept inbound or outbound events and operations, acting as a chain running through the Netty

- ChannelPipeline addFirst(ChannelHandler... handlers)Add a business processing class (handler) to the first bit in the chain - ChannelPipelineaddLast(ChannelHandler... handlers)To add a business processing class (handler) to the last position in the chainCopy the code

(3) ChannelHandlerContext

The event handler context object, the actual processing node in the Pipeline chain.

Each processing node ChannelHandlerContext contains a specific event handler, ChannelHandler, which also binds the corresponding pipeline and Channel information. Easy to call ChannelHandler

- ChannelFuture close(a), close the channel - ChannelOutboundInvokerflush(a), refresh - ChannelFuturewriteAndFlush(Object msg)Write data to ChannelPipeline when the next ChannelHandler of the previous -channelhandler starts processing (outbound)Copy the code

(4) ChannelFuture

The result of an asynchronous I/O operation in a Channel. In Netty, all I/O operations are asynchronous. I/O calls are returned directly, and the caller does not get the result immediately

- Channel channel(a), returns the channel where I/O operations are being performed - ChannelFuturesync(a)Wait for the asynchronous operation to completeCopy the code

⑤ EventLoopGroup and its implementation NioEventLoopGroup

Is an abstraction from a set of EventLoops

To make better use of multi-core CPU resources, there are usually multiple Eventloops working at the same time. Each EventLoop maintains a Selector instance. You can get one of the EventLoops from the group according to certain rules to process the task. In Netty server-side programming, we generally need to provide two EventLoopGroups such as BossEventLoopGroup and WorkerEventLoopGroup

- public NioEventLoopGroup(a), the construction method -publicFuture<? >shutdownGracefully(a), disconnect and close the threadCopy the code

6 ServerBootstrap and Bootstrap

ServerBootstrap is the Netty server startup assistant. You can use the Bootstrap to configure the Netty client

- public ServerBootstrap group(EventLoopGroup parentGroup, EventLoopGroup childGroup), this method is used on the server side to set two Eventloops -public B group(EventLoopGroup group)This method is used on the client to set an EventLoop -public B channel(Class<? extends C> channelClass)This method is used to set up a server-side channel implementation -public <T> B option(ChannelOption<T> option, T value)To add configuration - to ServerChannelpublic <T> ServerBootstrap childOption(ChannelOption<T> childOption, T value)Is used to add configuration - to the received channelpublic ServerBootstrap childHandler(ChannelHandler childHandler), this method is used to set the business processing class (custom handler) -public ChannelFuture bind(int inetPort), this method is used on the server side to set the occupied port number -public ChannelFuture connect(String inetHost, int inetPort)This method is used by the client to connect to the serverCopy the code

“Netty Version case Implementation”

Target: The netty client is used to send data to the server, and the server receives messages printed

① Introduce Maven dependencies first

<dependency> 
  <groupId>io.netty</groupId> 
  <artifactId>netty-all</artifactId> 
  <version>4.1.6. The Final</version> 
</dependency>
Copy the code

② Server implementation code

NettyServer.java

public class NettyServer {
    
    public static void main(String[] args) throws InterruptedException {
    
        // Create two NioEventLoopGroups (currently these two instances represent two thread pools, the default number of threads is CPU cores multiplied by 2)
        // bossGroup is used to receive clients
        // workerGroup is used to process requests
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
    
        // Create the service startup helper class ==> Assemble the necessary components
        ServerBootstrap serverBootstrap = new ServerBootstrap();
        // Set the group. BossGroup is responsible for connection, and workerGroup is responsible for I/O processing after connection
        serverBootstrap.group(bossGroup, workerGroup)
                // Channel method -> specify the channel type that the server listens for
                .channel(NioServerSocketChannel.class)
                .childHandler(new ChannelInitializer<NioSocketChannel>() {
    
                    @Override
                    protected void initChannel(NioSocketChannel nioSocketChannel) throws Exception {
                        // Transfer channel
                        ChannelPipeline pipeline = nioSocketChannel.pipeline();
                        // Add a channel handler to the channel, which can also be a listener
                        pipeline.addLast(new StringEncoder());
                        pipeline.addLast(new StringDecoder());
                        // Add our own handler to the listener queue
                        pipeline.addLast(new SimpleChannelInboundHandler<String>() {
                            @Override
                            protected void channelRead0(ChannelHandlerContext channelHandlerContext, String s) throws Exception { System.out.println(s); }}); }});// bind listens for ports
        ChannelFuture channelFuture = serverBootstrap.bind(8000).sync();
        System.out.println(" Tcp server start success ! ");
        // Block and wait until the server's channel is closedchannelFuture.channel().closeFuture().sync(); }}Copy the code

③ Implementation part of client NIO

public class NettyClient {
    
    public static void main(String[] args) throws InterruptedException {
        // The client startup helper class
        Bootstrap bootstrap = new Bootstrap();
        // Thread pool instance
        NioEventLoopGroup group = new NioEventLoopGroup();
        // Add to the group
        bootstrap.group(group)
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline().addLast(newStringEncoder()); }}); Channel channel = bootstrap.connect("127.0.0.1".8000).channel();
        while (true) {
            channel.writeAndFlush(new Date() + "hello world");
            Thread.sleep(2000); }}}Copy the code

Part 6 – Custom RPC based on Netty

As mentioned earlier, RPC is called a remote procedure call

There are two familiar remote calls:

1. The generalized remote call based on HTTP restful form is represented by Feign and restTemplate of Spring Could. The protocol adopted is HTTP layer 7 call protocol, and the parameter and response serialization of the protocol are mainly in JSON format and XML format

2. TCP based on the narrow sense of RPC remote call represented by Ali’s Dubbo, mainly through NetTY to achieve layer 4 network protocol, NIO to asynchronous transmission, serialization can also be JSON or Hessian2 and Java own serialization, etc., can be configured

“Requirement Description”

“Thought steps”

1. Create a common interface project and create interfaces and methods for conventions between consumers and providers.
2. Create a provider that listens for consumer requests and returns data by convention.
3. Create a consumer class that transparently calls its own non-existent methods and internally requests the provider to return data using Netty

“Implementation”

① Adding a dependency

<dependency> 
  <groupId>io.netty</groupId> 
  <artifactId>netty-all</artifactId> 
  <version>4.1.6. The Final</version> 
</dependency>
Copy the code

② Implementation of the server

Source link

Modular structure

• Common introduces Netty dependencies
• Provider and comSumer rely on common (pointing to in both pom.xml)

---

public interface IUserService {
    public String sayHello(String msg);
}
Copy the code

public class ServerBoot {
    public static void main(String[] args) throws InterruptedException {
        UserServiceImpl.startServer("127.0.0.1".8999); }}Copy the code

public class UserServiceHandler extends ChannelInboundHandlerAdapter {
    
    /** * This method is called when the client [reads data] **@param ctx
     * @paramMSG The parameter (UserService#sayHello#"R U OK?") passed when the client sends the request. ) *@throws Exception
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        // 1. Determine whether the current request complies with the rule
        if (msg.toString().startsWith("UserService")) {
            // call the implementation class to get result
            UserServiceImpl userService = new UserServiceImpl();
            String result = userService.sayHello(msg.toString().substring(msg.toString().lastIndexOf("#") + 1));
            // 1-2 writes the result to the clientctx.writeAndFlush(result); }}}Copy the code

public class UserServiceImpl implements IUserService {
    
    /** * The method that the client will call remotely in the future@param msg
     * @return* /
    @Override
    public String sayHello(String msg) {
        String str = "Server returns data :\t" + msg + "\t===>\t" + new Date();
        System.out.println(str);
        return str;
    }
    
    /** * start the server *@paramIP address *@paramPort Indicates the port number */
    public static void startServer(String ip, int port) throws InterruptedException {
        Create two thread pool objects
        NioEventLoopGroup bossGroup = new NioEventLoopGroup();
        NioEventLoopGroup workerGroup = new NioEventLoopGroup();
        // 2. Create a server boot object
        ServerBootstrap bootstrap = new ServerBootstrap();
        // 3. Configure the boot object
        bootstrap.group(bossGroup, workerGroup)
                // Set the channel to NIO
                .channel(NioServerSocketChannel.class)
                // Create a listener channel
                .childHandler(new ChannelInitializer<NioSocketChannel>() {
                    @Override
                    protected void initChannel(NioSocketChannel nioSocketChannel) throws Exception {
                    
                        /* Get the pipe object */
                        ChannelPipeline pipeline = nioSocketChannel.pipeline();
                        /* Set the code for the pipeLine object */
                        pipeline.addLast(new StringEncoder());
                        pipeline.addLast(new StringDecoder());
                        /* Add our custom ChannelHandler to the pipe */
                        pipeline.addLast(newUserServiceHandler()); }});// 4. Bind ports
        ChannelFuture future = bootstrap.bind(8999).sync(); }}Copy the code

③ Implementation of the client (consumer)

One caveat for the consumer is that the invocation needs to be transparent, that is, the framework consumer doesn’t care about the underlying network implementation. Here we can use the JDK’s dynamic proxy for this purpose

public class ConsumerBoot {
    
    // Parameter definition
    private static final String PROVIDER_NAME = "UserService#sayHello#";
    
    public static void main(String[] args) throws InterruptedException {
        // 1. Create proxy objects
        IUserService service = (IUserService) RPCConsumer.createProxy(IUserService.class, PROVIDER_NAME);
        // 2. Write data to the server
        while (true) {
            String result = service.sayHello("R U OK ?");
            System.out.println(result);
            Thread.sleep(2000); }}}Copy the code

public class RPCConsumer {
    
    // 1. Create a thread pool object ==> it handles custom events (thread count is the number of CPU cores on the current computer)
    private static ExecutorService executorService =
            Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
    
    // 2. Declare a custom event handler UserClientHandler
    private static UserClientHandler userClientHandler;
    
    /** * 3. Initialize our client (create connection pool ->bootStrap-> Set up bootStrap-> Connect server) */
    public static void initClient(a) throws InterruptedException {
        // 1) Initialize UserClientHandler
        userClientHandler = new UserClientHandler();
        // 2) Create a connection pool object
        NioEventLoopGroup group = new NioEventLoopGroup();
        // 3) Create a client boot object
        Bootstrap bootstrap = new Bootstrap();
        // 4) Configure the boot object
        bootstrap.group(group)
                /* Set channel to NIO */
                .channel(NioSocketChannel.class)
                /* Set the request protocol to TCP */
                .option(ChannelOption.TCP_NODELAY, true)
                /* Listen on channel and initialize */
                .handler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel socketChannel) throws Exception {
                        /* Get the pipe */
                        ChannelPipeline pipeline = socketChannel.pipeline();
                        /* Set the encoding */
                        pipeline.addLast(new StringEncoder());
                        pipeline.addLast(new StringDecoder());
                        /* Add a custom event handler */pipeline.addLast(userClientHandler); }});// 5) Connect to the server
        bootstrap.connect("127.0.0.1".8999).sync();
    }
    
    /** * 4. Write a method that uses the JDK dynamic proxy to create objects *@paramServiceClass Interface type ==> On which interface the subclass proxy object * is generated@param providerParam "UserService#sayHello#..."
     * @return* /
    public static Object createProxy(Class
        serviceClass, String providerParam) {
        
        return Proxy.newProxyInstance(Thread.currentThread().getContextClassLoader(),
                new Class[]{serviceClass}, new InvocationHandler() {
                    @Override
                    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
                        // 1) Initialize the client
                        if (userClientHandler == null) {
                            initClient();
                        }
                        // 2) Set parameters for UserClientHandler
                        userClientHandler.setParam(providerParam + args[0]);
                        // 3) Using a thread pool, start a thread to process call() and return the result
                        Object result = executorService.submit(userClientHandler).get();
                        // 4) Return the result
                        returnresult; }}); }}Copy the code

public class UserClientHandler extends ChannelInboundHandlerAdapter implements Callable {

    // 1. Define member variables
    private ChannelHandlerContext context; // Event handler context object (store handler information, write operations)
    private String result; // Record the data returned by the server
    private String param; // Records the data to be sent to the server
    
    /** * 2. If the ChannelActive client is connected to the server, this method is automatically executed@paramCTX context *@throws Exception
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
    
        // Initialize ChannelHandlerContext
        this.context = ctx;
    }
    
    /** * 3. Implement ChannelRead ==> when we read data from the server, this method automatically executes *@paramCTX context *@paramMSG Read data from the server *@throws Exception
     */
    @Override
    public synchronized void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        // Save the read data
        this.result = msg.toString();
        notify();
    }
    
    // 4. Write the client data to the server
    public synchronized Object call(a) throws Exception {
        // context writes data to the server
        context.writeAndFlush(param);
        wait();
        return result;
    }
    
    // 5. Method for setting parameters
    public void setParam(String param) {
        this.param = param; }}Copy the code

④ Running test

Run main() on the Server and main() on the Cliend

Read and write every two seconds.