A distributed
1.1 Distributed Theory
1 cap theory
The CAP principle, also known as the CAP theorem, refers to the principle that Consistency, Availability and Partition tolerance are incompatible in a distributed system.
- Strong consistency (C) : For a specified client, the read operation can return the latest write operation result
- High availability (A) : Non-failing nodes return reasonable responses at reasonable times
- Zone fault tolerance (P) : Zone fault tolerance refers to whether the system can continue to perform its duties when the network is partitioned (two nodes cannot communicate)
When a network partition occurs, if we want to continue service, it is a choice between strong consistency and high availability.
That is to say, when P is the premise after network partition, C and A can be selected only after P is determined. That is to say, we must realize Partition tolerance.
If Partition is displayed, a node in the system is writing data. To ensure C, read and write operations on other nodes must be prohibited, which conflicts with A. C will be in conflict with A if the read and write operations on other nodes are normal.
For different businesses, different CAP choices need to be taken into account. For e-commerce websites, AP is used for member login, personal setting, personal order, shopping cart and search, because these data are inconsistent within a short period of time. The backstage commodity management needs CP to avoid the inconsistency of commodity quantity; The payment function needs CA to ensure the security and stability of the payment function
2 the Base theory of
Because CAP can only choose one of the two, it is a compromise of CAP theory
-
Basic availability: Reduced availability, degraded running services
-
Soft state: Allows the system to have an intermediate state
-
Final consistency: Data synchronization can be delayed
3 waro mechanism
Success is returned only when all copies are written successfully
=== Advantage: The read is guaranteed first, and the correct data can be read if only one server is alive
Faults: Difficult to write
4 the quorum mechanism
10 replicas, three updates can return success
=== “Advantages: write fast, write friendly
=== To read data of multiple nodes, select the one with the highest version number
5 Paxos algorithm
It is a model of the algorithm, not an implementation
6 RAFT
7 ZAB
If A’s ZID is less than B’s,A will vote for B
1.2 Cluster distributed microservice concept
Hotel chefs, side dishes, dish members, waiters are distributed; There is more than one cook, garnish, passer and waiter. This is a cluster. Distributed is a form of microservice, distributed is the deployment level, microservice is the design level
Clustering: Multiple people doing the same thing together, achieving high availability and spreading stress
Distributed: split at the business level, with multiple people doing different things together ==== at the deployment level of machines
Microservices: Microservices means that modules are divided into independent service units to realize data interaction through interfaces. ===== at the design level
1.3 Common Distributed Cache Schemes
Client-side caching: pages and browsers are cached
CDN: content storage and data cache
Nginx: static resource caching
Server-side cache: local cache (in Java), external cache (redis)
Database layer: cache of persistence layer framework (Mybatis), mysql query cache
Operating system cache: Page cache, Buffer cache
1.4 Consistency between database and cache
- In general, caching is best done with deletes rather than updates: deletes are more lightweight and a form of lazy loading, and updates may design computations for multiple tables
Solution:
- Update the database first, then delete the cache
The problem is that after updating the database, the old data is still read before the cache is deleted
- Delayed double deletion: The cache is deleted first, and then the cache is deleted again after the database is updated
The problem is that after deleting the cache, another thread comes to read it, and the old data is cached again. The cache can be deleted for 1s after updating the database
The final solution, access serialization
1.5 Downgrade and circuit breaker
Circuit breakers are one way to downgrade
For example, the X function of service A depends on an interface of service B. If the interface of service B responds slowly, the response of service X of service A is also delayed. As A result, the threads of service A are stuck on X, and other functions of service A are also blocked or slowed down. In this case, the circuit breaker mechanism is needed, that is, service A does not request interface B, and service A directly returns an error if interface B is found internally, so as to avoid the whole service A being slowed down.
Service fuses are generally caused by a service (downstream service) failure, while service degradation is generally considered from the overall load
Service downgrading: Keep core business and abandon non-core business
Service circuit breaker: To protect the current business from being affected by lower-level business
Service degradation scheme
- Page denial of service: The service is suspended.
- Service interface denial of service: read-only. For add, delete, or modify interfaces, a message is displayed indicating that the server is busy.
- Lazy persistence: The page is accessed normally, and changes are logged to an asynchronous queue or log, which is executed after the service is restored.
- Random denial of service: The service interface randomly denies service, causing users to retry, resulting in poor user experience.
1.6 Actual system traffic limiting with high concurrency
Time window, token bucket, leaky bucket
1.7 Liability equalization algorithm
- polling
- Weighted polling
- randomly
- A weighted random
- Hash address ==== can guarantee access to the same session
- Minimum number of connections