High concurrency

Small to the portal website concurrent reading, online chat, large to the 12306.cn concurrent ticket sales, double 11, 618 and other e-commerce promotion when the concurrent transaction, e-commerce activities second kill system, wechat red envelope, etc. These all reflect the rigid requirements of high concurrency technology.

In the double 11 e-commerce promotion activities, repeatedly set the highest transaction volume. Another thing that stands out is the amount of concurrency each platform handles at its peak. Obviously, the control of high concurrency, to a certain extent, reflects the technical level of an e-commerce platform. It is also common to see some Internet enterprises at home and abroad frequently launch a variety of new technical frameworks with high concurrency. It can be seen that high concurrency is also a research direction that many Internet giants constantly challenge and enjoy. In fact, since the birth of software developers have been constantly exploring and studying high concurrency technologies, such as the Java language in the development of high concurrency scenarios. Keep iterating.

“High concurrency technology” is a broad concept, which refers to an efficient solution to realize concurrency requirements. It is the name of the technical field, which can include the collection of many knowledge systems such as architecture design, service-oriented architecture SOA, distribution, microservices, data processing, multithreading and so on.

In order to avoid the impact of ultra-high frequency high concurrent requests on the servers of e-commerce platforms, it is necessary to process all the concurrent requests. Generally, illegal requests are intercepted by means of verification code mechanism and IP restrictions, and then server clusters are set up to divert legitimate concurrent requests. After that, the maximum number of connections, the maximum number of concurrent service parameters can be set inside the server, and the massive concurrent requests can be peak-clipped through message queues. In order to make the database can handle the high concurrency request stably, we also reduce the amount of database request through caching technology, and reduce the access pressure to the system during the high concurrency peak through service degradation and other strategies. Finally, to ensure data security in extreme cases, you need to set up database clusters and set up proper isolation mechanisms. Therefore, high concurrency technology is actually throughout all aspects of project design, from the gateway to the server, to the database design, which should be considered in the case of high concurrency strategy.

How do you control a large system from a global perspective

When developing large-scale systems, in addition to implementing corresponding functions according to service requirements, the system needs to be designed from multiple dimensions such as high performance and high availability. Here’s a look at the principles of large development systems in terms of high concurrency, fault tolerance, and scalability.

High concurrency principle

The problem of high concurrency is unavoidable for every large project. How to ensure the normal operation of the project in the high concurrency environment? Simple, it can be done vertically and horizontally.

Vertical scaling: Improving the performance of a single machine through software technology or hardware upgrades is like replacing a pony with a strong horse to pull the load when it can’t.

Horizontal expansion: system performance can be horizontally expanded by increasing the number of nodes of the server. That is to say, if a horse does not move, several batches of the same horses can be found and the goods can be divided into several small pieces and then pulled.

Obviously, using the vertical way is the fastest, just need to buy powerful hardware to support, you can quickly improve performance. But standalone performance is limited, just like a horse can’t pull a mountain no matter how powerful. So the big factories spread out horizontally, by a few horses, and then cut the mountain into small pieces. To move mountains.

On the technical level, caching can be used to reduce database access, fusing or downgrading can be used to improve response speed, and traffic limiting can be carried out at the entry point by means of traffic peak clipping. Break up the project first or use microservices to quickly build modules. Then unified service governance is used to manage these modules and middleware is used to build a high availability database cluster based on read/write separation.

Common measures of high concurrency are response time, throughput, or QPS, number of concurrent users, etc.

Fault-tolerant principle

If the high concurrency technology is used improperly, it will not only fail to improve the performance of the system, but also cause the performance of the system to degrade, resulting in a variety of logic confusion, resulting in unpredictable risks. Therefore, it is necessary to preplan potential problems to ensure that the system can have a certain fault tolerance. For example, Spring Boot+Redis is used to achieve distributed caching, MQ is used to achieve transaction consistency in distributed scenarios, MQ, PRG mode and Token are used to solve the problem of repeated submission, “duplicate table” is used to achieve idempotency of operations, and cluster or Zookeeper is used to solve the problem of failed migration.

In distributed system, network delay and other problems are inevitable. In general, this can be handled through the “heartbeat detection mechanism” in the context of long connections. For example, in a normal network environment, when the user clicks the exit button of an APP on the mobile phone, the exit() method on the server side will be called to wait for the exit, so as to log out of the user status. If a user’s mobile network has problems, how can you determine the user’s status? In addition to determining the Session validity duration, heartbeat detection can also be used. If the server receives the heartbeat check, the user status is normal. If the server does not receive the heartbeat check, the user status is normal. If the server fails to connect for several consecutive times, the client is offline.

In order to improve the fault tolerance, can first by means of isolation, for example, to kill system such as predictable the flow of time, you can in advance to kill system isolation into a separate service, prevent seconds kill traffic problems may affect the other services in the system, of course, if the traffic is expected to increase in small, Multi-level caching can also be used to address high concurrency.

The principle of CAP

A distributed system contains multiple nodes. How should data be synchronized between multiple nodes? What factors should you pay attention to when synchronizing data? The CAP principle provides answers to these questions. CAP principles are the basis for understanding and designing distributed systems, including Consistency, Availability, Availability, Partition tolerance and fault tolerance.

  • Consistency C: Data on all nodes is the same at the same time.
  • Availability A: The system can provide normal services within A reasonable period of time.
  • Zoned fault tolerance P: When one or more nodes in a distributed system fail and disconnect from the network environment of the whole system, the system can still provide reliable services, that is, when some nodes fail, the system can still run normally.

As shown in the figure, the write request sent by the client successfully updates service A. However, it is not updated to service B due to network failure. The data inconsistency between service A and service B will be caused when the client accesses service B next time, sacrificing data consistency. Either at design time, the data in service A and service B must be consistent, and when A write to either service fails, all writes are undone. That’s sacrificing usability.

Therefore, in any distributed system, the three CAPS cannot have both, and can only meet two at most. Generally speaking, distribution is bound to encounter network problems, and fault tolerance of partition is the basic requirement, so CP or AP can only be considered in the design.

The idempotent principle

Each module provided by a distributed system interacts with each other through the network, and network problems will affect the number of times users request services. For example, in a redistributed system, if the commodity module has successfully called the payment service, but the payment module fails to return due to the network problem, users may not be able to perceive whether the commodity service has been successfully executed, so they actively execute the repeated execution of the commodity service. In other words, the user doesn’t receive the payment success message and keeps clicking the payment button. The system actually succeeded on the first click, but made an error on the return.

In fact, when using message queues and asynchronous invocations, you need to consider whether the triggered action will be executed repeatedly. To avoid this problem, you can use the principle of idempotency.

The idempotence principle is a limit on the number of times a service can be invoked, meaning that the result is the same whether the interface provided by a service is invoked multiple times or once.

For distributed or microservices, idempotency can be achieved by performing read operations before write operations. For example, when goods and services call the payment interface, only the following steps are required to achieve idempotency.

  • 1. Read operation: Query the payment status in the payment service.
  • 2, write operation: if the payment has been made, directly return the result; If the payment is not made, the payment operation is performed first and then the payment result is returned.

For a variety of systems, such as distributed, microservice, or single-system, idempotency can also be achieved using the more general “de-duplicating” approach as follows.

  • (1) When each operation is executed for the first time, a globally unique ID will be generated, such as the order ID.
  • (2) In the Deduplication table, check whether the ID in 1 exists
  • (3) If there is, return the result directly; If not, the core operation is performed, and the ID in “1” is stored in the “de-duplicate table”, and the result is returned.