preface

High concurrency and high availability are often asked in the interview, so I want to sort out this aspect. The truth is that not many programmers are really exposed to excessive concurrency, and I’m not either, but while I’m not, I should be, because there are so many things involved, and it can reach all parts of the technology stack.

The problem

Common problems are:

  • How do you optimize a slow query interface?
  • What do you consider when designing a system architecture?
  • How do you transform a system into a highly concurrent and highly available architecture?

And so on…

High concurrency reference indicators

High concurrency depends not only on the number, but also on the specific business scenario. For high concurrency, there are different metrics depending on the product type. Take e-commerce system as an example, commodity browsing is QPS, and order module is TPS. At the same time, you need to pay attention to the number of active users and so on

High concurrency optimization practices

  • Single machine performance optimization
  • Write complex pressure test scripts to automatically realize the pressure test tasks of different modules
  • Hashmap causes an infinite loop, with no cached file IO stream to read the file
  • Optimizing the database
  • Cache data consistency
  • Use load balancing to share high concurrency
  • The load algorithm is adjusted based on service requirements. For example, if the access demand is greater, the forwarding volume is greater
  • Using middleware
  • Local calls are changed to RPC remote calls
  • Inter-module calls use MQ for messaging
  • Frequently accessed data placed in a relational database is stored in a NOSQL database (NOSQL is more flexible by not having to create fields for the data to be stored beforehand)

There are two common design methods for high concurrency: horizontal scaling and vertical scaling

  • Vertical expansion is to improve the processing capacity of a single machine. Hardware can be improved by adding memory, CPU cores, storage capacity, or upgrading disks to SSD and other heap hardware components. Software can use cache to reduce I/O times, and use concurrent or asynchronous methods to increase throughput
  • Scale-out is through cluster deployment to further improve concurrent processing capabilities
  • Do a good job in hierarchical architecture:

Static and static separation, reverse proxy, Web layer API gateway, business service layer can be disintegrated into micro services, storage layer can be heterogeneous database (relational and non-relational), master/slave library

  • Each layer is extended horizontally

Concrete practice

  • High performance practices
  • Cluster deployment
  • Multistage cache
  • Sub-database sub-table and index optimization, using ES to solve complex queries
  • Consider the use of NoSQL
  • asynchronous
  • Traffic limiting (Based on service considerations)
  • Traffic peak clipping, using MQ to undertake traffic
  • Concurrent processing, using multiple threads to parallelize the serial logic
  • Reduce I/O Count
  • Use pooling techniques, HTTP request pooling, database connection pooling, thread pooling
  • JVM optimization, new generation and old age size with GC algorithm selection
  • Lock selection, read more write less use optimistic lock, consider segmental lock to reduce lock conflicts
  • High availability practices
  • Timeout Settings, retry policies, and idempotent design at the interface level.
  • Disaster drills
  • Failover of peer nodes
  • Monitoring alarm
  • Logging system
  • Message reliability assurance in MQ scenarios
  • Limiting and downgrading: Core services are guaranteed at the expense of non-core services
  • High scale practice
  • Rational layered architecture: microservices
  • The storage layer is split, and the database is divided into tables
  • Business layer separation

Refer to the link

This article is very simple. I just read the article and listed some points, but did not expand it in detail. This is also because I still do not have a clear understanding of it now

  • Zhihu: I have no experience in high concurrency projects, but I am often asked questions about high concurrency and performance tuning in interviews. Is there any solution?
  • I hired a gray programmer