Introduction to the

  1. Web1.0 (circa 1996)
  2. Web2.0 era (circa 2006)
  3. Internet Era –> Internet + Era –> Smart City: Didi Taxi, Ele. me (around 2012)
  4. Big data + cloud computing
  5. AI is coming of age
  6. .

Background the evolution

For the first time

1. Introduction to single application architecture

  • In the earliest period of the development of the Internet, all the code and business were written in JSP, which was also the earliest structure in the early stage of the website.
  • All in One: All modules and code together, no layering (circa 2000).
  • The database and code are on the same server

Cause problems (maintenance)

  1. Code together is not maintainable (the code is all in the JSP page)
  2. Poor fault tolerance. When an exception occurs, users can view the exception information directly. This error may cause service downtime

2. Introduction to layered stages

  1. Layered development to improve maintainability (Controller, Service, DAO)
  2. MVC Architecture (a Design pattern based on Java Web applications)
  3. The database and application servers are deployed separately

Generating problems (high concurrency)

  • As the number of users continues to increase, a single application server cannot meet the requirements.

3. Cluster Phase Overview

  • Cluster: The same service is deployed on multiple servers.
  • Supports high concurrency and high availability.

Generating problems (forwarding, Session)

  • What are all those server users accessing?
  • Where is the session stored?

4. Nginx, Redis stage

  • Niginx uses a Nginx reverse proxy to unify user request addresses, and Niginx uses load balancing to forward user requests.
  • Session sharing using Redis Cluster (Redis Cluster).

Generate a problem (database)

  • The high concurrency of the Nginx+Tomcat cluster causes excessive database strain

5. Database optimization stage

A master-slave replication

  • It adopts the master-slave replication and read-write separation technology of database
  • Data synchronization between master and slave databases: Master is responsible for adding, deleting and modifying data; Slave Performs read/query operations
  • Mysql already supports master-slave replication.

Solr/ElasticSearch

  • The database itself for large data volume query efficiency is slow, fuzzy query support is not very good.
  • For e-commerce websites, search is a very core function. Even with read/write separation, there are still many functions that cannot be effectively solved (word segmentation technology).
  • To solve this problem, the function of full-text retrieval server is introduced

Redis cache

  • In the case of large concurrency, the number of requests to the database for hot data is massive. However, the database cannot cope with such large query requests, resulting in the database cannot provide services externally (abnormal connection).
  • Redis is used for caching to store hot data in Redis without requesting the database every time.
  • And Redis has excellent read and write performance, rich data types and atomic support

Split database tables

  • With 1000 entries in a table, query performance is high.
  • If there are 1 million data in the table, the performance of the query is very low.
  • Individual table performance is improved, but the room for improvement is still limited.

The vertical resolution

  • Suppose a user table has 30 fields: ID, name, age, ID number, height, weight, gender, mobile phone number, home address…..
  • Hot fields: ID, Name, ID number, age, gender, mobile number
  • Unpopular field: other remaining
  • We can split the table by putting popular fields in the User table and unpopular fields in the Userinfo table

Horizontal split

  • Split by demand (province/time)
  • We can split the data from one user table into different tables according to different provinces

Middleware that uses third party sub – library sub – table

  • Mycat, Sharding-JDBC, DRDS (Ali)
  • Design for high concurrency and availability (by adding more servers)

https://blog.csdn.net/jornada_/article/details/82947677

Problems arising

  1. The server is very expensive, and there is a problem of balance between busy and idle time (the number of visits on the days of Double 11 is large). The maintenance of the server requires a lot of labor costs (operation and maintenance engineers).
  2. Poor maintainability: A business change requires redeployment on each server
  3. Poor scalability :(referring to servers) adding another server requires a reconfiguration of the environment
  4. Co-development is not convenient: code conflicts/errors occur when everyone changes the same business code
  5. Single architecture: As service requirements increase, application code increases. As a result, hard disk space is too large when the application is deployed on the server

The second period

Overview of vertical application architecture

When the volume of traffic gradually increases, the acceleration caused by the increase of a single application machine is getting smaller and smaller. The application is divided into several unrelated applications to improve efficiency. At this point, a Web framework (MVC) for accelerating front-end page development is key.

Horizontal split (horizontal split)

Situational requirements

Now the website is divided into the foreground (for users) and the background (for administrators). Is it necessary to split into two projects and deploy them independently? If you need to dismantle, there are some front and background need common code, how to do these codes?

Split method

  • Split according to different levels
  • Divide a large application into smaller applications
Jar # Java bean mapper.jar # Data persistence layer service.jar # business logic layer web.war Web access layer manager.warCopy the code

advantages

  1. Independent JAR package, module reuse
  2. The web. War package can be used to deploy multiple servers, and the manager.war package can be used to deploy one server

Vertical split (vertical split)

Split method

  • Split according to different functions
  • Divide a large application into discrete smaller applications by function, and each application is a separate Web application (moving toward distribution)

advantages

  1. Maintainability: If requirements change, only one application module needs to be adjusted
  2. Function extensibility: to increase different business requirements, corresponding Web modules will be added
  3. Collaborative development: Different teams are responsible for different modules
  4. Deployment performance tuning: Deploy more servers that are responsible for the most visited modules

Problems arising

  1. The page needs to be changed frequently, but the front and back ends are not separated (each application is complete from beginning to end), and the backend application needs to be redeployed each time.
  2. As business requirements continue to increase, there may be business interactions between different modules (different modules are deployed on different servers). How to solve this problem?

In the third period

  • With the increasing number of vertical applications, the interaction between applications is inevitable. Core businesses are extracted as independent services, gradually forming a stable service center, so that front-end applications can respond to changing market demands more quickly. At this point, the distributed Services framework (RPC) for business reuse and integration is key.
  • RPC :(Remote Procedure Call) is a computer communication protocol that allows a program running on one computer to Call a subroutine on another computer without the programmer having to program for this interaction. RPC is a CS mode of distributed computing. The Client always sends a request to the Server to execute several procedures. The Server accepts the request, uses the parameters provided by the Client, and returns the result to the Client after the calculation. There are many RPC protocols, such as the original CORBA, Java RMI, the RPC style of Web Services, Hessian, Thrift, and even Rest apis

Overview of Distributed Architecture

  • Divide a service into multiple sub-services and deploy them on different servers
  • Development/deployment with front-end separation (page + business code)

  • Use RPC, Http, HttpClient and other low-level technologies to solve the call between different modules

Problems arising

  1. Distributed transactions: two-phase commit
  2. Distributed lock: Setnx key value in Redis
  3. Distributed Session: Redis cluster
  4. Distributed logs: ELK (Elasticsearch, Logstash, and Kibana)
  • When there are more and more services, the calls between services are very confusing (I don’t know which services you have) (RPC)
  • Waste of resources (payment management has a small number of visits but deployed 100 servers; Logistics management is heavily visited, but only 50 servers are deployed)

The fourth time

Overview of mobile computing architecture

  • As the number of services increases, problems such as capacity evaluation and waste of small service resources gradually emerge. In this case, a scheduling center needs to be added to manage cluster capacity in real time based on access pressure to improve cluster utilization. At this point, a resource scheduling and Governance center (SOA) for improving machine utilization is key.
  • SOA: Service Oriented Architecture: It is a design approach that includes multiple services that ultimately provide a set of functions through their interdependence. A service usually exists in a separate operating system process. Services are invoked over the network.

The solution

  • Distributed service governance middleware :(Dubbo/SpringCloud)
  • For example:
Take a company as an example: there are grassroots employees, there are management, there are bosses. Initially everyone listen to the command of the boss, who's doing who, according to the need, you could do A thing today, tomorrow do B, while more and more, also more and more things, do things efficiency more and more, management is very chaotic, department division (service), began to do special department to do special things, do research and development, the IT department The hr department only does recruitment; This time will not be able to avoid the happening of the interagency collaboration (server), but how do you know there is such a department can do the things, will depend on administrative department (registry), the new department in administrative where do one for the record registration (services), and then released, let other department know (services), Everyone can happily go to work in the new work order, while still functioning in the organization of the companyCopy the code
  • The RPC protocol underlying Dubbo (Dubbo is an RPC framework).
  • HTTP protocol used at the bottom of SpringCloud.

Microservices and SpringBoot

Micro service

  • Individual applications are broken down into discrete small applications, each of which is called a microservice.

  • Because SOA (Service-oriented Architecture) is also based on small applications, we can also call it microservices architecture

advantages

  1. Microservices are more detailed in the separation of business functions (with stronger reuse), which can improve development efficiency
  2. Strong extensibility: can choose to use the latest technology according to demand
  3. Suitable for Internet projects: short iteration cycle, fast development efficiency

disadvantages

  1. There are too many micro services, and the management/governance cost of services is high
  2. Bad for service deployment: Docker/K8S is available
  3. The technical difficulties are increasing: distributed transactions, locks, sessions, logging
  4. The demands on programmers are increasing: Dubbo/SpringCloud

Problems arising

  • Single application architecture we use the SSM framework
  • When we break down the monolithic application architecture into multiple applets, each applets is a separate Web application, each with SSM configuration (containing a large number of duplicate JAR packages and configuration files)

Springboot

Used to simplify code development and simplify code framework construction.

PS:

  1. Almost all hand, bold for the key part of the operation, easy to read, including original content
  2. Some resources network arrangement, if there is infringement please inform to delete.
  3. Please indicate the source for reprinting this article