The history of distributed architecture
In 1946, the world’s first electronic computer was born in the University of Pennsylvania in the United States. Its name is ENICAC. ENICAC is a computer with large weight and slow computing speed, but it represents the coming of the computer era and has fundamental significance in the development of the Internet.
The composition of the computer has five parts, respectively is: input devices, output devices, memory, the memory by arithmetic unit and controller inside, there is a von neumann only by the composition of the model is very image of the object computer are described, but the computer is a data flow, the instruction stream and control flow for calculation and normal operation. As shown in figure:
After ENIAC, electronic computers entered the era of the ibM-dominated mainframe. The first IBM mainframe, SYSTEM/360, was introduced in 1946, which enabled IBM to dominate the mainframe industry in the 1950s and 1960s. In the mainframe era, computer architecture evolved in two directions: CISC(computer language instruction set executed by microprocessors) CPU-based inexpensive personal PCS and RISC(compact instruction set computers) inexpensive small UNIX servers.
The emergence of mainframe, with computing capacity and processing capacity, high stability and security, in a long period of time to lead the development of the field of computing. But centralized computer systems have brought some problems, which are increasingly unable to meet the needs of users, such as:
1. Large mainframe is very expensive, ordinary small businesses can not afford to use.
2. The mainframe is complicated and the cost of talent training is high.
3. Single point problems, such as the failure of the mainframe, the whole system is suspended and cannot run, causing great losses to the enterprise.
With the progress of technology, the performance of personal PC is getting higher and higher, and the cost is getting lower and lower.
Alibaba launched a drive to de-ioE in 2009
IOE refers to IBM’s minicomputer,Oracle’s database and EMC’s high-end storage devices, the movement to go IOE in 2009, until alipay’s last IBM minicomputer went offline in 2003.
Why go to IOE
Alibaba used to use Oracle for its databases and use minicomputers and high-end storage devices to provide high-performance data processing and storage services. With the increase of the company’s business volume and user scale, Oracle database, the traditional centralized architecture, has encountered a bottleneck in expansion. To the traditional Oracle,DB2 are mainly centralized, the existing disadvantage is the lack of scalability, centralized expansion is mainly adopted upward expansion rather than horizontal expansion, such a long time, will encounter system bottlenecks sooner or later.
Common concepts of distributed architecture
The cluster
The small restaurant is a chef, cut vegetables wash vegetables prepare stir-fry all dry. Later, there were too many guests in the kitchen for one cook, so another cook was hired. Both cooks could cook the same dishes. The relationship between the two cooks was cluster.
distributed
In order to make the chef concentrate on cooking and make the dishes to the extreme, a chef is invited to be responsible for cutting vegetables, preparing vegetables and preparing materials. The relationship between the chef and the chef is distributed, and even a chef is too busy to come over, so the relationship between the two chefs is cluster. So distributed architectures may have clusters, but clustering does not mean distributed.
node
A node is an individual program that can independently complete a set of logic according to a distributed protocol. In a specific project, a node represents a process on an operating system.
A copy of the mechanism
Replication refers to providing redundancy for data or services in a distributed system.
Data copy refers to persisting the same copy of data on different nodes. When data on a node is lost, data can be read from the copy. Data copy is the only means of resulting data loss in distributed systems.
A service replica is a high availability scheme in which multiple nodes provide the same service through the master-slave relationship.
The middleware
Middleware bits and operating system to provide services, and does not belong to the application, it is a bit and application and system layer for developers to facilitate the processing of communications, input and output of a class of software, can let users care about their applications.
The evolution of the architecture
A mature large-scale website system architecture is not very perfect at the beginning of the design, nor is it the beginning with high performance, high availability, security and other features, but with the increase of users, the expansion of business functions slowly perfect evolution. In this process of development, the development model, technical architecture and so on will change very much.
If the system has the following functions:
User module: user registration and management
Commodity module: commodity display and management
Transaction module: create transaction and payment settlement
Phase 1: Single application architecture
The primary system is the application and database are placed on a server.
Phase 2: Application server and database server are separated
As the number of website users increases and the traffic increases, the application server and database server should be deployed separately. In this way, the system performance can be improved, the access efficiency can be improved, and the load capacity and disaster recovery ability of a single machine can be improved.
Phase 3: Application server cluster-Application server load alarm
As the number of visits and traffic increases, application server clusters are used to stream requests and improve application performance, assuming that the database does not encounter bottlenecks. Existing problems: Who forwards user requests and how the session is managed.
Stage 4: Database pressure increases – database read/write separation
Read/write separation, so that future requests, query requests can read data from the library, write data can be in the main library, but there are several problems:
1. Data synchronization between master and slave databases: You can use the master-slave mode provided by mysql to implement master/slave replication
2. Select the corresponding data source: Use the third-party database middleware, for example, Mycat
Stage 5: Use search engines to relieve library reading stress
Do read the library database, often on the performance of the fuzzy query is not very good, especially for large Internet companies, want to search module is the core, it is can use a search engine, although can greatly improve the speed of query, but also brings some problems such as construction of the index.
Phase 6: Introduce a caching mechanism to relieve pressure on the database
For some hot data, redis and memcache can be used as the cache of the application layer. In some scenarios, mongodb can be used instead of a relational database for storage.
Phase 7: Horizontal/vertical split of the database
Vertical split: Split different service data into different databases.
Horizontal split: Split the data in the same table into two or more databases. The reason for horizontal split is that the volume of data in some services has reached the bottleneck of a single database. In this case, you can split the table into multiple databases.
Phase 8: Application splitting
As the business grows, there are more and more businesses, and the pressure on the application becomes more and more. Projects are getting bigger and bigger. At this point we can consider breaking up the application, breaking our users, goods, and transactions into subsystems based on the domain model.
After such a split, there might be some of the same code, such as user actions, transaction queries for goods, so that each system will have user queries and operations related to access. The same code and modules must be abstracted. This facilitates maintenance and management.
After services are split, communication between services can be achieved through RPC technology. Typical examples are WebService, Hession, HTTP, and RMI.