The beginning of the story is always like this. Today I hum “also on the beach at dusk, there are footprints in two halves……” Walking in the seaside, the face came to a man wearing gold armor, to the seaside still wear so fancy, is really a silly X. Take a look. Isn’t that Gavin?
Background: Gavin IV, the son of the Demesian emperor, was a famous rich and handsome man. With Galen, chrysanthemum letter and called the grass three musketeers, all day Shouting, “I debang, although far will be executed.”
Gavin: I started my own bank with the support of my dad, and I have a problem that’s been bothering me lately: how can I make my banking work fast and steady?
Q: inner OS: I depend, tuhao is tuhao ah, all opened the bank. (Pretending to be calm) What’s your current pattern? Can you talk a little bit about how your model changed?
The good old days of standalone MySQL
Scarlett:I first set the bank in a suburb with great development potential. I admire myself for the way things have gone. There were not many people here at the beginning, so I hired a teller to handle the simple business of depositing and withdrawing money.
Narrator: Seeing this single machine mode, I believe everyone feels very friendly, there was a time when everyone started from this single machine project. At that time, the project was relatively small and the business logic was relatively simple. In order to realize the business system and verify the market as soon as possible, all the business data would be stored in the same database.
Single machine + Cache
Q: Then you can’t handle a few people’s business this day, and you can’t receive dozens of people if you catch up with them on the weekend.
Narrator: With the increase of visits, almost all sites using MySQL architecture began to have performance problems in the database. Web applications no longer focus on function, but also pursue performance.
Gavin: Yeah, so I went to other banks, and almost every bank has a cute little sister there. Bah, bah, bah, is to help customers with business, so I hired a lobby manager to handle those small and simple business.
Narrator: In order to relieve the pressure of user access as soon as possible, on the basis of optimizing the structure and index of the database, a cache layer will be added between the application server and the database server to offset part of the database query operation.
Primary/secondary replication: Read/write separation
Ah Q: Well, that does solve part of the problem. Hey, I heard you got the subway last year? This will increase the flow of people, so that, plus the lobby manager should not be enough.
Narrator: Adding the database cache layer can only relieve the database reading pressure and block some database access requests. However, with the further growth of user visits, centralized reading and writing on one database makes the database overwhelmed, and the bottleneck of database access becomes more prominent. At this point, the architecture of the data layer has to be changed.
Gavin: Well, luckily I was smart enough to hire two more people and optimize the window: A for deposits and BC for withdrawals. In this way, deposits and withdrawals are separated and processing efficiency is increased.
Narrator: We would enable multiple database instances and divide the database into master and slave: the master is responsible for writing data, also known as the write library; The slave library is responsible for reading data, also known as the read library, which can have more than one.
The synchronization mechanism is used to synchronize data from the master database to the slave database. If you need to query the latest data, you can write more data to the cache to obtain the latest data.
The master and slave libraries can be deployed on the same server, but to improve performance, it is best to deploy them on different servers when resources are sufficient.
Vertically split service data
Ah Q: good good, I didn’t think you still have this move, let me sit up and take notice. So you don’t have any other business except deposit and withdrawal business?
Gavin: I’m sure there will be. I can’t have them hanging around talking. At the end of last year, I added the fund business, thinking to cultivate these young people to establish a sense of financial management, I am not too selfless. But since the addition of these services, there are not enough people. Ah!
Narrator: As we used the master-slave database architecture, we found that we could support more user visits and requests. However, with the further development of business, such as the increase of commodity inventory system in the e-commerce system, it will seize database resources with the original order system, mutual impact on performance, resulting in the further increase of pressure on the database.
Q: I heard you didn’t earn less last year, how did you solve it?
Scarlett:(Smug) I hired a senior executive from outside and told him and his team to focus on funds and other wealth management. My existing team was still in the traditional business. So they can work together without affecting each other.
Narrator: In order to alleviate the database pressure caused by business expansion, we can split the database vertically according to business: separate the database of the order system from that of the commodity inventory system, reduce the resource competition between businesses, and use a unique database for storage.
The level of table
Ah Q: Now that everyone has achieved a well-off society in an all-round way and people have money in their hands, are there more people saving money now?
Gavin: Well, Um, I’ve expanded my deposit desk a few months ago by hiring more college students.
Narrator: Take e-commerce as an example, with the increase of user transaction volume, single order table can no longer meet the requirements of storage. At this point, we can split a table into multiple tables for storage, adopt certain strategies for horizontal expansion, distribute requests as evenly as possible to each small table on the server, and further improve the concurrency.
Cluster deployment
Q: I heard from Galen that your community has a high occupancy rate, the flow of people has become larger, and the business has also made rapid progress.
Scarlett:Ha ha, that’s what I’m most proud of, opening another bank. Copy the original model, and make another set, so that there is one east and one west don’t have to worry about busy.
Narrator: As the volume of data continues to soar, we can use clustering to reduce the pressure of access. However, the performance of cluster can not well meet the requirements of the Internet, but provides a very large guarantee of high reliability.
NoSQL database and search engine
Ah Q: I go, too ox old iron, that you return the stem what of sad face, listen to you so say, should be sad face of be me.
Gavin: You don’t know. The mall opened here a few days ago, and it radiated to the neighborhood, attracting a lot of people from the neighborhood. There are so many deposits and withdrawals that I can’t afford to open another one. The cost to benefit ratio of opening another one is too low to be worth it.
Ah Q: Ao ao, I see. I think you can plan something like this: put a few ATMs in both the eastern and western banks, so that they don’t go over the counter for small deposits and withdrawals; Rent a store next to the new mall, throw in a few ATMs, and reduce labor costs, so you can satisfy the needs of mall shoppers.
Narrator: When the amount and diversity of data in our database reached a certain scale, MySQL alone could no longer meet our needs, so other data storage methods were introduced. Taking e-commerce as an example, we can roughly classify the information:
- The basic information of the product is saved to
MySQL,OracleRelational database; - Description, details, evaluation and other information of the goods may be adopted
MongDBSuch as document database to store, can improveIORead and write performance; - A distributed file system is used for product images
HDFS; - Product keywords can be used by search engines
ES,Solr; - In-memory database can be used for the high frequency information of commodity’s wave band hot spot
Redis,Tair; - Third party interfaces or payment systems can be used for commodity transactions, price calculation and points accumulation.
Gavin: That’s a good point. Specialize so I can get a good night’s sleep. Let’s go! Let’s take you out!
Ah Q: Yes!
The above story is pure fiction, just to show you the evolution of database architecture. Ah Q did not study the real banking system and how to deal with the business, but made up the story to increase people’s understanding.
If you have different ideas or better ideas, please feel free to contact Ah Q: follow GZH “Ah Q said code”, or add your friend Qingqing-4132, Ah Q is looking forward to your arrival!