Recall from the company was set up on the first line of code is present and nearly three years, platform technology architecture, technical system is experienced four major upgrade transformation (currently fourth generation architecture system is under way), also want to spare some time to review, a small company from the beginning of zero trading volume over the technology behind the billions change up to now.
General introduction
In the financial industry, the Internet is not more than $one hundred in fact big platform, which is the secondary camp, actually every time the architecture of the upgrade is accompanying with business progress, problems on the previous generation of system architecture, the business development process to accumulate some excellent development case, in the next generation of systems development will promote the structure upgrade. On the one hand, it can smooth the transition; on the other hand, the company’s resources can provide strong support; at the same time, technical partners can use cutting-edge technology and have a sense of achievement in development. Thus, we upgrade the system architecture once every nine months, and we have reached our current architecture.
, many netizens often ask you platform TPS is how much ah, what is the maximum concurrent ah, how about performance, to be honest we are a small company, the most exaggerated is tens of thousands of people bid at the same time, but as a medium of the Internet financial platform to do also really many, is more than just these parameters can say clear; What we’re not big on the platform, the use of technology is the current mainstream open source products, but with the constant development of the company also met a lot of problems in the process, also try to use more mainstream, open source, suitable for some of our solution to build the entire system, here share platform development behind the change of technology and upgrading, At the same time, I hope to make more exchanges with you and give more suggestions.
We made four major architectural changes, and each generation is summed up in one sentence:
Features of the first-generation architecture: the services are concentrated, the functions meet the needs of investment and financial planning, and the services are quickly launched
Second-generation architecture features; Distributed system transformation, platform taking shape, various vertical business systems set up online, product end greatly enrich user investment, big data platform research and use
Third generation architecture features; SOA governance, using ZooKeeper as the registry and Dubbo as the monitoring and scheduling center; Cas implements single sign-on, using Shiro for permission control
Fourth-generation architecture features; Fully enable micro service development mode, Springboot + SpringCloud technology stack as the fourth generation architecture technology support
The following is a detailed introduction
First generation system architecture
The year 2014 should be regarded as the first year of Internet finance. Before that, there were a lot of Internet companies using various modes to survive, which had been lukewarm. However, it suddenly became popular in 2014. Later, there were more and more reports that various Internet financial companies received XXX dollars of investment, and the policies were gradually clarified. Therefore, many large companies (groups) took advantage of this enthusiasm to follow up, including us.
The first generation of the system is the most important to fight against time, the company hoped to ensure the system in the shortest time to launch, at that time the mobile wave has been started, so it decided to give priority to the launch of mobile terminal, website can temporarily not consider. At that time, the company had PHP and Java development language technology reserves, because PHP has a great advantage in rapid development, so it decided to adopt the model of front-end PHP+ back-end Java. The system is divided into three layers: user layer: Android and IOS mobile terminal; Interface layer: PHP provides user and transaction interfaces; Back end: The back end has two parts, background and timing system. Background PHP development and interface layer of a common system, the other is a timing system, responsible for interest, dividend, due and other scheduled tasks using Java development.
For basic services and middleware, mysql does the most basic master/slave support. The first generation system only uses the master library of mysql, and the slave library is only synchronized backup. Memcached is used to handle the concurrent issue of users grabbing tokens, and this is the only one used; ActiveMQ is used to use secondary market transfer matchmaking and other asynchronous message notifications. Project deployment: PHP using Apache deployment, timing service using Tomcat6 to do the application server, using LVS to do the front-end Apache load, basically the first generation of these technologies, the following is the architecture diagram of the first generation system.
After the first generation system online, the website and H5 (mobile browser or WeChat side) system construction is particularly prominent, as an Internet financial company without official website can’t endure, and so on started to develop its web site and H5 system, in the meantime do before the background of the PHP picked out, using Java redesigning a version, So far, PHP is responsible for the three systems of website, APP interface and H5. The three systems share a core transaction, while Java is responsible for background management and timing service. We generally call this architecture 1.1 generation architecture.
The 1.1 generation system architecture diagram, the green part is the changed part
The disadvantage of the first generation system is that the business is too centralized and rushed online, and there are many problems in the later period
Second generation system architecture
The background of the second-generation system is that with the rapid development of the company’s business volume, a lot of technical debts in the early stage broke out, and many problems appeared online. The most serious one was repeated dividend payment to individual users, which was scolded by all kinds of people. On the other side, there is constant demand from various business units and the company’s products, so this phase is busy fixing various production problems while developing vertical business systems. During that time, I was almost driven crazy. The first generation system was closed development, and before I came back, I rushed to put it on the shelf immediately. It was really painful and happy.
The first vertical subsystem came online: the contract system. At that time, there was no contract after users bid, so many users were very worried and put priority to the first one. Later, the contract system changed three versions, the first version only generated PDF, the second stage online electronic signature, the third stage watermark, custom dynamic generation OF PDF; Then the development of points system: user invitation, investment and other production points, used to exchange cash roll, etc.; Pull out the message system: station messages, SMS, mail, etc. On-line monitoring system, business monitoring and service monitoring, business failure warning; All business departments continue to demand, online financial system: financial personnel statistics and calculation amount; Risk control system: monitoring abnormal users and abnormal transactions; Developed sales system for sales; Because of docking with a lot of third party systems, and developed external access system.
The first generation system is very rushed, and the product interface is very bad, then started planning website 2.0, APP2.0, H52.0, according to the requirements of the front-end system, the back-end development of CMS system to release projects, the company’s announcement news, etc. The second generation product side common planning a lot of big data analysis some of the requirements, in the website to show the whole amount after data analysis, the amount of investment investment preference, all ran to where go to, front end use maps to show, for personal calendar will have the repayment, the collecting data analysis, etc., because of the need to run the whole amount of data, at the time of planning is offline to process design, Synchronize data from mysql library to mongodb cluster, and use Mongdo’s MapReduce technology to process a large amount of data, so our database layer becomes the following architecture
Mysql real-time synchronization to the mongo, we use the tungsten – relicator (https://github.com/vmware/tungsten-replicator) this tool, can start the mysql server a monitoring agent, At the same time, a server is also set up on the server side of mongodb. Agent monitors data changes and sends them to the server side, which then inserts them into the mongodb cluster to achieve real-time synchronization. As shown in the figure above, I wrote an article to introduce: Big data practice – Data synchronization tungsten relicator (mysql-> Mongo), in fact, this tool is not particularly stable in use, but there are not many options at the beginning, fortunately, it is stable after slowly getting familiar with later.
We boldly used Golang to develop the data cleaning system. Golang version was 1.3 at that time, but now it is 1.8. We had no contact with golang before and trained our team, but golang language itself is very simple and efficient. Later, golang was used to develop a background, which was based on the Beego framework. The big data analysis system was later upgraded. In each front-end business system, UI user layer made a lot of buried points to collect user data, which was transmitted and received by activeMQ and stored in mongodb. After data cleaning, the cleaned results were stored in the result library for use by front-end business system. Later, beego+ Echart was used to make a new version of the data analysis system.
Architecture diagram of big data system
Due to the increasing pressure of back-end database, back-end management system and business system are separated from each other. Background management system to increase the cache, started redis cache; Using Nginx to build an independent picture server; During the development of the second-generation system, which was also the fastest development stage of the company, more than N activities were launched.
Second-generation System Architecture Diagram:
In the second generation architecture, each business system is put on line, the master and slave are separated, and the big data platform is built, which provides the technical foundation for more data processing in the future. Disadvantages: After the business system is divided, the call between each project is complicated; There are various background systems and separate account systems between them. Operation needs to switch back and forth to complete platform operation monitoring
Third generation system architecture
After the completion of the second generation of system development, we were left with three painful problems. The first one was that with the continuous increase of business systems, the call relationship between systems increased exponentially. At the beginning of the third generation of system, we developed many basic components, which exacerbated this problem. The second problem is complementary to the first problem. There are too many calls between systems. If one subsystem is moved, it may be necessary to modify the configuration file of the associated system and restart the service. The third problem is that we have developed a lot of background systems, but the accounts are not unified. Each subsystem has its own account center. Operation and business personnel need to log in back and forth to complete daily work.
The first problem was to introduce SOA service governance and solve the decoupling between the systems through the registration and discovery of services. After a lot of investigation, dubbo was finally selected. The reason was that a large number of people used the foundation and the water had been crossed. The second problem was to introduce the configuration center. After investigating Qihoo360/QConf of 360, Spring-Cloud-config of Spring, Diamond of Taobao, and Disconf of Baidu, I finally chose Disconf after a long period of deliberation. But it was from here that we noticed that Spring-Cloud and Spring-Boot left a foreshadowing for the selection of the fourth-generation architecture. In fact, disconf was only used in a few projects at last, and was not fully promoted. The third problem is the account center, which uses CAS to implement single sign-on, Shiro to control permissions, and Dubbo to provide server-side interfaces such as the list of permissions after login.
Architecture diagram after transformation
The comOMn component handles common basic classes, including character classes, date classes, and encryption classes…. , set up fastDFS cluster to deal with the file system, did the redis cluster test; A separate scheduling system is developed to integrate all scheduling tasks into the scheduling system. Scheduling policies can be automatically added to the page when scheduling tasks are needed in that system. The front-end PHP system has been reformed, including separation of master and slave, static optimization and so on
Later, the company started the construction of the crowdfunding platform. This time, the system was completely developed in Java language, and the app side adopted the mixed development mode, in which all the first-level pages of the APP adopted the native development, all the second-level pages adopted the H5+ Vue mode, and the back-end all adopted dubbo for servitization. The final architecture was as follows:
Only part of the system is listed in the diagram, and other services are used instead
The third generation system started SOA service governance, introduced a unified account center, basic components; The disadvantage is that the development environment is more complex
Fourth generation system architecture
People are always not satisfied, and technology always wants to use the best architecture system. In the development of the third-generation system architecture, I learned about Spring Cloud and Spring Boot. After continuous learning, I am more and more aware of the convenience of Springboot and the advantages of rapid development. Spring Cloud system also fully meets all aspects that a large system needs to consider. The concept of micro-service is constantly put forward. The above is the technical background. On the other hand, the country began to strictly require P2P companies to access the bank depository. After analyzing the relevant interfaces of banks, we found that if we strictly follow the rules, our system needs to be greatly reformed. Meanwhile, in order to meet the regulatory requirements, the company also developed ious related products, which is also a big project. We decided to embrace microservices as well as bank deposits and IOUS.
There are three reasons why we should ditch Dubbo and embrace Spring Cloud. 1. Dubbo hasn’t been updated in years, and Spring Cloud is constantly updated. 2. Dubbo is mainly engaged in service governance and monitoring. Spring Cloud considers almost all aspects required by micro-services, such as unified configuration center and routing center. 3. Spring Cloud is more invastion-free and perfectly integrated with other Spring projects, resulting in higher development efficiency.
Since spring Boot + Spring Cloud has been selected for the transformation, the selection of micro-service technology has been decided. Then how to start the transformation? After all, the transformation of the new generation system cannot affect the original business, and the main problem is that although the initial system is carried out according to the distributed development mode, Due to the old system, some systems still share a database. Microservice requires each independent subsystem to have its own independent library operation. If other systems need to modify or query the data of the subsystem, they need to obtain it according to the service indirect call. Therefore, we plan to start the SpringCloud project from the newly developed project and the project to be transformed, and other systems will communicate through the router mode for the time being. The final system architecture diagram is as follows:
There is no end to the architecture. Change is constant, and the architecture is upgraded to better support the business. The two complement each other.
Sixty cents for your support
Open source software
We also wanted to contribute a little bit to the open source world over the years, opening two pieces of software altogether:
generator-web
We used Mybaits extensively in the project. We reformed the generator of MyBaits and made a system interface to facilitate automatic production of relevant codes according to relevant parameters (the system will automatically generate codes of Mappper, Entity and DAO layers only after the table structure is designed). Finally, we opened source the system
generator-web：https://github.com/zhongxintech/generator-web
Collection of cloud
In order to exercise technology learning Springboot we developed a cloud collection of open source software, using the technology is mainly Springboot/Spring data JPA/Redis/Thymeleaf/gradle, The main function is to help users collect, share and organize their favorites in the cloud.
favorites-web：https://github.com/cloudfavorites/favorites-web
Feel free to leave a comment and talk about the evolution of architecture.
