Snowball launched fund trading in early 2016 and developed an independent App — Egg Roll Fund (hereafter referred to as “Egg Roll”). In May 2016, the first white box strategy (strategy details, positioning algorithm fully open) portfolio was launched, which was unanimously recognized by the market and caused a storm in intelligent investment. By the beginning of 2018, Egg Roll supports the sale of more than 4,000 funds, including: money fund, securities asset management, bond, hybrid, stock, etc. At the same time, on the basis of the strategies provided by fund companies, nearly 20 white box strategies have been implemented, including rotation, allocation, life cycle, overseas assets, etc.

In two years, the egg roll fund trading system has grown from nothing, from using a third party to using a self-developed system, from single function to rich function, which has undergone a series of transformation, upgrade, reconstruction, and without the user’s perception, migrated to the new system, achieving a great leap forward.

| business introduction

The fund trading system has a lot in common with the e-commerce transactions we use daily. In the fund trading system, funds are merchantable products, and the user transaction process is similar to that of e-commerce. Users need to authenticate their identity, bind their cards, make payment, and then purchase relevant fund products. Back-end capital processing is similar, involving the clearing and transfer of funds.

Therefore, in the whole system, there are two core processes: one is the business flow, namely: transaction generation, buying and selling behavior; The other is the flow of capital, where money comes from and where it should go. The two core processes are independent from each other, but interrelated: the effectiveness of the business needs to be confirmed by capital, and the business needs to be annulled if the capital is not in place. The liquidation and transfer of funds shall be based on business flow.

Fund trading time is based on one T day (generally working days, 15:00 of the previous day to 15:00 of the following day is a T day)

Fund subscription business flow, as shown in the figure:

  1. The user initiated the purchase on T day, and the egg roll was deducted by the third-party payment

  2. T+1 fund companies will confirm whether the subscription is successful and confirm the fund share

Fund subscription fund flow:

  1. The user applies for purchase on the day T, and the funds on the day T+1 are transferred from the payment institution to the supervising bank of the egg roll

  2. On T+1, the bank supervising the egg roll transferred the money to the fund company

  3. In the capital flow, the supervising bank is responsible for capital supervision

The above is the business flow and capital flow of a typical fund subscription business. For a fund trading system, user application processing, third-party deduction and capital clearing are the core functions.

1.0 | trading system

Egg roll was the first to use a third-party fund trading system, and on it through API encapsulation and direct database call, as shown in the figure:

In the above architecture, the third-party trading system is a completely invisible black box to the omelet, and there are some problems that continue to plague us:

  1. The core system is out of control. As the most central part of the system, the omelet cannot control itself, which means huge risk. We can check the transaction data through the database, but we do not know the logical relationship of the whole system data, external docking, internal module relationship, etc.

  2. Daily problems are difficult to solve. Because the system needs daily maintenance, but because the egg roll does not know the core process and implementation, it needs the cooperation of a third party, which leads to problems in processing efficiency. Some problems (possibly bugs), due to the limitations of on-site personnel experience, although the impact exists for a long time, they continue to be unable to be solved, requiring third-party internal coordination.

  3. Upgrade/maintenance difficulties. The whole third-party system is built on the basis of the database, the business depends heavily on the database. Trading system upgrade means database needs to be upgraded, which means downtime!! Upgrade downtime is unacceptable for an Internet application. If the upgrade fails, the database will be rolled back, which is a huge risk.

  4. New feature development is slow. As the third party maintains the trading system, the transformation involving the trading needs the support of the third party. The long process of communication, development and testing of requirements is still unacceptable for a fast-iterating product.

Based on the above problems, Omelet decided to develop its own trading system.

2.0 | trading system

To develop your own trading system, there are two problems: first, is it feasible? Second, how to migrate existing users?

The whole fund trading system relies on a large number of external systems, such as third-party payment, regulatory bank and SHENzhentong (TA). These third-party systems are all production environments, and most of them do not have a test environment. Even if there is a test environment, they do not support the construction of specific test cases. In addition, in the fund trading system, the whole platform is interdependent. If the data of any fund sales agency is not uploaded on T day, the whole system needs to wait. Based on the above, if we want to develop a good trading system, and then at a point in time full online, is basically not feasible.

So there was only one option left for the omelette — direct production testing, complete function by function; Migrate users one by one, that is, a step-by-step migration scheme from 1 to N.

Using this scheme guarantees several things:

  1. The scope of the problem is manageable. With only one test user at the beginning, it’s easy to deal with data problems.

  2. Test cases can be constructed. With a production environment, the test time is longer, but the actual efficiency is much higher.

To achieve this solution, it means that there are two trading systems in parallel for a long time, so two key problems need to be solved:

  1. Users need to be routed to different systems based on user IDS. The problem is relatively simple.

  2. Because there are third-party systems on the outside, there are two systems inside the omelet, but only one can be seen outside. Fortunately, the external dependence of the entire fund trading system, most of which is based on file exchange, is easier to deal with this problem.

So the omelet made the following changes to the trading system:

  1. As shown by (1) in the following figure, the processing of routing according to UID is added, and different users are routed to different trading systems

  2. As shown by (2) in the following figure, a Proxy for file processing is added for external system interaction, which is responsible for merging and splitting external files

  3. As shown by (3) in the following figure, part of the management and configuration data remains in the original system, and information is directly synchronized through the database

Based on the above architecture, omelet spent 10 months to build its own trading system from scratch, and successfully realized user perception-free migration:

  1. At the beginning of the project, routing, file Proxy and several independent functional scenarios were implemented, such as subscription and redemption scenarios. We went straight into production, using a user test. Follow-up function side development, side test. In extreme cases, the user encounters a problem and manually modifies the file.

  2. Use caution when migrating users. Since the two systems are in parallel, there is no way to switch back to the third party system if the user makes a transaction after migrating from the third party system to the egg roll. In particular, it is important to note that migration is also risky when users have in-transit transactions or incomplete transactions (such as subscriptions). Because some data exists in an intermediate state, omelet migrates these users until there is no intermediate data at all.

  3. User migration is in parallel with the new functionality. The user migration period is relatively long, lasting about 3-4 months. In order to ensure the development pace, many new functions are directly implemented on the new system, but these functions are not available to users who have not migrated. In view of this situation, we have made a function configuration switch on the App side to ensure that migration and new function development can be parallel.

The core transaction module (related to transaction) of the fund is roughly as follows:

Compared with previous third-party systems, the use of our own system has brought a huge improvement in efficiency:

  1. New functions are released at any time, online at any time, there is no need to stop the problem

  2. A lot of automation. For example: fund business related documents import, business logic processing, without manual intervention

  3. Dynamic routing of payment link

  4. The system of a large number of reconciliation extraction, abstraction, modeling, simplify the repeated process

| pay link optimization

In the fund trading system, user subscription is a very important link, so egg roll is connected with a number of third-party payment. Different third-party payments can be supported by different banks, and the same bank may support different payment amounts and payment rates. On the basis of our own transaction system, we can combine the characteristics of different banks to realize the dynamic configuration of payment channels. The diagram below.

It combines manual configuration with automatic configuration. Manual configuration is used to recommend certain channels, individual banks, and some system maintenance processes. Auto-configuration is based on routing rules that allow automatic switching or service degradation in the event of a payment channel failure, as well as specific payments based on code (e.g., choosing the channel with the cheapest overall rate).

| summary

Reviewing the whole process of self-built trading system, it is the only feasible and safest solution to go online as soon as possible and migrate users one by one. Although it seems to go online without full functions, as long as the problems are controllable, limited and predictable, there is no problem. This approach also exposes the risk of going online as early as possible, although there will be more problems in the early stage, but the later will be more comfortable.

Finally, third party dependencies that are out of your control are very painful and should be removed as far as possible. If something needs to be done sooner or later, consider whether to do it now.

| there’s another thing

Snowball’s engineer team is recruiting Java engineers, operation and maintenance development engineers, test development engineers, algorithm engineers. Interested students can check the original text to see the specific positions and requirements, waiting for you.