Original: GuoJin Baidu APP technical team – senior technical experts

Componentization is a commonplace and wide-ranging topic. It is not to do one thing well, but to do a series of things well. It includes various architecture levels, construction systems, dependency management systems, and supporting anti-degradation mechanisms and rules and specifications, including the componentization framework. Based on the background, goals and componentization process of Baidu App, this paper mainly describes the means to ensure parallel development and component reuse, and tries to avoid diverging too much into the specific sub-directions such as building system, dependency management system and componentization framework. The importance of componentization depends on application scale, team size, and product technical goals; While this is all about iOS, the methodology and implementation path apply to most platforms.

Dry: making

Background and Objectives

Source of Baidu App(large App) complexity

  • Large business scale: baidu App technical direction and sub-direction 70+, single-end code 180W +; Objective: Isolate impacts among components to avoid fault spread, and control overall App complexity;
  • Large team size: hundreds of people with code permissions; Objective: To ensure efficient parallel development;
  • The company has many internal access services: more than 30+, which are not only basic databases, but also have a complex relationship with Baidu App. Objective: To deal with the relationship between access business and Baidu App architecture and its components, and ensure fast and efficient access and reuse of basic capabilities.
  • Fast iteration speed: 3 weeks for a version, 2 weeks for development and 1 week for testing; Goal: Avoid degradation of componentization in the case of high-speed iterations.
  • H5, NA, Hybrid, Talos and Flutter coexist. Objective: To ensure the reuse of basic capabilities and build system support.

In addition, starting speed, volume and other constraints of access process; And the diversity of goals is also a source of complexity for large apps; Goals generated by the background are inherent technical needs. In addition, Baidu App has different product technical goals in different stages. Different goals at different stages of Baidu’s App

  • Third-party database reuse of cooperative business; A single technical component output (the earliest demand, 2014), for a single component output, how to avoid the output, pull out the turnip to bring out the “mud “;
  • Matrix product incubation (2017 ~2019);
  • Open reuse of small programs (2018) : Output components are compatible with different hosts, keeping some dependent components fungible; Objective diversity requires that the requirements of each objective be taken into account during development, and that the solution be designed to avoid conflicts with these objectives.

Important Architecture iterations

Initial state -2013(Drill wood to make Fire) : During this period, Baidu App browser played a heavy role, everyone was in the same project development, each business and the basic logic interlocked, there is no boundary, you are in each other; UI architecture is relatively complex, each RD should start from the main entrance of the App to understand the main process code, careful development. The architecture of this period is as follows:

The main problems of this period were:

  • Some basic libraries, even open source third-party libraries will have business intrusion; No clear stratification and anti-modification mechanism, the invasion cost is very low;
  • There is no container isolation between the first screen business, touch the whole body, very easy to affect each other;
  • There is no service componentization for the shared services (remote configuration, terminal capability), if else/ Switch case logic infinite spread;
  • Logic and resources are not properly owned, data is not split, and it is difficult to output basic components.
  • The plug-in interface layer is not systematized and has poor stability (Fragile). The access service has become a super module in Baidu’s App, and the dependency relationship is difficult to control.

2014-2015(Steam Engine Age) :

Although the team size was only a few dozen people at that time, the importance of componentization was realized. Access services gradually become more and more, at the same time, there are also some technical components of external output needs; This stage:

  • Firstly, the three-party library is removed, and the coarser-grained basic library is removed to the lower layer of business components. Baidu App and access service reuse this part of the basic library;
  • A frame container is introduced to isolate the services on the first screen and the services in the stack navigation container.
  • For emerging business components or businesses that need to be reconstructed, the componentization mode is first adopted to develop, and the logic, resources and data are owned separately, and the external dependence is clarified.
  • The dependency specification is preliminarily formulated to prohibit the hierarchical reverse dependency. This stage is just the specification without the mandatory support of the tool chain.
  • A component’s dependencies beyond the base library are implemented through Adapter injection.

The architecture of this period is as follows:

The main problems of this period were:

  • The fuzzy attribution of components, some components are separated between the basic library and business components, and the dependence and calling relationship between the components of the same layer is not clear enough;
  • The components are decoupled one to one by Adapter. Although there is a clear external dependency, the decoupling efficiency is not high.
  • The main App also has a legacy side capability interface and some SDKS that are accessed through the plug-in system.

During the period of 2016-2017 (Power Era), the construction of modular framework (Pyramid, SchemeRouter), distribution framework (RemoteConfig, PMS, prefetch distribution) and data separation framework (CocoaSetting) was emphasized. Further ensure that each component can do logic, data have their own attribution;

The architecture of this period is as follows:

image

2018-2019(Ideal – Nuclear Age)

During this period, the componentization framework is relatively perfect, and each component has its own logic, resources and data. The main project is further weakened;

  • The hierarchy is more clear and clear, and the common services between the free layer and the basic library layer and the business component layer have attribution; Components can be exported from the bottom up;
  • The entire App is assembled via EasyBox via Central Repo Specs in a Central warehouse.
  • Framework container loading and system event distribution are unified into lightweight AppLauncher;
  • The access SDK belongs to the architecture hierarchy attribute; If it is referenced by only one business component, this business component is responsible for management, reducing external complexity.
  • The service layer can share the service relatively perfect.

In addition to cloud integration and reuse, other higher requirements are also put forward for componentization. Shared component library + building system (EasyBox) together, has been able to achieve the output capability of matrix product portfolio.

image

Componentization implementation path

Component construction from bottom to top

1. Build compilation isolation, architecture hierarchy and hierarchical access restriction mechanism

  • Compilation isolation: Through the construction system (EasyBox) provided by the compilation rules file to clarify the external interface of each component, clear external dependencies of components (in this respect, IDE often do bad things, so that components can be low-cost access, gradually blurred the logical boundary of components, deepen the dependency between components);
  • Interlayer restriction: Establish reverse access restriction through EasyBox, that is, the lower component can not access the upper component;
  • Same layer access: there is no unlimited call between the same layer components. Communication and access restriction are accomplished through the componentized Pyramid framework. Maintain clear interface and logical boundaries between components.

2. Standardization of tripartite database and systematization of basic database

1) The basic library mainly has the following problems:

  • No anti-modification mechanism, low cost of business intrusion;
  • Cross dependencies: the same underlying dependencies logically belong to the same component;

The basic library should be abstracted to the bottom layer of the architecture to a certain extent under the condition of no business intrusion, and the component responsible person system should be implemented in binary, and the system construction should be carried out to avoid the above problems.

2) The three party database mainly has the following problems:

  • No anti-modification mechanism, low cost of business intrusion;
  • Under a certain user scale or business scale, the three-party library does have bugs, but Github push request is not timely or does not respond;

Update all third-party libraries to the latest release and binary to avoid business intrusion; The difference part of the clear modification point, through the run time separately patch; Be clear about this when exporting.

3. Establish runtime distribution and quarantine services

To avoid centralized processing of common logic and common data among components, establish containers and distribution mechanisms to distribute events, data, and logical calls.

image

Pyramid componentized framework:

  • Here we focus on the distribution role of the Pyramid framework. Pyramid distributes system events to subcomponents.
  • In addition, the componentization framework serves two additional functions: 1) Communication between the Pyramid framework components: One-to-one adapter decoupling is upgraded to one-to-many decoupling; 2) It changes the strong dependence between components into weak dependence, which makes the dependent components have some degree of substitutability when the technical components are exported;

The ability to:

  • SchemeRouter and SchemeHandler logic are separated. SchemeRouter belongs to the componentification framework of service layer and SchemeHandler belongs to each component.
  • As Scheme parameters are not clear enough, Scheme in Baidu App is mainly used for communication with H5, but less for page routing.

Configure distribution service: transform centralized parsing and invoking business processing logic into distribution mechanism, and finally upgrade to cloud control service; Data splitting service: Together with the configuration distribution service, data is split to each component for internal management. Resource/Prefetch distribution service: Establish the resource/prefetch distribution service. Frame container: Split UI data of each controller to each child controller and distribute events to each child controller through Tab navigation container and stack navigation container;

Distribution and isolation mechanism, container mechanism is an important guarantee of parallel development.

4. At the service layer, low-dependent components with multi-business invocation are established to be abstracted into general services, such as account, sharing, cloud control, statistics, performance, AI, etc. 5. Component model is established to rapidly componentize each business module.

image

  • Generally speaking, it is to guide each business module to define its functional scope, so that logic, resources, data and private SDK have their own ownership.
  • Ultimately, each component is an independent functional unit, logic unit, data and resource management unit, H5 communication unit, performance quantization unit, and compilation output unit (one or more).
  • In order to make the composite output more flexible, the interface encapsulation layer and service docking layer of the component can be divided into different granularity compilation units. The main purpose is to separate dependencies and meet the output flexibility;

6. Business componentization According to the component model, the functional scope, logic and interface boundary of the business can be determined for fast componentization. 7. The interface change, dependency change and Warning number change of the deterioration control component will be recorded and notified to the relevant responsible person, which will be managed in Tekes platform; Please continue to pay attention to the following public number article; There is no anti-deterioration mechanism, filling pit speed is never as fast as digging pit speed; One person can never fill a hole as fast as many people can dig a hole.

Income summary

1. The improvement of R&D efficiency is mainly reflected in the following aspects:

  • Complexity control: Complexity control is internal to components, external “simple and reliable”;
  • Parallel development: The componentized framework and distribution services are isolated at design time to ensure the efficiency of large-scale parallel development. For example, in remote configuration, a new development time of 4+ hours is now only 0.5 hours; Efficiency increased by 8 times +;
  • Reuse: output wheels for matrix products; Referring to Baidu App matrix products, the reuse rate is above 50%;
  • Improved compilation speed: Because components have independent compilation unit attributes, component source code and compilation products can be equivalent replacement in the compilation process, so componentization also lays the foundation for the binarization of subsequent components, baidu App compilation speed is also optimized from 15 minutes per time to 2 minutes per time on average;

2. In terms of quality, componentization can be isolated during design to ensure that the impact of a single component’s failure can be contained within its own scope and will not cause the overall crash; 3. Provide quantization units for starting speed, volume and other directions;

4. Establish and improve the architectural system, and optimize the components in depth.

In the Light of Civilization, Dr. Wu Jun commented on the Contribution of the Greeks to world civilization: System of many modern natural science are lay in the ancient Greek times, the greeks in the academic research is different from the eastern civilization is not one or two scientific inventions and discoveries, but that they will be natural science interdisciplinary classify, for each discipline is to establish a set of system, on this basis, the deductive or inductive out common regularity, namely theorem and law, Then it became the cornerstone and pillar of all disciplines of natural science.

Although there is no such height, but for software development, there is a similar function.

conclusion

To conclude with a quote from “Every architect Should Study Conway’s Law.”

The goal of architecture is to manage complexity, variability, and uncertainty to ensure that changes to one part of the architecture do not adversely affect other parts of the architecture unnecessarily over the long term of system evolution. This ensures agile business and r&d efficiency, allowing the volatile parts of the application to change frequently with as little impact as possible on the rest of the application.

Earned on: Github