Original: GuoJin Baidu APP Technology team – senior technical expert article source: Baidu APP technology wechat official account

Componentization is an old platitude involving a wide range of topics, that is, not to do one thing well but to do a series of things to achieve; It includes various architecture levels including componentization framework, construction system, dependency management system, and supporting anti-degradation mechanism and rules and specifications.

This paper mainly describes the means to ensure parallel development and component reuse based on baidu App background, objectives and componentization process, and tries to avoid too much divergence into specific sub-directions such as construction system, dependency management system and componentization framework. The importance of componentization depends on application scale, team size and product technical goals. Although the content is started on iOS, the methodology and implementation path can be applied to most platforms.

Background and Objectives

Complexity sources of Baidu App(large App)

  • Large scale of business: Baidu App technology direction and sub-direction 70+, the number of single-end code 180W +; Objective: Isolate the influence between components to avoid fault spread and control the complexity of the overall App;
  • Large team size: hundreds of people with access to code; Objective: To ensure efficient parallel development;
  • The company’s internal access business: 30+, not simple basic library, complicated relationship with Baidu App; Objective: To deal with the relationship between access business and Baidu App architecture and its components, and ensure fast and efficient access and basic capability reuse.
  • Fast iteration speed: 3 weeks for a version, 2 weeks for development and 1 week for testing; Objective: Avoid componentization degradation in high-speed iteration.
  • There are many technical forms: H5, NA, Hybrid, Talos and Flutter coexist. Objective: To ensure the reuse of basic capabilities and build system support.

In addition, start speed, volume and other access process constraints; And the diversity of goals is also a source of large App complexity; The goals generated by the background are innate technical requirements. In addition, Baidu App has different product technical goals at different stages.

Different goals for different stages of Baidu App

  • Tripartite library reuse of cooperative business; Output of single technical component (earliest requirement, 2014). For output of single component, how to avoid pulling out radish and bringing out “mud “;
  • Matrix product incubation (2017 ~2019);
  • Open source reuse of small programs (2018) : Output components compatible with different hosts, keeping part of the dependent components fungible; Diversity of goals requires that the demands of each goal should be taken into account during development, and conflicts with these goals should be avoided as far as possible during project design.

Critical Architecture iterations

Initial state -2013(Drill wood for fire) :

In this period, Baidu App browser played a more important role, we all developed in the same project, each business and basic logic interlaced, no boundary, there are you in me, there are you in me; UI architecture is relatively complex, so each RD needs to understand the main process code from the main entrance of App and carefully develop it. The structure of this period is as follows:

The main problems of this period are:

  • Some basic libraries, even open source tripartite libraries will have business intrusion; There is no clear stratification and anti-modification mechanism, and the invasion cost is very low.
  • There is no container isolation between the businesses on the first screen, so it is very easy to affect each other.
  • There is no service componentization for services shared by all services (remote configuration, terminal capability), and if else/ Switch case logic spreads indefinitely;
  • Logic and resources are not properly assigned, data is not split, and the external output of basic components is difficult.
  • There is no systematic construction of plug-in interface layer and the stability is fragile. Access business has become a super module in Baidu App, and the dependency relationship is difficult to control.

2014-2015(Steam Engine Era) :

Although the team was only a few dozen people, the importance of componentization was already recognized; There are more and more access services, and some technical components need to be exported. This stage:

  • First of all, the tripartite library is removed, and the coarse-grained basic library is removed to the lower level of business components. Baidu App and access services reuse this part of the base library;
  • The framework container is introduced to isolate the services in the first screen and the stack navigation container.
  • For emerging business components or businesses that need to be reconstructed, componentized development mode is adopted first, with logic, resources and data belonging to each other, and external dependence is clarified.
  • The dependency specification is preliminarily formulated to prohibit reverse dependency of hierarchy. This stage is just a specification without the mandatory support of tool chain.
  • Dependencies of components other than the base library are implemented through Adapter injection.

The structure of this period is as follows:

  • Fuzzy component ownership, some components are separated between the basic library and business components, the dependency and call relationship between components of the same layer is not clear;
  • The components are decoupled one to one through Adapter. Although there is a relatively clear external dependency relationship, the decoupling efficiency is not high.
  • In the main App, there are still terminal capability interfaces, and some SDKS accessed through the plug-in system.

2016-2017 (Electric Power Era)

In this period, the emphasis was on the construction of componentization frameworks (Pyramid, SchemeRouter), distribution frameworks (RemoteConfig, PMS, pre-requisition distribution), and data separation frameworks (CocoaSetting). To further ensure that each component can achieve logic, data belonging;

The structure of this period is as follows:

2018-2019(Ideal State – Nuclear Energy Era)

The main project was further weakened

  • The hierarchy is clearer, and common services are assigned between the base library layer and the business component layer. Components can be exported from the bottom up;
  • The entire App is assembled by EasyBox from the Central Repo Specs list in the Central warehouse;
  • Framework container loading and system event distribution are unified into lightweight AppLauncher;
  • Access SDK, according to the architecture level attribute attribution; If it is referenced only by a service component, the service component manages it to reduce external complexity.
  • Service layer shareable services are relatively complete.

The Advanced stage of componentization – Mediaitaization (Starcraft)

The tide of centralization is rolling in. Besides cloud integration reuse, componentization also puts forward other higher requirements. The combination of shared component library and EasyBox can reach the output capacity of matrix product portfolio.

Componentization implementation path

Construction of componentization from bottom to top 1. Establishment of compilation isolation, architecture hierarchy and hierarchical access restriction mechanism

  • Compiler isolation: by building the system (EasyBox) to provide the compiler rules file to clarify the external interface of each component, clear the external dependency of components (this aspect IDE often do bad things, so that components can be accessed at low cost, gradually blurred the logical boundary of components, deepen the dependency between components);
  • Interlayer restriction: Reverse access restriction is established by building system (EasyBox), that is, the lower component cannot access the upper component;
  • Hierarchical access: There are no unlimited calls between components of the same layer. Communication and access restrictions are implemented using the componentized Pyramid framework. Clear interface and logical boundaries are maintained between components.

2. Standardization of tripartite libraries and systematization of basic libraries 1) Basic libraries mainly have the following problems:

  • No anti-modification mechanism, low cost of service intrusion;
  • Cross dependencies: logic belonging to the same underlying dependency into the same component;

The basic library should be abstracted to the bottom of the architecture to a certain extent without business intrusion, implement the component responsible system in binary, and carry out systematic construction to avoid the above problems.

2) The tripartite library mainly has the following problems:

  • No anti-modification mechanism, low cost of service intrusion;
  • Under certain user scale or business scale, tripartite library does have bugs, and Github push request is late or does not respond.

Update all third-party libraries to the latest release and binary to avoid business intrusion; The modification point of the difference part is clear, and the patch is made separately at runtime. Make this clear when exporting.

3. Establish runtime distribution and isolation services to avoid centralized processing of common logic and common data by components, establish containers and distribution mechanisms to distribute events, data, and logical calls.

  • Pyramid distributes system events to child components.
  • In addition, a component-based framework serves two other purposes: 1)Pyramid communication between components: Adapter one-to-one decoupling is upgraded to one-to-many decoupling; 2) It changes the strong dependencies between components into weak dependencies, which makes the dependent components have some degree of substitutability when the technical components are exported;

The ability to:

  • Separate SchemeRouter and SchemeHandler logic. SchemeRouter belongs to the componentized framework of the service layer, and SchemeHandler belongs to each component.
  • As Scheme parameters are not clear enough, Scheme is mainly used for communication with H5 in Baidu App, but rarely for page routing.

Configure distribution service: transform the centralized analysis and invocation of business processing logic into distribution mechanism, and finally upgrade to cloud control service;

Data splitting service: Separates data into components for internal management together with the configuration and distribution service.

Resource/prefetch Distribution service: establish resource/prefetch distribution service;

Frame container: through Tab navigation container and stack navigation container, UI data of each controller is split to each sub-controller, and events are distributed to each sub-controller; Distribution and isolation mechanism and container mechanism are the important guarantee of parallel development.

4. In the service layer, low-dependency components of multi-business invocation are decommercialized and abstracted into general services: account, sharing, cloud control, statistics, performance, AI, etc

5. Establish component model Establish component model, and the business modules are rapidly componentized.

  • Generally speaking, it is to guide each business module to clarify the functional scope, so that logic, resources, data, private SDK have their own ownership;
  • Finally, each component is an independent functional unit, logical unit, data and resource management unit, H5 communication unit, performance quantification unit, compilation output unit (1 or more).
  • In order to combine output more flexibly, component interface encapsulation layer and service docking layer can be divided into different granularity compilation unit. The main purpose is to separate dependence and satisfy output flexibility;

6, business componentization According to the component model, determine the functional scope of business, logic and interface boundary, rapid componentization.

7. Interface change, dependency change and Warning number change of deterioration control component will be recorded and notified to the relevant person in charge, which will be managed on Tekes platform; Please continue to pay attention to the follow-up public article; There is no anti-deterioration mechanism, the speed of pit filling will never be as good as the speed of pit digging; One person can never fill a hole as fast as many people can dig a hole.

Income summary

1. The improvement of R&D efficiency is mainly reflected in the following aspects:

  • Complexity control: complexity control is inside the component, and externally “simple and reliable”;
  • Parallel development: componentized framework and distribution services are isolated at design time to ensure the efficiency of large-scale parallel development. Taking remote configuration as an example, the development time of a new item was 4+ hours, but now it only takes 0.5 hours. Efficiency increased by 8 times +;
  • Reuse: output wheels for matrix products; Referring to Baidu App matrix products, the reuse rate is above 50%;
  • Improved compilation speed: Because components have the attribute of independent compilation unit, source code and compilation products of components can be equivalently replaced in the compilation process, so componentization also lays a foundation for the binary of subsequent components. The average compilation speed of Baidu App is optimized from 15 minutes/time to 2 minutes/time.

2. In terms of quality, componentization can be isolated at design time to ensure that the scope of failure of a single component can be confined to itself without causing the whole crash; 3. Provide quantization unit for starting speed, volume and other directions; 4. Establish and improve the architecture system, and optimize the sub-components in depth. In the light of Civilization, Dr. Wu Jun evaluated the Contribution of The Greeks to world civilization as follows: System of many modern natural science are lay in the ancient Greek times, the greeks in the academic research is different from the eastern civilization is not one or two scientific inventions and discoveries, but that they will be natural science interdisciplinary classify, for each discipline is to establish a set of system, on this basis, the deductive or inductive out common regularity, namely theorem and law, Then it becomes the cornerstone and pillar of every discipline of natural science.

Although there is no such height, but for software development, also have the same effect.

conclusion

From “their” final shallow, cited a paragraph as a conclusion, is adapted from “every architect should research the conway’s law”, the goal of architecture is used to manage complexity, variability and uncertainty, the evolution process of the system to ensure that in the long run, the change of the part of the architecture not unnecessary negative impact on the rest of the architecture. This ensures that business and r&d efficiencies are agile and that one volatile part of the application can change frequently with as little impact on the rest of the application as possible.

reference

  • Every architect should study Conway’s Law: www.infoq.cn/article/eve…
  • Three Key Software all of You Must Understand: code.tutsplus.com/tutorials/3…