1. The background

1.1 Introduction to mobile terminal cross-platform technology

Mobile cross-platform technology is not a new topic. A few years ago, WebView container, React Native, Weex, Flutter, apts and other mobile cross-platform frameworks began to emerge. Why is cross-platform so attractive? Let’s imagine if we can do one development, multiple reuse, then for the company, it can reduce labor costs. For development, you only need to learn a framework to develop on both Android and iOS platforms. The cost saved can be invested in quick product verification and quick online. This is a great attraction for everyone. This section provides a brief introduction to some of the cross-platform technologies on mobile terminals so that readers can better understand the following content.

1.1.1 WebView container

The working principle of WebView container is to realize the interface and function based on Web technology. By encapsulating and exposing the native interface to JavaScript, the page written by JavaScript can run in the WebView of the system. The advantage of this approach is that it is friendly to front-end developers and can be quickly implemented across pages, while retaining the ability to call native through bridging layers and native capabilities. In this design, however, cross-end capabilities are limited by the bridge layer, and Bridges need to be added when invoking native capabilities that were not previously available. In addition, the browser kernel rendering is independent of the system components and cannot guarantee the native experience, rendering will be much worse.

1.1.2 the React Native

In 2015, Facebook launched React Native, which was an instant hit. Its idea is to maximize the reuse of front-end ecology and Native ecology, and the biggest difference between WebView container and View rendering system. React Native ditches inefficient browser kernel rendering and instead uses its own DSL to generate intermediate formats, which are then mapped to the corresponding platform and rendered as platform components. Compared to the WebView container, the experience is somewhat improved. However, rendering requires communication between JavaScript and native, which can cause stuttering in some scenarios. On the other hand, rendering is still in the Native layer, which requires developers to have some familiarity with Native.

1.1.3 Flutter

In 2018, Google launched Flutter to build a set of cross-platform development components using the Dart language. All components are self-drawn based on the Skia engine, and their performance is comparable to that of Native platform View. Flutter stands on the shoulders of its predecessors. It references React state management, the Web’s self-drawing UI, React Native’s HotReload, and other features. It also takes into account the Channel mechanism for communicating with Native, self-rendering, and a complete development tool chain. Flutter is fundamentally different from these Recat Native and WebView containers. Instead of using WebView, JavaScript interpreter, or Native controls that come with the system platform, Flutter has its own set of widgets. The underlying rendering is self-painted using its own high-performance C/C++ engine. However, most mobile terminals have formed their own architectures until now. Adding Flutter on the existing basis will cause the coexistence of the original architecture and the dual platforms of Flutter. At present, it is the most promising cross-end scheme for new apps.

1.2 Introduction to Meituan take-out business

As a leading lifestyle service e-commerce platform in China, Meituan is committed to connecting consumers and businesses with technology, providing services to meet People’s Daily “eating” needs, and further expanding to a variety of lifestyle and tourism services. As one of the most important businesses of the company, Meituan Takeout has expanded from a single category to nearby food, fruits, vegetables, supermarkets, flowers, cakes and other categories since its establishment in 2013, from breakfast, lunch and dinner to afternoon tea and night snack. Chinese food, Western food, home cooking, snacks, fast food, seafood, hot pot, Sichuan food, cake, barbecue, fruit, beverage, dessert and other kinds of food. Meituan take-out can be said to be one of the most complex businesses in the current e-commerce field.

The complexity of business also brings great challenges to the system architecture. Meituan take-out business is the most complex business in the current e-commerce field, mainly due to the following characteristics:

  • Meituan takeout business is carried on three apps: Meituan Takeout App, Meituan App Takeout Channel and Review App Takeout channel.
  • As an important user entrance of Meituan, Meituan takeout also plays the role of a traffic platform, providing platform capabilities to support the development of channel business, such as flash shopping, errands, finance, etc.
  • As a mature business, Meituan Takeout needs to provide reusable platform capabilities to support the development of new businesses, such as caidaquan App.
  • As a super business side, Meituan takeout also operates multiple businesses within its business, such as traffic business, transaction business, merchant business, commodity business, marketing business, advertising business, etc.

To sum up, it can be found that not only the business of Meituan Waimai is complicated, but also its external role is complicated. Within Meituan, take-out is not only a channel business of Meituan platform, but also a platform business itself. At the same time, Meituan take-out also assumes the role of a platform for new business development. This means that it is very challenging to support the development of Haomeituan’s takeaway business.

1.3 Historical architecture of Meituan Waimai mobile terminal

Good architecture comes from constant evolution, not design. The architecture of Meituan Takeout has gone through many iterations in its history. As the delivery business is changing constantly, the original design needs to evolve with the business. In the course of exploration and practice, we went through several big architectural changes. From considering how to efficiently reuse code to support takeout App, it gradually evolved into how to solve the problem of multi-channel code reuse, and then from multi-channel code reuse to support the platform architecture of other channel business. After the construction of platform architecture was completed, we began to try to use dynamic technology to support the demand of fast online business. Nowadays, we are faced with multiple business scenarios such as multi-terminal reuse, platform capability, platform support, single-page multi-business team, and strong business dynamic appeal. The following is a brief overview of the evolution history of Meituan Takeout mobile terminal architecture, so that readers can have better continuity when reading this article.

1.3.1 Component Architecture

In the early stage, Meituan Take-out, as an incubation business of the company, completed version 1.0 of its Meituan Take-out App at the end of 2013. With the success of the verification and operation of the takeaway business, the order volume also increased rapidly, exceeding 1 million orders per day at the end of 2014. Subsequently, in February 2015, take-out was connected to Meituan App in Native form and became a business channel of Meituan App. During the access process, we copied a large number of codes from Meituan Takeaway App to the takeaway channel of Meituan App, and the takeaway business codes on the two apps were maintained by two independent teams respectively. In the early days, the takeout business was changing rapidly, with frequent App iterations and extensive code writing methods. Meanwhile, Meituan App was also in a period of platform transformation, and the stability and quality of code were changing and improving. These factors lead to serious coupling and fuzzy boundary among subsystems in the takeaway code, and the situation of “I am in you, I am in you” can be seen everywhere. This has a significant impact on code quality, functionality extension, and development efficiency. At this point, the purpose of our architecture reconstruction is to divide each subsystem into relatively independent components, which can be directly reused. The architecture is shown in the following figure:

1.3.2 Platform Architecture

As mentioned above, meituan Takeaway and Meituan Takeaway channel are maintained and developed by different teams. In 2015, considering the consistency of business development, the company officially assigned meituan Takeout channel team to Meituan Takeout. In terms of organizational structure, Meituan Takeout and Meituan Takeout Channel gradually merged into a team. However, due to the differences between the two sides, we had to maintain the original two teams for the maintenance of the independent takeout App and Meituan Takeout Channel. How to solve this problem? Reusing code from both ends seems to be the only way. In addition, with the rapid development of business, takeout apps carry more and more business modules, product functions are becoming more and more complex, and team size is also increasing. For example, flash shopping, errand running and other businesses want to access takeout apps in the form of independent Native packages, and the establishment of takeaway r&d teams in different places have brought challenges. This led us in 2017 to embark on a second architecture refactoring — platform-based architecture, with the goal of enabling multi-reuse and supporting business growth across different teams. By abstracting platform capability layer, business decoupling and building shell container, the platform architecture is finally realized, as shown in the figure below:

1.3.3 RN hybrid architecture

After the platform architecture, the functions of Meituan food delivery continue to increase, and the volume of meituan food delivery client installation package continues to increase. Looking back at 2017 and 2018, it was almost a 100% increase every year. Without an effective means, the installation package will become more and more bloated. In addition, as native applications need to be updated based on the application market, each product update must rely on users’ initiative to update, resulting in a long iteration cycle of versions. These business pain points constantly force us to think about whether there is a framework to solve these problems.

In 2015, Facebook released the highly disruptive React Native framework, or RN. From the name, we can clearly understand that this is a hybrid development mode. RN uses Native to render and JS to code, thus realizing cross-platform development, fast compilation, fast release, efficient rendering and layout. RN is a cross-platform mobile application development framework, and its features are very suitable for us. Meituan is also actively exploring RN technology. On the basis of RN, Meituan React Native (MRN) technology system has been formed by comprehensively customizing and optimizing scaffolding, component library, preloading, subcontracting construction, release operation and maintenance and other aspects, greatly improving the efficiency of RN’s development and release operation and maintenance.

Since 2018, meituan takeout client team has tried to solve business problems by using MRN framework. Another advantage of using RN is that it can gradually eliminate the problems caused by Android and iOS development technology stacks. By using one set of code and launching the two platforms, theoretically the human efficiency can be doubled and the supported business requirements can also be doubled. The architecture is shown in the figure below:

2. Panorama of Container architecture of Meituan Takeout

2.1 What is containerized Architecture

As mentioned above, take-out business has developed to the business stage of multi-app reuse and single-page multi-business team development. To meet such a business scenario, it is not easy to find a sustainable business architecture. Through our previous architecture evolution, we have gained valuable experience: in the process of platform architecture, we decoupled App and business, made App into a shell container, and formed an independent business library, which was integrated into the shell container, thus shielding the problem of multiple Apps and improving the reuse of business. In RN hybrid architecture, we introduced RN container, which enables the business to mask the platform differences between Android and iOS. With the help of these successful experiences, we further thought, if we try to further subdivide the business scenarios of takeout, build the basic capabilities in different scenarios into shell containers, and integrate business into containers, can we better support the current situation of our multi-app reuse, single-page multi-business team?

The vision of the container architecture is:

  • We hope to abstract out the environment of front-end business presentation, standardize capabilities, and form a unified container through which to shield the differences between platforms and terminals. The container provides a unified capability interface of the upper level, enabling business developers to focus on the realization of business logic in the container and maximize the use of existing capabilities, regardless of whether the current environment is Android or iOS, and whether the current end is Meituan App or Dianping App.
  • The container and the remote reach a rendering agreement, so that the content on the end is ready to change. The implementation of containerized architecture has certain preconditions. If the business development itself is in the exploratory stage and there are many variable factors, it cannot form a stable capability layer. At this time, the construction of containerized architecture will make the architecture more complex. However, for the takeout business scenario, after years of precipitation and fixation, the takeout business has gradually formed a set of stable business forms, and has entered the stage of scene segmentation and rapid iteration of business modules. At such a stage, containerized architecture can be implemented.

2.2 Advantages of containerized architecture

When we abstract and standardize the environment in which the delivery business is hosted, there are several benefits. First of all, the dynamic property is improved. We can put the original business that must be written on the client side into the remote side, which greatly improves the dynamic nature of the business and makes it possible to put the business online at any time. For the development process, the speed of compilation and deployment has also been greatly improved. If a client-side code change is involved, the client-side compilation package, even incremental compilation, is at least second speed. With containerization, we only package the necessary business and dynamically deliver the business to the container for rendering. The client code itself does not change, thus reducing compilation from second to millisecond. Similarly, dynamic service delivery helps to reduce the package size of the client.

Then, the container is located in the application, so we introduce the same container SDK into the application, and the container shields the differences between applications. For Android and iOS platforms, the container layer is designed to shield the differences between platforms as much as possible, so that business developers only need to know the container. You don’t have to pay a lot of attention to the differences between applications and platforms, which leads to a huge increase in development efficiency.

Secondly, after containerization, containers have interface protocol requirements on the contents carried by containers. Only the contents carried by containers meet the protocol defined by containers can get the benefits brought by containers, which promotes finer granularity of business subdivision and ensures the awareness of modularity during business development. In addition, the interface provided by the container layer is standardized on Android and iOS, and business development tends to be standardized due to the standardization of dependence, which improves the consistency of business on both ends. These potential architectural benefits lay a good foundation for future business maintenance and expansion.

2.3 Panoramic view of takeaway container architecture

The entire takeout container architecture can be interpreted from a bottom-up, left-to-right perspective:

The bottom layer is system services, because we use cross-end technology stacks like H5 and RN, which makes iOS and Android the bottom layer.

On top of the system service is the group’s infrastructure based on Native construction, which is common to the whole company and covers all aspects of basic services of R&D projects.

On top of the infrastructure is the container layer that we define. We try to solve all problems with a single technology stack. But after our exploration, it seems unlikely. A good architecture should match the business form. Business demands determine that we cannot choose a unique technology stack to solve all problems. The following three page categories can be obtained by subdividing the takeout business scenarios:

  • High PV main process page, such as home page, king Kong page, business page, etc., this kind of page PV is much higher than other pages. This type of page is characterized by strong interaction, high visibility, and multi-team construction and maintenance. On this page, experience, to provide users with the utmost cannot be realized by using the H5 or RN, even able to meet the performance, but these pages are more team to participate in the construction, if use the H5 or RN implementation, then a single team changes, will cause the whole page is affected by the unpredictable, other unchanged business parties is certainly not acceptable. So this kind of page, we use the local dynamic + page modularization scheme, we carry out container transformation for the page, the page into a container, container bearing module. Each module is maintained by different service providers and decoupled from the module dimensions. Each module can be dynamically configured and delivered.
  • Low PV auxiliary pages, such as help, footprint, collection, etc., the PV of this kind of page is low, but it is better than the number of pages, and the cost of native implementation is relatively high. If all the pages are implemented with H5, the association between many pages and native is still relatively close, which is not very suitable. For this kind of page, we use MRN infrastructure provided by the group to carry it. As a cross-end technology stack, MRN has built rich components in the previous RN mixed architecture, and made rich construction for the custom MRN container. At the same time, MRN has the ability of dynamic delivery to meet business demands.
  • The page of operation activities released to the outside, such as Christmas activities, has a strong timeliness. This kind of page is completed by H5 technology stack. On the one hand, it can meet the timeliness. On the other hand, H5 has the strongest dynamic delivery ability, which can fully meet the demands of business. We use Titans infrastructure provided by the group and support the business by building custom Titans containers.

Further up, it is vertical business, takeout currently has traffic business, transaction business, business, commodity business, advertising business, marketing business, flash purchase business. Businesses are vertically dependent, directly visible to the container, visible to the infrastructure, can well access to various built capabilities to meet the needs of the business.

At the top is the App terminal, which currently has four terminals, including takeout, comment, Meituan, flash purchase and so on.

On the right are test publishing and online monitoring. Compared with the conventional mobile App architecture, containerized architecture is more refined in test publishing and monitoring. It’s not just about the availability of the end itself, but also about the availability of the container, the modules that the container hosts, the templates that the module presents, and the styles in the templates.

2.4 Containerization challenges

Compared with the conventional mobile terminal architecture, containerized architecture changes from managing mobile terminal code to managing mobile terminal container construction code and business remote development code, with more container and business remote delivery. This is not only a technical challenge, for long-term client development students, but also a jump in thinking change.

Consistency challenges: Containers need to run across multiple host applications, and the consistency of the host application’s environment directly affects container consistency. Our strategy is to take advantage of the takeaway business to align the environment of the host application; On the other hand, the container is built into SDK, which will keep the consistency of the container for a long time and shield the differences between applications through the design of SDK. For The Android and iOS platforms, we layered the design so as to mask the platform differences as much as possible. To sum up, the conformance challenge is (1) the consistency of the environment in which the container runs the host application; (2) Consistency of containers of different versions for different applications; (3) The business consistency of Android and iOS platform containers.

Challenges of dynamic release: For a long time, the concept of App version was only included in the development concept of the client side, but when we gradually make the business code to be delivered remotely, an online dynamic release concept will be added. When we release the business, compared with previous work, we need to consider the version of the business, and the upper and lower versions of App corresponding to the container that can be run. In addition, the launch cycle will also increase the launch cycle of the business, not only the App launch cycle. These two together will generate new sparks, business version and App version how to fit, business dynamic release cycle and App release cycle how to fit. The solution for take-out is to build a model of major release iteration + weekly iteration.

The challenge of monitoring operations: In the past mobile architecture, we focused more on the availability of the end itself, but when we evolved to container architecture, just focusing on the availability of the end is far from ensuring that the business is available. We need to extend the availability from the end to download links, load links, use the availability of links, and monitor o&M for each important environment.

3. Delivery cross-end container construction

3.1 the MRN container

3.1.1 INTRODUCTION to MRN Containers

The React Native framework itself is just a rendering engine in a runtime environment. It can render the same JS code in a Native way on both Android and iOS systems, thus providing basic cross-terminal capabilities for apps. However, from the perspective of engineering, if RN technology is to be applied in App on a large scale, in addition to THE RN framework itself, cooperation in development, construction, testing, deployment, operation and maintenance and many other aspects is required. Meituan React Native (MRN) is a dynamic solution reconstructed and improved by Meituan based on React Native framework, which provides containerization capability, dynamic capability, multi-terminal reuse capability and engineering guarantee on the basis of RN. MRN has been upgraded and expanded in terms of development efficiency, stability, performance experience, dynamic and monitoring operation and maintenance to meet the needs of Meituan RN development engineering. At present, MRN has access to more than 40 MEituan apps, core framework and ecological tools have more than 100 internal code contributors, and the total PV exceeds 400 million.

3.1.2 Roo Component library

Here are two more UI-related technical projects for takeout construction, the Roo component library and component style dynamic configuration.

  • Roo component library: In addition to the UI components provided by RN and MRN frameworks, Roo extends the standardized UI component library suitable for takeout business. On the one hand, the UI component library unifies our design specification and development specification and improves UI consistency; Component encapsulation, on the other hand, improves the efficiency and quality of RN page development.
  • Component Style dynamic configuration: With standard Roo components, we further add style dynamic configuration capabilities to the standard component dynamics. When using components, many styles can be dynamically delivered, such as fonts, rounded corners, background colors, etc., which is convenient for UI adaptation and revision.

Waimai began experimenting with MRN containers in its takeout business at the end of 2018, and launched a large area of pages in the first half of 2019. At present, nearly 60 RN pages of takeout have been launched, accounting for more than 80% of the takeout pages, including Tab page “my”, bill of Lading selection red envelope page, order evaluation page and other high PV pages. The access of MRN containers has made a great contribution to the containerization, dynamics, human efficiency improvement, bag size slimming and other aspects of takeaway App.

3.2 Titans to container

3.2.1 An introduction to Titans

Titans container is a unified Web container component of THE US Group App. Based on the WebView component provided by Apple, it containerizes WebView, unifies the UI display and interaction mode of WebView, standardizes the use paradigm of bridge protocol, and presets many basic capabilities and business capabilities. Titans containers greatly improve the efficiency of Web page development and consistency in the user experience.

  • Web containers: Titans provide uniform UI presentation and custom styles, such as navigation bar style, page Loading state, progress bar style, etc. There are unified interaction methods, such as page hopping, Scheme protocol parsing, etc.
  • KNB Unified Bridge service: Although Web container makes up the deficiency of Native well in dynamic and dual-end reuse of Android and iOS, it is difficult to reach the standard of Native in many places. Therefore, KNB bridge came into being. KNB defines the standard way of communication between Native and JS, which is convenient for bridge protocol extension during development. At the same time, KNB also has a lot of Native basic capabilities built in, which greatly improves the user experience and development efficiency of Titans.
  • Web Business enhancements: Rich basic business pages, such as login pages, share pop-ups, are pre-built into Titans containers.
  • Provide link enhancement: provide long connection, offline and other ways to improve the speed and success rate of network requests.
  • WebView preloading: After the App starts, the main HTML document and basic library should be loaded in advance before the user clicks on the web portal. In this way, when the user clicks on the web portal, the App directly uses the prepared WebView and only needs to load a small amount of business code. In this way, the white screen time is short and the page is loaded quickly.

The usage scenarios of Titans container in takeout business are very rich, among which the most important use scenarios are various operation pages and active pages, such as click the landing page of the Banner Banner at the top of the home page, optimize for you, and the active operation pages such as time limit. There are customer service page, help feedback page, business settlement page, Meituan public welfare page and other functional pages; The Meituan member page, which is a level one entry page, is also a Titans container based on Enlight.

4. Container construction of takeaway page

In the construction of takeaway container, the first thing to distinguish is the core page and non-core page. In takeout business, the definition of core page is page DAU> 5% of Meituan DAU or the critical path to order. Why split the core pages in the first place? The point is the cost of retrofitting. The business complexity of the core page determines that it is not easy to make the whole page dynamic, it is more suitable for local dynamic scheme. The complexity of the core page lies in the complexity of the business itself. The most important thing is that the core page is often jointly developed and maintained by multiple vertical business teams, who have their own key modules to make the whole page dynamic and cannot achieve effective physical isolation.

For non-core pages, the business functions and interactions are relatively simple and the organizational relationship is more determined, which is more suitable for standard MRN and Titans containerization. Therefore, our strategy is to achieve the business dynamic and reuse of the core page module level, while the non-core page can achieve the dynamic and reuse of the page level. The core meaning of page containerization is to divide a page into several modules, and each module becomes a business container. The filling of each container can be realized either in Native way or in Mach (Mach is a self-developed technology of local dynamic page). Can support iOS/Android/ small program three cross-platform running. The page itself becomes the manager of the container, responsible for the arrangement and layout of the child containers and supporting their dynamics.

4.1 Design idea of page container

Page container design is mainly divided into three stages, module order, module arrangement, gradual business landing.

  • Module ordering: split the coupled takeaway business codes according to module dimensions, establish a standardized combination and interaction scheme between modules, and reduce the impact of changes in modules on other modules. At this stage, we completed the standardized transformation of both Native module and local dynamic module.
  • Module arrangement: page one characteristic of container is the capability of layout module, at this stage we are on the client side increased support for business module structure arrangement ability, at the same time, after we talk to the students to build the configuration platform, through the configuration of the way through the AB experiment platform, unified data service and so on multiple platforms. With the support of standardized data protocols, the page supports the dynamic launch of AB experiments, greatly reducing the development cost of AB experiments on the client side, achieving zero cost on the client side and low cost on the back end, effectively supporting the big revision of the sixth anniversary of the takeout home page.
  • Progressive business landing: The most obvious benefit after page containerization is to support dynamic online of business requirements. This goal can be achieved only when the business modules in the page container have dynamic capability, which involves the process of migrating Native modules into Mach modules. In this regard, our thinking is to follow the business needs of gradual landing. In the module arrangement stage, we designed the ability of Native module to migrate to Mach module, and at the same time designed a triple degradation scheme covering the dimension of template and API interface to ensure the stability of module dynamic migration.

4.2 Standardization of Service Building Modules

From the perspective of App page development, a complete page can be divided into different modules according to different functions and business attributes.

Business building to denote by multiple business combination module assembled as a page in the process, involving the page itself (UIViewController/Activity), and each business module construction, before and after the end of the business data and the data interaction between page and business module, Data processing and view refresh process within the business module.

Module standardization refers to the standardization of multiple processes involved in business construction to form a unique standard. On the one hand, module standardization can provide special solutions for business difficulties on the basis of solving common business problems. On the other hand, it can form capacity constraints on the basis of the framework and reduce the problems of repetitive construction and low quality construction.

In business building module standardization, we abstract four layers, which will be interpreted separately below.

  • The last layer is the framework capability layer, which is a two-terminal modular framework developed by the takeaway business team in line with the characteristics of the takeaway business. The framework addresses common issues across page scenarios and supports typical business pain points. It is a kind of implementation scheme of page frame design in iOS and Android double-end alignment. This double-end alignment capability provides guarantee for the double-end consistency of page container design.
  • Unified interface layer is a standardized abstraction of the framework capability layer, which can ensure that the capabilities invoked by any module are implemented consistently in various business scenarios. With this layer of abstraction, any module can be directly reused in various scenarios.
  • In the upper layer is the global module reuse layer of App. Standardized modules can register modules with the module reuse pool through unique labels. This centralized registration mode enables business modules to be flexibly reused in the scenarios of cross-business libraries.
  • The top layer is the core business scene layer of takeout, and each scene corresponds to a standardized page container. The page container completes the construction of the page container by implementing containerized interfaces.

By standardizing business building blocks, business modules are already standardized and can be freely composed across pages, laying the foundation for page containerization.

The most basic capabilities in page containerization are as follows: the ability to arrange business modules in the page, the ability to conduct AB experiments in the front end of dynamic on-line, and the ability to incrementally launch dynamic modules. The most important way to realize these capabilities is to standardize the front-end and back-end data protocols. The client matches and constructs a service module according to the unique identifier of the module in the data protocol. After filling the module data, the client completes the module layout according to the module layout information in the data protocol. For Mach dynamic modules, we created a module matching and data filling process based on template ID, which can support incremental on-line of Mach dynamic templates. In the data protocol, we reserved AB experiment and general parameter fields for the front-end AB experiment, which were passed into the dynamic module through containerized interface during the data filling stage to support the dynamic on-line of AB experiment.

During the containerization process, the interface version is upgraded. To ensure high availability of containers, the client implements two degraded Dr Schemes at the module level and THE API level.

Module-level degradation schemes are mainly aimed at Mach dynamic modules. Different from Native modules, Mach dynamic modules need to download dynamic templates in advance to normally complete module loading and rendering. In order to ensure the loading success rate of dynamic modules, we used Eva (Internal system of Meituan) to preheat the download of Mach templates before the interface went online. On the other hand, we designed an active demotion scheme for dynamic modules. For the dynamic on-line of dynamic modules, Native modules were used for bottom-of-the-pocket demotion, and for the scheme of the same version of dynamic modules, App built-in templates were used for bottom-pocket demotion.

The API-level Dr Solution is used to ensure that clients can degrade to the old interface when the new interface is unstable. In order to solve this problem, we design a data structure mapping scheme for the old interface on the line. The data structure of the old interface can be mapped to the data structure of the new interface through configuration on the client. In this way, the Dr Solution can be switched on and off without the awareness of upper-layer services.

4.3 summary

Through page containerization, pages are only concerned with how they are constructed at the page level, not how they are dynamically implemented within a module. The separation of pages and page modules also conforms to the current organizational structure of takeaway clients, which is conducive to collaboration between business groups. At the same time, page container makes the core page of takeout have the dynamic capability in line with the takeout business scenario, and gradually transfer the Native static module to the module with dynamic capability, so that the whole page has the dynamic capability from the dimension of module. This progressive migration scheme separates container migration from business module migration and greatly reduces the risk of page containerization.

5. Measurement indicators of takeaway container architecture

5.1 Characteristics of container architecture measurement Indicators

Quality and performance indicators are an important basis to measure the quality and user experience of our App development, and are the key data that we always pay close attention to. In the container, we most of the indicators are closely related to the use of the App link, because in the container to the era, logical link is relatively simple, such as when we open a new page, we first create a page instance, and then launched a network request, page will experience a series of lifecycle methods at the same time, the final rendering. At this time, we may pay attention to the success rate and request time of network request, the rendering time of page, and whether Crash occurs after success. This process is relatively short and has fewer indicators, so it is easier to monitor.

Containerization of takeaway services greatly improves the reuse capability, dynamic capability, modularization, and development efficiency of takeaway services, but also brings longer logical links. Links are divided into download links, load links, and use links in terms of time. For example, when we use MRN containers, we will have to start or warm up the download of the bundle, decompress the bundle, initialize the MRN container engine, load the bundle, load and execute the JS, render the page and other steps, each of which may have performance problems and quality risks. Therefore, we need to upgrade our measurement system to meet the new challenges of containerization.

5.2 Link Specifications

  • Download link: The download link is used to download various resources required by the container. In MRN and Mach, the download task is mainly for the bundle. Only after the bundle is downloaded successfully, the following operations can be performed. Therefore, the success rate of bundle downloading is the most important indicator in the download link, and the timing of bundle downloading is also very important. There are hundreds of bundels in the takeout service. If all bundles are pulled during startup, the cold startup time will be greatly affected. We adopted the method of preheating bundles. To release bundles, we Tag them with the corresponding tags and download them at the appropriate time to avoid centralized downloading.
  • Loading link: The important work in loading link is to decompress and parse the downloaded resources in the download link. For example, when using PGA to load the page, module parsing, module matching, module degradation, data model generation and other steps will be carried out. In MRN, bundle decompression, engine initialization, and bundle loading are performed. Link loading consumes computing resources and has a significant impact on page opening and loading time. Therefore, we pay attention to link loading performance indicators.
  • Using link: The use stage of using link and non-containerization is basically the same, mainly focusing on page loading time, Crash rate, page FPS, page lag and other indicators. In addition, we will also pay attention to some indicators related to the characteristics of the container itself. For example, in MRN container, we will also pay attention to JS error rate, JS rendering time, white screen rate and other indicators.

5.3 Key Indicators

Because the use of containerization forms a serial link, if a key node fails, it will lead to the unavailability of container functions. The task of key indicators is to screen out these key nodes from the above numerous indicators. Such as bundle in the download link to download the success rate of the success rate and API to load the link in the bundle loading and module matching success rate, success rate, download or load failure are unable to link the subsequent steps, in view of the above the success rate of indicators, we will add minutes level real-time monitoring alarm, do it in time, Quick response and emergency repair.

The success rate of module rendering, Native Crash rate and JS error rate in using links are also key indicators. The failure of these tasks will also lead to the unavailability of containers. For these indicators, we will adopt real-time monitoring measures and add degradation measures, such as rolling back the bundle version. Or demote MRN and Mach containers to Native ones.

6. Monitoring operation and maintenance of takeaway container architecture

As mentioned above, the measurement indicators of container architecture, the work to put these indicators into practice is online operation and maintenance monitoring work. For the monitoring operation and maintenance work, it is necessary to have appropriate monitoring tools to assist and cooperate to get twice the result with half the effort. There are many excellent monitoring statistical tools available in the company. The difficulty here is how to choose the appropriate tools according to the needs of monitoring. Then there is the division of reasonable monitoring dimension and the priority of data index, for example, can affect the stability of the link key indicators, we need to do minute level monitoring, once appear problem, can receive timely warning, for non-critical indicators, then through the way of generating daily, statistics and analysis for developers.

The use of tools is mainly divided into large scale tools, specific abnormal tools, gray scale degradation tools, alarm tools, etc. (the following are internal tools used by Meituan).

  • Market tools: CAT, Skynet, Crash platform and other tools are mainly used. These tools collect, aggregate, and generate visualized charts and curves. It is convenient for developers to understand the overall situation and changing trend of the market.
  • Exception tools: Use tools such as Sniffer and Logan. These tools can be used to obtain the context and device information when exceptions occur, and retrieve user behavior logs to locate and troubleshoot specific problems.
  • Gray scale reduction tool: use ABTest platform, Horn and other tools. Used for delivery configuration for grayscale control or switching control.
  • Alarm Tool: Alarm assistant. Send alarms to individuals or groups in a timely manner through IM software.

Service coverage dimension monitoring can be divided into global monitoring and local (single service) monitoring.

  • Global monitoring: monitor all containerized quality indicators and performance indicators, and generate daily reports to facilitate tracking and monitoring of containerized overall quality.
  • Local (single service) monitoring: Real-time monitoring of online data on each page and each container, to find, locate and deal with problems in a timely manner.

Time dimension monitoring: The time dimension can be day, hour, or minute. Day-level monitoring is mainly about some non-critical path indicators, such as some performance indicators, page loading time, page FPS, JS rendering time, etc. We can generate data reports by day dimension and send daily mail data. When the App gray scale goes online, we will start hour-level monitoring and broadcast some key indicators through IM software every half hour, so as to facilitate developers to track the stability of online data. Minute-level monitoring monitors the changes of key indicators in minutes. If key indicators exceed the threshold or fluctuate excessively, an alarm is generated.

Let’s take a look at the monitoring operation and maintenance system of takeaway container architecture from a developer’s perspective. From the way of obtaining information can be divided into active query and passive push, developers can monitor the trend of global and local data through monitoring tools, and can also analyze specific abnormal cases. You can also receive relevant push data from IM tools, emails, etc., for timely response. In terms of operation and maintenance control, developers can release gray scale through the internal gray scale system of Meituan, such as Eva and Horn. When problems are found in the gray scale period, they can timely degrade the DISASTER recovery process by stopping gray scale, rolling back version, and closing entrance.

7. Ability to publish takeaway container architecture

7.1 Containerized Architecture Release system

Container is changed to delivery business has the strong ability of dynamic but dynamic capabilities and needs corresponding ability to support, release ability is our business development plays an important role in promoting the quality and efficiency, and also the key link in our container to change the working process of the construction, this section mainly introduce takeout containers release ability.

From the perspective of publishing capability types, it can be mainly divided into three types :(1) publishing container content, including publishing the whole page or publishing partial modules in the page; (2) Configuration delivery: delivers layout protocol, AB test, style configuration, function configuration, template configuration, container configuration, etc., through API or other configuration platforms, greatly improving service flexibility and online verification ability; (3) Grayscale and downgrade delivery, and control ability of grayscale release and rollback through UUID, user portrait and other information.

From the perspective of publishing resources, there are two main types: one is common resources, such as publishing a Web page, or releasing a new version of API to control the display of local container of the page and the location of the display. At the same time, we can also conduct some AB Test operations; The other is bundle resources, mainly for MRN containers and Mach containers. The resources of each MRN container and Mach container are packaged into a bundle and delivered to the terminal through the publishing system. The terminal then parses the code and resources in the bundle and renders the page.

From the point of view of the release stage, it can be divided into test stage, on-line stage, gray stage and full scale stage, of which the on-line stage is the final link, we added a lot of verification and protection means to ensure the correctness of on-line operation.

7.2 Release process

Although we have the ability to dynamically release at any time, normal release iterations still exist, so the rhythm of the delivery side is weekly dynamic iteration + biweekly release iteration, which ensures a clear cycle for our development work. The most critical stage in the dynamic release stage is the go live stage. Taking MRN as an example, there should be more than 70 bundles in the takeaway business at present, and there are nearly 150 bundles including the bundles in the test environment. Managing these bundles is a complicated task. Moreover, the online operation still involves the release of the target App, the upper and lower bounds of the App version, and the upper and lower bounds of the MRN version. If you are not careful, the operation will be wrong, so the online operation needs to be very careful.

We conducted a transaction pipeline for the online operation stage, and established protection measures through the pipeline. The online operation of a bundle must go through several operations of the pipeline. First of all, the operator shall carry out several inspection operations according to the online SOP manual, and compile the release instructions in standard format, and then publicize them to the relevant core personnel and launch the online application on the operating system. The Leader and QA will check and approve the application after receiving it. After the application is approved, the operator shall avoid the peak period of App use or the online application in holidays. After going online, observe all data indicators through grayscale release, and release in full when the indicators are normal.

7.3 Publishing Bundle Resources

Bundle is the most common resource type that we publish. Here’s how to publish bundles with publishing tools. Both MRN and Mach are delivered to device terminals in the form of bundles. We mainly use two tools when releasing bundles: Talos and Eva (internal tools of Meituan). A bundle’s project file consists of three parts: the configuration file, the source code, and the resource file. The configuration file instructs Talos to package the project file. Multiple bundles can share one configuration file. When we are ready to publish a bundle, we first find the bundle’s publishing template in Talos, select the publishing environment (test or online), and perform one-click packaging. Talos then performs a series of pipelining operations, including Clone code, configuring the environment, Lint checking, build, and upload. After Talos is packaged, the bundle is uploaded to Eva system. Then Eva is responsible for the subcontracting, online, offline, gray scale and other operations of the bundle, and finally delivers the bundle to the terminal device.

In the future, we will introduce MRN-DevOps of Meituan Accommodation to further shield the current problems of multiple systems, reduce the cost of the whole cycle management, especially the cost of manual inspection before release, and gradually realize that RD can operate on one platform from research and development to release operation and maintenance. Reduce labor costs and improve automation as much as possible.

7.4 Comprehensive use of multiple publishing capabilities

The publishing process described above is in the form of bundle resources, which is relatively clear and simple. The following introduces the release method of partial containerization in combination with the takeout home page. Takeout home page is a typical streaming list. Under the local containerization architecture, the home page is arranged in ListView mode by rectangular containers, which can be divided into two types, Native container and Mach container. Mach container is a universal container, and we can write different style templates. It is delivered to the terminal and rendered by the generic Mach container, so as to achieve the purpose of using only the generic container to display different UI styles, which involves the publishing system of Mach.

Each sub-container of the home page is like a block of building blocks. Their position arrangement, display or not, and template selection are ultimately controlled by API, which has the ability to control the layout and style display of the home page, rather than a simple data source. At the same time, the home page also involves AB capability, gray scale degradation strategy and other actual configuration items delivery system. It can be seen that the containerization of the takeout home page is supported by a variety of publishing capabilities, which is the “comprehensive” of the takeout publishing capability system.

8. To summarize

A good architecture is one that evolves to suit the business as it evolves. Meituan Takeout has gradually grown from a small business with only a few thousand orders per day to today, with a peak of more than 40 million orders per day. Its organizational structure has also evolved from a team of a dozen people to a multi-role, multi-vertical business direction with thousands of people working together. In order to adapt to the requirements of business development, the architecture on the mobile terminal has also experienced the changes of componentization, platformization, RN mixing, and now to containerization.

Compared with the traditional mobile terminal architecture, container architecture makes full use of the current cross-terminal technology and maximizes the dynamic capability of business. It can shorten business iteration cycle, accelerate compilation and improve development efficiency. At the same time, it also solves the business problems we are facing such as multi-terminal reuse, platform capability, platform support, single-page multi-business team and strong business dynamic appeal.

Of course, the benefits of containerization come with stricter requirements for online availability, container availability, and online distribution that supports the business. We ensure the availability of dynamic services on the line by monitoring the availability of downloads, loads, and uses on the link. For containers, we use mature testing infrastructure to build automated testing of containers to ensure the availability of containers. For releases, we build an iterative process that works with the release pipeline to make online releases more manageable.

Up to now, the takeout business has gone through dozens of dynamic business on-line Windows, and has been issued more than 100 times in total. Next six months, we will also further, from the perspective of the business needs, to subdivide the business requirements, make product side gradually establish a container and the concept of dynamic demand, can from the source, gradually the business division, eventually making every business needs, can be classified into abstract business that may be dynamic and container capacity building, To further improve the ability of the container architecture and support more business scenarios.

9. Reference materials

  • React Native’s practice in Meituan takeout client
  • Promotion, support and thinking of Meituan Takeout iOS multi-terminal reuse
  • Container evolution practice of Meituan Takeout front-end
  • UC browser client container architecture evolution
  • Do you know how the container structure of Alipay is built?
  • Talk about the evolution of cross-platform technology on mobile
  • Review of mainstream mobile cross-platform UI technologies: implementation principles, technical advantages and disadvantages, horizontal comparison, etc

10. Author profile

  • Guo Sai, Tong Tong and Xu Hong are all iOS engineers of Meituan Takeout.

11. Job information

Meituan takeout is looking for senior/senior engineers and technical experts in Android, iOS and FE. Please send your resume to [email protected].

For more technical articles, please follow the official wechat account of Meituantech.