This article is an interview with Ant Financial technical expert Hong Feng, who will share the practice of Mobile R&D DevOps in Alipay App at QCon Shanghai 2019. Welcome to follow.
An article on the Microsoft MSDN have such a words: “the ideal environment for mobile application need to satisfy two conditions, one can know exactly what the customer needs immediately to mind, 2 it is to meet these requirements and write code can be passed to the customer immediately, in simple terms, is no gap between the customer requirements, development and transfer.”
The ideal environment for mobile application development mentioned in this paragraph just happens to be met by DevOps. Therefore, the development mode of some super-scale apps, especially those with project-oriented r&d collaboration and many modules, have begun to slowly study the DevOps system. So, how does mobile DevOps construction differ from Web DevOps construction? What are the challenges of building mobile DevOps? How does mobile DevOps cope with the onslaught of new technology? … To answer these questions, InfoQ spoke with Ant Financial technology expert Hong Feng.
Mobile DevOps vs. Web DevOps
In the past two years, DevOps has become the mainstream of enterprise software development and has been adopted by many enterprises. There are a lot of practice sharing about DevOps, but most of them focus on Web DevOps. In comparison, the practice sharing of mobile DevOps is relatively small.
The reason for this, According to Ruck, is that Web DevOps has a more stable execution and validation environment, where all development, validation, delivery, and so on can usually take place directly on the server. The biggest challenge with mobile DevOps is that mobile apps don’t have a stable validation environment. Most mobile apps can be used on multiple devices, which means dealing with various technical specifications, operating system versions, screen sizes, etc. At present, there are two mainstream mobile terminal operating systems, namely Android and iOS. Among them, Android is famous for its own operating system. Each device manufacturer has created its own operating system, and the storage devices vary greatly. IOS also has multiple variants, and the OS versions of the storage devices span widely. In addition, mobile applications also have the characteristics of human-computer interaction, such as the current popular scanning code, face recognition and other technologies, so how to minimize human intervention in mobile DevOps, make the whole become smoother, and improve the overall research and development efficiency, this is the problem we need to think about.
Although there are some difficulties in the construction of mobile DevOps, mobile development and DevOps are naturally compatible, because client development, especially intelligent terminal, is an emerging field, with rapid technology iteration and update, coupled with the particularity of end products, such as the inability to roll back degradation in real time, etc. Therefore, the core construction of client products mainly focuses on R&D collaboration, R&D efficiency, implementation of new technologies, quality assurance and other aspects, which coincide with the construction goal of DevOps.
Aside from whether DevOps is appropriate for mobile development, I’m sure a lot of people will be interested in when DevOps is built. “The main purpose of DevOps is to improve overall r&d efficiency, so the best time to use DevOps is when you have multiple businesses growing in parallel, each with dozens of people,” Ruck said. When enterprise development efficiency and quality will become a bottleneck, in order to break the bottleneck, there will be many employees into the tool in the research and development, to realize the automation of a certain ability, hard to avoid can appear a lot of repeat creation of the wheel, then carry on construction of enterprise unified convergent, can avoid repeated made wheels.”
Alipay mobile DevOps construction
DevOps is closely related to business development, and DevOps without business scenarios is just empty talk. Therefore, The DevOps construction of Alipay was not started until the business development reached a certain stage.
According to Liu, Alipay’s technology and business development can be roughly divided into the following processes:
- In the early stage of
A warehouse, a set of code, then instrumental componentization, finally dynamic ecological and intelligent. In the early stage, developers may be a few digits and business functions are not complex. At this point, a few development and a few tests can complete the whole development. At this point, the investment in DevOps is far more than the output, and there is no need to build DevOps.
- The intermediate stage
With the rapid development of business and the outbreak of client technology, Alipay also implemented componentization and modular technology. At this time, the r&d staff reached dozens of people, and the communication and cooperation between people made quality assurance very difficult. It was also at this time that the unified construction was carried out.
Early stages of a warehouse, a set of code of code all fix model, under the condition of the code increases amount brings many problems, including the coupling between each other is serious, a compilation of build time is too long, coupled with the business development brought the problem come from line, need online repair ability, so pay treasure to the research of modular programme, implements the module isolation, compiled in advance, Dynamic update and other capabilities.
In the process of DevOps construction, Alipay mainly focuses on three aspects: R&D collaboration, R&D efficiency and quality assurance.
Take quality assurance as an example. As the code volume grows from one warehouse to hundreds of warehouses and millions of code levels, simple single-test or black-box testing can no longer guarantee overall quality. Ryuk’s team, therefore, through analyzing the characteristics of the pay treasure to business and business scenarios, in combination with alipay, technical framework, the depth of the custom development, fell to the ground a lot of static and dynamic quality assurance ability, static quality assurance ability including custom static code analysis, dependency analysis, etc., and dynamic quality assurance ability including real machine performance stability test, Use case recording playback, etc.
As the producer of the final product, the Alipay team also undertook the construction work, because large apps have extreme pursuit of performance and package size, and every little bit of performance improvement and package size reduction may directly affect user retention. Alipay team developed file rearrangement, code rearrangement, debuginfo deletion and other construction technologies through construction digging, and finally went online through gray level verification.
Difficulties in mobile DevOps construction
Jason Wong, Research director at Gartner, noted in a blog post: “Not all DevOps companies use DevOps for mobile development. According to Gartner, only 42% of DevOps users report that DevOps is used to support mobile application development.” Why is there so little DevOps practice in mobile application development scenarios?
According to Ryuk, the main reason lies in the ratio of input and output. Except for super App, most mobile App development adopts the way of small team and quick steps, with only a few developers and testers, almost in single digits. However, due to the particularity of mobile applications, to achieve CI capabilities such as automated use case testing and real machine testing, relatively large labor costs and capital costs are often required, such as the construction of laboratories, the purchase of a large number of terminal equipment, and the construction of a complete set of automation systems. In comparison, due to the low complexity of small apps, the quality of apps can be guaranteed by developing unit tests for testing students, manual use case writing and testing for most scenarios.
The specific construction of mobile DevOps, according to Ruck, includes the following difficulties:
First of all, the mobile application technology stack is fragmented. Android and iOS are two sets of completely different technology stacks. How to unify the r&d process of the two sets of technology stacks is a challenge. Alipay’s solution is to abstract a mobile TERMINAL RESEARCH and development model, such as unified abstract modular construction, unified dependency management model, unified iterative research and development flow, etc. Specific differences are placed on the implementation route of each model. For example, Android module construction will be configured to Linux server through route configuration and constructed through Docker. The iOS module is built by routing configuration to the Mac physical machine.
Secondly, the verification of mobile applications is very complicated, especially in the case of millions of alipay codes and thousands of people’s research and development, each code change can have a large number of codes and functions affected. At the code level, Alipay accurately tracks the impact surface of each Method through in-depth compilation and customized code scanning and dependency analysis capabilities of self-research at the framework level, and controls the risks brought by every change through the evaluation system. In the real phone verification environment, Alipay has a complete set of real phone laboratory, all-round monitoring of the overall compatibility, stability, performance of mobile applications.
Third, continuous integration is also one of the difficulties in building mobile DevOps. Continuous integration requires fast and stable feedback ability. Therefore, Alipay chose a single point of breakthrough on the continuous integration node and made many optimizations, such as construction performance. In terms of UI automation testing, Alipay has built a laboratory and customized hardware to support more equipment and improve stability and throughput. Faced with the connection of various verification service capabilities within the group, we have also developed our own CI orchestration tool by referring to CI tools such as Gitlab CI and Github Action in the industry, providing super experience for students in development and testing. Task execution DSL again we directly use Jenkins Pipeline to not duplicate the wheel.
In addition, new technologies have developed rapidly in mobile applications in recent years, such as Facebook’s React Ecosystem, Google’s Flutter, and recently Huawei’s Ark compilers and Hongmeng system. Because many scenes are customized based on the technology scene and technology level at that time, the emergence of new technology has a great impact on ecology and technology selection.
In order to face the change of new technology, mobile DevOps construction needs to separate the coupled and uncoupled parts of the new technology to minimize the impact. For example, in terms of quality engineering related to end technology, Alipay will carry out pre-research and technical reserve in advance. For example, in the field of static analysis, there were only Java and OC languages before, but JavaScript and TypeScript languages have been gradually introduced with the emergence of small programs. At the same time, static analysis and Native analysis of the front-end language will be done to ensure the quality of the entire client product. In the aspects unrelated to the end technology, there will be related teamwork skills, including requirements management, branch management, dependency management, personnel management, etc.