Idle fish technology – Cambodian super

With the advent of cross-end frameworks such as Flutter, business development students often need to cross-end business development, problem location, etc., on Android/IOS. The construction of new and unfamiliar environment will always encounter a variety of problems, leading to the failure of the construction, especially the IOS development environment, which is the most complex, not only the construction of the environment is tedious, but also the packaging speed after cutting branches is very slow, so we designed and implemented two tools to optimize the IOS development experience of Xianyu.

Problems with the IOS development experience

The development environment is difficult to set up

  • Development environments depend on specific software versions and are complex to configure

Xianyu IOS project relies not only on XCode, but also on taobaoenv 1.2.0 and Cocoapods 1.2.0 package management tools. Based on your experience, these two tools have fewer problems with Ruby 2.3.x. Specific software version, software version conflict, environment variable setting and other operating steps lead to complex environment construction, need to ask IOS development students to solve.

  • Maintenance difficult

After a Mac upgrade, CocoapoD was prone to problems and had to rebuild the development environment. There are a variety of reasons for this: the system environment variables change so that a particular version of Ruby cannot be found; As Ruby is upgraded, Cocoapod cannot be used and needs to be re-installed. Gem version issues; Ruby source issues, etc. This also leads to many developers dare not easily to upgrade the system, unable to timely experience the characteristics of the new system.

  • Pod relies on large downloads

Because of the way Cocoapod works, pod updates the download project dependencies and downloads the file information of each version, which is extremely large. Take the Xianyu IOS project as an example, a total of nearly 20G cache files need to be downloaded, and most of them are small files of several K size. The download time may last more than ten hours, resulting in a long time span from the construction of a new environment to the first experience.

APP packaging is slow after sharding

When developers are working on multiple branches/releases, it is often necessary to switch between branch development debugging and Bugfix. However, after switching branches, the entire IOS project takes about 30-40 minutes to pack. Sometimes you have to switch branches and repackage debugging to fix bugs in a version. Fixing and verifying bugs may take five minutes, but packaging takes more than 30 minutes, with a disproportionate amount of input and output.
In order to solve these existing problems, we have carried out a series of explorations, share with you, and welcome better solutions.

IOS Environment Setup

The continuous development of virtualization technology provides new ideas for us to unify the end-to-end development environment. We imagine that if the IOS development environment can be decoupled from Mac, and can be transplanted, and people can easily reuse, then the first and second problems will be solved. We tried a few things:
Vm Solution
Set up a vm on the local MacOS and install the MacOS. Set up an IOS development environment in a VM, and use vm image copy to transplant the IOS development environment to solve the problem of setting up an environment.
! [](data:image/gif; base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQImWNgYGBgAAAABQABh6FO1AAAAABJRU5ErkJggg== “image.png”)! [](https://p6-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/50981c3bdb2f44c7af99021474ab8fb5~tplv-k3u1fbpfcp-zoom-1.image)
This scheme has the following problems:
A. Performance problems: IOS compilation is an IO – and CPU-intensive operation. The performance of VMS running on disks and cpus of the virtual HOST system is greatly compromised, resulting in long compilation time and poor development experience.
B. Safety issues. To install a VM on a Mac working machine, you must pass the company’s security audit.
C. Black Apple problem: The Mac system in the VM is not authorized, which may cause piracy and infringement risks.
Virtual machines are a heavy virtualization technology, so we moved to a more lightweight Docker technology.

Completely Docker,

All the software and environment variables that IOS development depends on are dockerized. Through docker image to achieve the transplantation of IOS development environment. Ruby class tools such as Cocoapod and Taobaoenv can be easily migrated to Docker due to the cross-platform nature of Ruby. But for XCode that relies heavily on MacOS, we tried to use XcBuild from Facebook instead. Xcodebuild is compatible with xCodeBuild, and there are people on the web who use it to build IOS builds.
! [](data:image/gif; base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQImWNgYGBgAAAABQABh6FO1AAAAABJRU5ErkJggg== “image.png”)! [](https://p9-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/50ec1a2075524f07a471efff01d0d32f~tplv-k3u1fbpfcp-zoom-1.image)
This scheme has the following problems:
A. The compatibility between Xcbuild and Xcodebuild cannot be evaluated
B. After xcode is upgraded, xcBuild can only go back to the original development plan and switch back and forth to the development environment during a period of compatibility with the upgrade, resulting in poor experience.
The above two solutions do not solve the problem of IOS development environment migration and decoupling well. However, in the attempt of full Docker-based, we found that the most complex cocoapod and Ruby installation and configuration can be Docker-based. Xcode installation does not require special configuration, so we designed and implemented a compromise solution: In-host development (partial Docker-ization)

In-host development (partial Docker-ization)

In this scheme: the development, compilation and debugging work is still local to MacOS, using Xcode; Docker-like cocoapod and TaobaoEnv related software and environment variable configuration. This not only follows the development experience of the development students, but also takes into account the portability of the development environment. ! [](https://p1-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/3663a096e7d249938282dfc08867cdf6~tplv-k3u1fbpfcp-zoom-1.image)
! [](data:image/gif; base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQImWNgYGBgAAAABQABh6FO1AAAAABJRU5ErkJggg== “image.png”)
In order to enable the dependency files pulled by Cocoapod in Docker and the generated POD project to be recognized by local XCode, we mount the local POD cache directory to Docker, so that pod pulled dependencies can be updated in Docker and accessed by XCode in MacOS. The details are as follows (unified programming plane +Faas software architecture diagram) :

In this way, it not only simplifies the complexity of setting up the development environment, but also facilitates the students who want to try IOS development to quickly set up the environment, and gives the developers no different experience. And through this program, our IOS development environment can be easily migrated in the development environment of each student, and can also be upgraded uniformly.
This solution migrates Pod related dependencies to Docker and decouples them from MacOS, so IOS developers can upgrade Mac system freely without worrying about the development environment being damaged, which solves the problem of difficult maintenance.
In order to solve the problem that a large number of pod dependencies need to be pulled in the newly built environment, we upload the local intermediate files of POD to the OSS cloud disk (blue OSS cloud disk in the figure above). The development students only need to download the compressed package and decompress it to the local, and then incrementally update it.

APP packaging speed problem after branch cutting

Client development students often need to develop business in multiple branches (versions), and often need to switch back and forth for business development and problem location. One problem caused by this is that when developers switch from branch A to branch B, they need to repackage the APP, and the whole process takes about 30-40 minutes.
After analyzing the packaging process of Xianyu IOS project, we locked the time in two stages: Pod operation and XCode compilation. Packaging speed optimization will also be carried out in two stages:

Pod operation acceleration

The main job of Pod Install/Update is to read podfiles, do dependency versioning and conflict resolution, and generate Pod projects. The generated related files are stored in the Pods directory and pods.xcodeProj. The Podfile often doesn’t change when you switch back to the previous branch, so rebuilding the POD project is wasteful.
After the test, if we save these intermediate files, after switching branches for many times, these intermediate files can still restore the previous Pod project, so as to avoid the step of regenerating the Pod project after cutting branches, saving about 10 minutes of overhead.

XCode compilation speed optimization

For optimization of XCode compilation speed, there are many schemes on the Internet, which can be roughly divided into three categories:
  • Cocopods relies on compilation acceleration:
For example, cocoapods-Packager can package POD dependencies into a static library. IOS projects introduce pod dependencies as static libraries, eliminating the need for repeated compilations.
But there are problems with the scheme; Updating private and third-party libraries is cumbersome, requiring repackaging static libraries and uploading them to the repository each time; And difficult to debug source code
  • Distributed compilation: such as distcc
The principle of distributed compilation is to spread the files that need to be compiled to other machines in the compilation cluster, and then send the compiled binaries back. The local compiler then links these binaries together. Distributed compilation obviously improves the speed of large projects, but slows down compilation speed for small projects.
  • Cache the intermediate results of compilation: CCache, BUCK
More widely available acceleration schemes are caching the intermediate results of compilations, such as CCache, Buck, etc., which are detailed online and will not be described in detail. However, the introduction of these schemes requires the transformation of the current IOS project, and even the need to change the user’s development habits, so it does not meet our requirements. But the scheme of caching intermediate compilation results gives us some inspiration:
We know has the capability of incremental compilation XCode is the last time it is also used to compile the intermediate, local compile time again, if it is found that file does not change, is to ignore the file, if the source file timestamp is updated, then recompile the file, because every time a source is a small amount of change, This can achieve the purpose of speeding up compilation.
For the Idle IOS project, if we save the intermediate of the current IOS branch compilation before cutting the branch, and then restore the saved intermediate when switching back to the current branch, can we trigger the Incremental compilation of XCode? And so it is.
! [](data:image/gif; base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQImWNgYGBgAAAABQABh6FO1AAAAABJRU5ErkJggg== “image.png”)! [](https://p1-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/ef47c801f229471f8818ae4ab48d2078~tplv-k3u1fbpfcp-zoom-1.image)
Specific plan:

  1. Cache the current branch’s Pods Project, Flutter Project, compiled intermediates, podfile. lock, linkmap, and other related files before shard.

  2. Switch branch

  3. An intermediate cached before a new branch is restored

  4. Repackage the IOS APP. Through these two steps of optimization, we reduced the packaging time of Xianyu IOS project after branch cutting from the original 30-40 minutes to less than five minutes, and the efficiency was increased by nearly six times.

* * * *
After docker image and cache optimization, the complexity and time-consuming steps in IOS environment construction are greatly reduced, and even IOS novices can basically complete them within three hours

At the same time, by caching and reusing the intermediate products generated in the packaging process, the packaging time after switching branches is controlled within five minutes, which is reduced to one sixth of the original, and the development efficiency is improved.