The background,

Every team has this problem, and as your business grows, so does your project code. Since the release of iOS version, hundreds of versions have been released. The whole project is componentized, and the number of private libraries used has reached 130+. Such a large project has brought many problems to us:

  1. Project compilation speed is slow, complete compilation for more than 20 minutes

  2. The output packaging speed is slow. It takes more than 20 minutes to package the output during the development and test submission phase

When faced with a large number of business requirements to test, test queue packaging ridicule complain, we know that this problem can not wait, urgent solution.

Under certain resource conditions, external hardware conditions cannot be used to improve, so how to solve this problem at the technical level, which has become an urgent need to be solved in front of the team

Second, the research

1.Xcode compilation optimization

As the official tool of iOS development, it would be best if Xcode itself could improve the speed. However, after investigation and investigation, some parameters can be changed to improve the speed, but it is not obvious, and some parameter changes even need to sacrifice some other things. I don’t want to go into details here, the relevant information has been very rich

2. CCache, etc

In this class, the results of the last compilation are cached in a certain way and reused next time to achieve the purpose of reducing compilation time. However, currently there are some limitations, can not be friendly and simple access, some projects may have to make some changes to achieve the function of caching. This is not a detailed introduction, related content can go to the official website of each project to understand

3. The binary

Believe that this should be the most used scheme in the industry, the current industry popular componentization, a component library binary, avoid repeated compilation, in order to achieve the purpose of reducing compilation time. The overall detailed plan, the industry team also output suitable for their own projects. Here are the general steps:

  1. Make the plugin

  2. Compile the binary version of the component library

  3. Component source code and binary, simultaneously push remote

  4. During the build phase, determine whether the sublibrary references binary or source

  5. To pull the corresponding version, switch the source binary tool

As you can see from the steps above, there are several problems:

  1. The project should already be componentized

  2. Binary component library issues (how to binary, when binary, storage issues with binary, pull speed issues, Xcode Index duration issues, debugging issues)

As can be seen from the above, if this set of problems are solved properly, it will definitely require a certain amount of manpower, time, resources, or a certain amount of cost. When this whole set is deployed, it will certainly need to be maintained for a period of time until the process tools are stable.

3. Package speed improvement scheme based on Xcode cache

To sum up, several methods are introduced. Some projects need to be reformed, some bring unsatisfactory results, and some require external resources. So somebody would say, we

Don’t want to retrofit

Don’t want to deploy a series of automation tools, development tools

Corporate iOS is a small team without a well-developed development environment

Is there a way to speed up packaging compilation?

There are! A caching scheme based on Xcode itself, which comes:

  • Fully compliant with project 0 invasions, you don’t need to make any changes to existing projects

  • No remote server, no push, no pull

  • There is no need to develop additional tool AIDS, independent or team development is the same

1, ideas,

After we know the Xcode compiled a full amount, when you compile again, it would quickly compile, because it takes advantage of the last compilation cache, but apple USES a file timestamp to judge whether the use of the cache, so when we pod update after operation, the file operation, it is easy to cause the repeated compile, Although the file may not have changed at this point, there is no speculation as to why Apple only uses timestamps. So it was easy for us to think, could we take advantage of apple’s cache, and we wrote a simple project to test it, and we copied the intermediate of a build, and we cleaned it, and if we build it, which is technically a recompile, I restored the copy of the last compilation before build, when build found that it did not recompile the library I copied, and the results were fine. Boy, there seems to be nothing wrong with that!

2, implementation,

According to the above ideas, it is easy to think of ideas, before we compile, through a certain method to judge whether the current need to compile the library cache, a cache with the cache, without a cache to compile, library judgment for the whole project is completed, began to compile, output, and then compile the participating in the library cache. So the whole thing looks familiar, right? Yeah, the whole idea is basically the same as SDWebImage, the image loading library that all iOSer is familiar with. The solution is familiar, but you may not be quite sure how to implement it. Here are some key steps and questions you may be concerned about:

  • Cache hit?

Since we generally care about whether the file contents change, we use all the files in the library, along with some other defined parameters, to calculate an MD5 as the cache key for the library. You might wonder, with all these files, is this going to be slow? In our actual engineering operation, measured and there is no problem

  

  • How do I control whether Xcode should be cached or compiled?

Here you can go to learn about Xcode compilation related content, familiar with the.pbxproj file, here is a brief introduction:

An Xcode Project file contains the following information:

– Source code, including header files and implementation files

– Internal and external static and dynamic libraries

– Resource file

The project. Pbxproj file is a configuration file in the xcode. xcodeProj package. If you open this file, you can see the following fields:

PBXHeadersBuildPhase: The link phase used for framewrok construction

PBXShellScriptPhase: Shell scripts used to copy resource files during the build phase

PBXSourceBuildPhase: Source files are compiled during the build phase

PBXResourceBuildPhase: Resource files to be copied during the build phase

Xcode can’t read the fields in the target’s PBXPROj configuration, so it doesn’t need to compile them again. Don’t worry. Here Xcode provides interfaces for you to operate, and the actual process is very simple

  

  • Where do I get the cache, where do I copy it?

The Xcode compilation process has a set of environment variables. The following fields contain some of the paths generated during the compilation process:

  CONFIGURATION_BUILD_DIR: "/Xcode/DerivedData/project-dxdgjvgsvvbhowgjqouevhmvgxgf/ArchiveIntermediates/Project Distribution/BuildProductsPath/Distribution-iphoneos"

  CONFIGURATION_TEMP_DIR: "/Xcode/DerivedData/project-dxdgjvgsvvbhowgjqouevhmvgxgf/ArchiveIntermediates/Project Distribution/IntermediateBuildFilesPath/project.build/Distribution-iphoneos"

  TARGET_BUILD_DIR: "/Xcode/DerivedData/project-dxdgjvgsvvbhowgjqouevhmvgxgf/ArchiveIntermediates/Project Distribution/InstallationBuildProductsLocation/Applications"

  TARGET_TEMP_DIR:"/Xcode/DerivedData/project-dxdgjvgsvvbhowgjqouevhmvgxgf/ArchiveIntermediates/Project Distribution/IntermediateBuildFilesPath/project.build/Distribution-iphoneos/Project.build"
Copy the code

“CONFIGURATION_BUILD_DIR” is the location you need to fetch or copy, read the value of the field during compilation, copy the cache to that path, or save the cache to that path after compilation

  

  • How do you make Xcode compile to perform these actions?

Xcode provides external interfaces, and scripts can be injected during compilation. We write the event scripts that need to be executed and inject them into Xcode compilation tasks through Xcode interfaces. Each time we compile a target, we will execute our custom script to check the target cache, copy, etc

3, run,

According to the actual measurement, on the 2019 MacBook Pro, the time of one of our projects is about 850s under the condition of complete compilation. After using the above scheme, the time can be as low as 60s under the condition of complete cache hit, which is almost 10 times of improvement. Of course, when we actually packaged the output, it was impossible to hit the cache completely every time. During the period of online operation of the scheme, we calculated that the average time was 200s, which increased by more than 70%, which also greatly improved the efficiency of many branches and development and testing stages.

The figure shows the time comparison of two major projects before and after using this tool

4. Sustainable optimization

  • Cache hit stability problem, hit logic tuning

  • At present, it can only be used for output packaging and adaptation to the development and compilation stage

  • Currently, only local caches are used to provide cache synchronization

Four, in the future

At present, the overall program is output in the way of Ruby gem, which has zero intrusion on APP engineering and can be easily accessed with a single command. In the future, other functions will be added to solve the pain points in iOS development, so as to continuously improve the efficiency and experience of iOS development

  • Cocoapods process optimization

  • Branch switching cache

  • Compile the cache