How does iOS compile one of more than 10 times faster steadily


Four, dual private source binary component overall design scheme

1. Production process – Binary components

2. Use processes – binary components

3, analysis,

This is shown in the figure above

**server** is a self-built service, dedicated to binary files

** Binary repo** is a private source repository dedicated to binary PodSpecs

The design of both binary components products are well stored, and we use cocoapoDS – IMy-bin plug-in in pod install/update time to automatically switch source/binary components, ingenious to avoid the invasion of the original components of the problem. Even if we shut down the **server or delete the binary repo** and clear the cache, the original packaging process and development process will still be executed without any impact.

Plug and play, do not care whether the service exists, binary components have been made, convenient and flexible.

At the same time, github/ component source/source repo belong to source source, and ** binary repo** form a dual private source

Five, the principle of making binary components

Projects can be made as long as they compile and pass, there’s no need to worry about whether a component can pass pod Lint, there’s no need to worry about whether a component has a push to the repo, there’s no need to wait for pod Lint, there’s no threshold at all.

1. Why are you so flexible and how did you do it

Most wheels require components to be POD Lint-able, but most components are not so standardized. The dream is always good, but the reality is always cruel.

To make this work for most components, we’ve bypassed pod Lint, optimized some cocoapods processes, and just picked up the build. A product and assembled Headers and Resources from the corresponding PodSpec file. Many wheels only make current component libraries and do not provide support for dependent ones. Using all of the. A compiled artifacts to make binary components with the latest code is a great efficiency boost.

When creating a binary component, the PodSpec dependency is no longer forced to pull the component from the repo. Instead, the dependency is taken from the Podfile first.

2. Production of static file a

Make static files are divided into two kinds, one is the cocoapoDS-IMy-bin themselves, one is the use of the compilation has completed the compilation products to assemble binary components.

2.1. Make it through the xcodeBuild of the plug-in itself

You can use the pod bin auto command of the plug-in directly. After the initial configuration of the plug-in is complete, as long as there is a podSpec file in the directory, the build, binary component assembly, binary PodSpec creation, binary file upload, binary PodSpec upload to the private source repository will be automatically performed.

pod bin auto --all-make
Copy the code

Taking the — all-make parameter automatically makes all the components that the current component depends on into binary components.

Xcodebuild GCC_PREPROCESSOR_DEFINITIONS='$(inherited)' ARCHS='arm64 armv7' OTHER_CFLAGS='-fembed-bitcode -qunused-arguments' CONFIGURATION_BUILD_DIR=build clean build-configuration Release -target 'target' -project ./' Target project '2>&1Copy the code
Xxcodebuild GCC_PREPROCESSOR_DEFINITIONS='$(inherited)' - SDK iphoneos CONFIGURATION_BUILD_DIR=build-simulator clean build-configuration Release release-target 'target' -project ./' Target project '2>&1Copy the code

Ci packaging only needs real machine mode, development needs simulator mode + real machine mode. After building the corresponding architecture, use LIPO to merge the architecture. A /. Framework

Libo-create 'x86.a' 'arm64.a' 'armv7.a' -output 'target static library '.aCopy the code

2.2. Combined with third-party compilation products

A /headers/resource in each binary component is then assembled via cocoapods-Imy-bin’s Pod bin local, and the corresponding binary PodSpec file is automatically created. Upload the repo and storage service of the corresponding environment.

This was a good choice since we only had one iOS baler and didn’t have the habit of waiting a few minutes for gitlab-CI to finish making binaries after submitting code.

After submitting the code, we waited for gitlab-CI to complete the binary file before typing the overall app package. This way seriously changed our current development habits, and did not speed up the overall process during the process of submitting the code to the outgoing package.

We also used Gitlab-CI to trigger the creation of binary components and Jenkins to build binary components regularly.

2.3. Ccache + binary two-layer compiled cache

Ccache is also applied to components that are not binary in the Jenkins package. The mechanism of dual-compile cache can greatly accelerate the overall speed under certain circumstances.

3, Headers,

The Headers. H file is already assembled during the Pod install/ Update process. You can copy the Headers/Public/ XXX file under the Pods, instead of assembling it according to the corresponding PodSpec source.

4, the Resource

Resources are grouped by searching for resources in the corresponding component library using the Resource field of the corresponding PodSpec. Here, you need to pay special attention to check some Chinese files with Spaces.

6, development machine, local command

After the cocopods-imy-bin plug-in is installed on the development machine, the binary components can be automatically created and uploaded by using the pod bin auto or pod bin local command. No other configuration is required.

Binary PodSpec

Automate PodSpec creation

The binary component is now a new component library, and we need to configure a PodSpec configuration index file for it.

Cocopods-imy-bin does not require a template to create a binary PodSpec. It automatically extracts a version of the source PodSpec to create it. After modifying the source, source_files, vendored_libraries, and public_header_files fields, all other fields are read. For a static component, the other decorating fields are not important.

# Binary podspec.json
{

  "name": "YYModel"."version": "1.0.4.1"."source": {
    "http": "http://xxx:10240/frameworks/YYModel/1.0.4.1/zip"."type": "zip"
  },
  "source_files": "Bin_YYModel_1. 0.4.1 / Headers / *"."public_header_files": Bin_YYModel_1. 0.4.1 / Headers / *. "h"."vendored_libraries": "Bin_YYModel_1. 0.4.1 / *. A"
}
Copy the code

Bin_yymodel_1.0.4.1: After binary components are made, the directory will be changed to the corresponding component library + version number to identify the corresponding version.

Seven, shell engineering separation

Shell project as the name implies, the original project code is removed, resulting in an empty shell, only some project configuration options and dependency library management files.

Because automatic integration involves version number increment, machine modification of the project configuration class file is required. If new service PR is added during binary creation, the commit tree forks, resulting in conflicts and integration failure. After pulling out the shell project, our shell is only concerned with configuration options modification (rarely), and depends on version number changes. The normal PR process of the business code is moved to its own business component git to eliminate human-machine conflicts.

The significance of shell engineering separation is as follows:

  • Paving the way for automatic integration to avoid business PR and machine conflict.
  • Improved efficiency, subsequent Pods move code to Pods faster than Proj moves code to Pods.
  • Making binary components, speeding up compilation speed and improving research and development efficiency

Eight, multiple sets of completely isolated environment

1. What is multiple completely isolated environments

At present, we provide three sets of binary component environment, Dev, Debug_iPhoneos and Release_iPhoneos, respectively corresponding to the developer’s environment Dev, Ci packaging use Debug, Release, three sets of environment completely isolated, Corresponding to different private source warehouse, binary file storage services, and do not interfere with each other. The default is the Dev environment.

  1. Dev is Deubg Settings compiled x86_64 armv7 arm64.

A. the binary file server: http://xxx:10240/frameworks/ b. Binary private source repository: https://xxx/binary_spec_dev 2. Debug_iPhoneos is Deubg set to compile armv7 arm64. A. the binary file server: http://xxx:9192/frameworks/ b. 3. Release_iPhoneos Release set to armv7 arm64. A. the binary file server: http://xxx:20480/frameworks/ b. Binary private source repository: https://xxx/binary_spec_release_iPhoneos

2. Why more sets

Our daily CI packaging is also divided into Debug and Release.

Debug, often referred to as a Debug version, is a combination of a series of compilation options that often include debugging information and provide developers with powerful application debugging capabilities without any optimization.

A Release is usually called a Release and is intended for use by the user. Generally, customers are not allowed to debug on a Release. Therefore, no debugging information is saved, and it is often optimized to achieve minimum code and optimal speed. Provide convenience for users’ use.

Dev is designed for developers, while x86_64 is a 64-bit processor for the x86 architecture, iPhone5s and above, which aims to solve the efficiency problem of developers’ build.

Configuration, development and use

1. Use binary component configuration

1.1. Local configuration file – Podfile_local

The local component configuration file, Podfile_local, currently supports most functions under Podfile. You can put some locally configured statements into Podfile_local.

Scene:

  1. You do not want to transfer your local source/binary configuration to a remote repository.
  2. Avoid modifying the Podfile file directly, causing code conflicts or miscommits.
  3. Pod opens multithreading to speed up Pod speed
Usage:

In the same directory as Podfile, add a new Podfile_local file

#target 'Seeyou' do different items notice to change the value of Seeyou
#:path => '.. /IMYYQHome', as used in podfile

plugin 'cocoapods-imy-bin'
If you want to enable binary plugins, remove the following comment
# use_binaries!

# Need to replace components in Podfile to write here
Component library dependencies written here are switched to [source] dependencies by default
target 'Seeyou' do
  # Local library reference
  #pod 'IMYYQHome', :path => '.. /IMYYQHome'

  Override, custom component
  #pod 'IMYVendor', :podspec => 'http://overwrite, custom /'
end
Copy the code
The previous pod update --no-repo-update command was prefixed`bin`becomeCopy the code
pod bin update --no-repo-update
Copy the code

​ or

pod bin install
Copy the code

Pod install/update command parameters are supported

Configuration options in Podfile_local take precedence over Podfile, support the same configuration statements as Podfile, and support pre_install or post_install.

If you want to use binary components to speed up compilation

Uncomment these two lines of code. See the following for a detailed command explanation
plugin 'cocoapods-imy-bin'
use_binaries!
Copy the code

If you are not comfortable with using Podfile_local, you can write the command in your Podfile, and don’t need to add bin to pod, which is still pod update/install.

2, make binary configuration file – binarchive. json

Some components that do not need to be binary components can be added by adding a binarchive. json file to the Podspec sibling directory.

{
  # Whitelist
    "archive-white-pod-list" : [
        "Seeyou"."IMYFoundation"."IMYPublic"].Components whose source code is stored in Git are made into binary components
    "ignore-git-list": [
        "git@xxx:Github-iOS"].Components whose source code is stored over HTTP are made as binary components
    "ignore-http-list": [
        "https://xxx/Github-iOS"]}Copy the code

X. Gitlab-CI configuration

1. What is it?

GitLab CI is GitLab’s built-in continuous integration tool. You only need to create a. Gitlab-ci. yml file in the root directory of the repository and configure GitLab Runner. On each commit, GitLab will automatically identify the.gitlab-ci.yml file and execute the script using gitLab Runner. Can be provisionally understood as Jenkins microservices.

Currently we are considering using Gitlab-CI to configure ‘packaging capabilities’ for each component independently, to do binary components or other automated tasks.

2. What can they bring to us

  1. Functions such as OC-Lint or file monitoring can be independently checked each time the code is submitted
  2. Provide up-to-date binaries for each individual component in a timely manner
  3. Speed up and stabilize each Ci packing speed
  4. Provides automated testing and monitoring functions independently.

5. Thinking about the project

Our underlying library is relatively stable and standardized. At present, there are some upper-layer components. Due to the branch changes of the underlying components that they depend on, the binary components produced may not be all of the latest version, which is the biggest factor that has not been promoted strongly so far. This is probably the case with most corporate projects.

Would it be best to maintain the version dependencies of each component, or is there a better solution?

The current implementation scheme is:

  • At present, a scheduled build task is deployed on the CI machine of CI_3. Dev development and Debug_iPhoneos environment are built every 1 hour and every half hour from 9:00 to 19:00 on weekdays, and the binary components that need to be made will be generated, and the compilation products such as IPA/DSYM/Archive will not be generated.
  • If the Gitlab-CI deployment is complete, scripts can be written to trigger the two related Jenkins jobs to execute the build task when the code is submitted to avoid some business components failing to compile independently and relying on code that is not up to date.

Eleven, binary does not switch source library, program without re-run debugging ability

After reading the articles about the things behind the zsource command of Meituan iOS project, I was inspired. However, Meituan has no open source related projects, so I figured out how to achieve the same function as them. After some twists and turns, we finally have our own Zsource.

1. Effect display

Using binary, while it can increase the build speed of a project, creates a new problem: when debugging a project, components that use binary do not see as much debugging information as source code debugging. For example, if the program crashes in the code of a binary component, we can only see the stack information for that component and some unexplained assembly code:

Similar to most componentization solutions in the industry, The Mero App componentization solution provides a mechanism for switching a component from binary to source. The developers of Project Meiro can switch the source and binary states of components using a series of configurations and commands, but each switch requires re-executing Pod Install. This approach is not a problem in the early stages of componentization. But as the number of components in the App grew, the time to change the state of even one component increased to minutes per pod install. In this way, you have to recompile and run the App every time you switch. When tracking down some occasional crash problems, the development experience is very unfriendly, and it is not good for the quick location and analysis of crash problems.

In order to solve these problems mentioned above, we use CocoaPods plug-in mechanism, pod command for CocoaPods added bin code subcommand, developers can use the binary build project at the same time, very quickly a component out of the source code debugging, The specific effect can be seen in the following screen recording:

2. How to use it

In the function root directory, enter the command:

pod bin code IMYFoundation
Copy the code

IMYFoundation is the name of the component library that needs source debugging. Once successful, step through as usual and the console prints the variables. Let’s have both the convenience of using binary and the ability to debug source code.

Xii. Problems encountered

1. Cross-component macro definitions

Macro definitions that are processed during the precompilation phase are invalid when the component is binaries, especially when some debugging tools that rely on DEBUG macros are not visible after binaries.

Personal advice is to avoid using macro definitions across modules, especially macros that can be replaced by constants or functions. For example, if you have components A and B, B depends on A, they contain the following code:

/ / A # define TDF_THEME_BACKGROUNDCOLOR [[UIColor whiteColor] colorWithAlphaComponent: 0.7] / /. / / B m for use TDF_THEME_BACKGROUNDCOLORCopy the code

Let’s assume that both A and B have been binaries, and let’s assume that we subsequently modify A:

/ / A # define TDF_THEME_BACKGROUNDCOLOR [[UIColor whiteColor] colorWithAlphaComponent: 0.4]Copy the code

Because B has had the TDF_THEME_BACKGROUNDCOLOR macros in packaging precompiled binary is replaced with the [[UIColor whiteColor] colorWithAlphaComponent: 0.7]. Therefore, B will not be aware of this change in A, and we will have to repackage component B to synchronize the change in A, even if B has not changed anything. When there are many components that use the TDF_THEME_BACKGROUNDCOLOR macro, it is easy to miss some components to synchronize.

Future plans

  • Make CocoapoDS-IMy-bin a tool platform
  • Service integrated code check, resource check and other functions
  • The service integrates initial automated testing, performance monitoring and other functions
  • Improve and optimize gITlab-CI component scripts to minimize the range of compilation speed curve
  • Resource sharing, eliminating nearly 34% of the time spent copying Pods Resource, further reducing compile time to under a minute.
  • Create a more powerful platform tool library Github open source address

Xiv. Summary and thanks

After learning Ruby to write cocoapoDS-IMy-bin plug-in, after the efforts to finally complete the first stage of the idea, which involves the details of the realization of very much, this article is only about a general, there are some ideas to achieve and improve. Overall, CocoapoDS-IMy-bin for research and development efficiency is very considerable.

Thank you for your reading and continuous attention to us Pomelo technology, and also thank you for your trust and support, so that this system can be successfully implemented.

Xv. References

Use ccache to get Xcode running and packing flying

Component binarization practices based on CocoaPods

Behind the Zsource command for Meituan’s iOS project

Github open source address

Welcome to join us pomelo family, can help you push. Send your resume to [email protected]