Now that we know the underlying implementation of objects, classes, methods, etc., let’s take a look at the steps that our App goes through from code completion to startup

In general, an APP goes through two major steps from coding to running, namely compilation and running. This chapter mainly looks at the compilation of APP.

The general steps for compiling are as follows:

  • pretreatment
  • compile
  • assembly
  • link

IOS compiler

IOS code is written directly into machine code by the compiler, and then directly run the machine code on the CPU, so that our app and mobile phone can be more efficient and faster. C, C++, OC and other languages, are used by the compiler, generate related executable files.

On the other hand, scripting languages like Python and Shell use interpreters. The interpreter interprets the executing code at runtime, takes a piece of code, translates it into object code (known as Bytecode), and executes the object code sentence by sentence. That is to say, parsing the code at run time is naturally less efficient than running the compiled executable directly, but after running, you can directly modify the code to see the effect without restarting the compilation, similar to hot update, which can help us shorten the development cycle and function update cycle of the whole program.

To sum up:

  • The advantage of using a compiler to generate machine code execution is high efficiency, but the disadvantage is long debugging cycle
  • The advantage of interpreter execution is that it is easy to write and debug, but the disadvantage is that the execution efficiency is low

The current compiler for Xcode is LLVM(official link). LLVM is a collection of compiler toolchain technologies. And one of the LLD projects is the built-in linker. The compiler compiles each file to generate a Mach-O (executable); The linker merges multiple Mach-O files in the project into one.

LLVM performs the entire compilation process described above, which looks like this:

  • After you write your code, LLVM preprocesses your code, such as embedding macros in the appropriate locations.
  • After preprocessing, LLVM performs lexical and syntax analysis on the code to generate AST. AST is an abstract syntax tree that is structurally leaner than code and faster to traverse, so using AST allows for faster static checking and faster generation of IR (intermediate representation)
  • Finally, THE AST generates IR, which is a language that is more similar to machine code, except that it is platform-independent and multiple machine codes can be generated from IR for different platforms. For iOS, the executable generated by IR is Mach-O.

pretreatment

Create a project and use clang-e main.m to view the work done during the preprocessing phase

#import <Foundation/Foundation.h>
#define DEFINEEight 8

int main(){
    @autoreleasepool {
        int eight = DEFINEEight;
        int six = 6;
        NSString* site = [[NSString alloc] initWithUTF8String:"starming"];
        int rank = eight + six;
        NSLog(@"%@ rank %d", site, rank);
    }
    return 0;
}
Copy the code
# 10 "main.m"
# 1 "./AppDelegate.h" 1
# 11 "./AppDelegate.h"
@interface AppDelegate : UIResponder <UIApplicationDelegate>
@end
# 11 "main.m" 2
int main(int argc, char * argv[]) {
    @autoreleasepool {
        int eight = 8;
        int six = 6;
        NSString* site = [[NSString alloc] initWithUTF8String:"starming"];
        int rank = eight + six;
        NSLog(@"%@ rank %d", site, rank);
    }
    return 0;
}
Copy the code

The main pretreatment rules are as follows:

  • Delete all#defineAnd expand all the macro definitions. The macro definitions used in the source code will be replaced with the corresponding code
  • Inserts the included file into the precompiled instruction(#include)Location (this process is recursive)
  • Delete all comments: //, /* */ etc
  • Add line numbers and file name identifiers so that at compile time the compiler can generate line numbers for debugging and line numbers that display warnings and errors at compile time
  • Keep all#pragmaCompiler instructions, because the compiler needs to use them

When we can’t determine whether the macro definition is correct or whether the header file is included, we can look at the precompiled file to determine the problem

compile

The process of compilation is to perform a series of lexical analysis, syntax analysis, semantic analysis and optimization of the pre-processed file to produce the corresponding assembly code file, this process is often the core part of our entire program construction.

Lexical analysis

Clang-xclang-dump-tokens main.m is used for lexical analysis and the following results are obtained

at The '@'	 [StartOfLine]	Loc=<./AppDelegate.h:11:1>
identifier 'interface'		Loc=<./AppDelegate.h:11:2>
identifier 'AppDelegate'	 [LeadingSpace]	Loc=<./AppDelegate.h:11:12>
colon ':'	 [LeadingSpace]	Loc=<./AppDelegate.h:11:24>
identifier 'UIResponder'	 [LeadingSpace]	Loc=<./AppDelegate.h:11:26>
less '<'	 [LeadingSpace]	Loc=<./AppDelegate.h:11:38>
identifier 'UIApplicationDelegate'		Loc=<./AppDelegate.h:11:39>
greater '>'		Loc=<./AppDelegate.h:11:60>
at The '@'	 [StartOfLine]	Loc=<./AppDelegate.h:14:1>
identifier 'end'		Loc=<./AppDelegate.h:14:2>
int 'int'	 [StartOfLine]	Loc=<main.m:14:1>
identifier 'main'	 [LeadingSpace]	Loc=<main.m:14:5>
l_paren '('		Loc=<main.m:14:9>
int 'int'		Loc=<main.m:14:10>
identifier 'argc'	 [LeadingSpace]	Loc=<main.m:14:14>
comma ', '		Loc=<main.m:14:18>
char 'char'	 [LeadingSpace]	Loc=<main.m:14:20>
star The '*'	 [LeadingSpace]	Loc=<main.m:14:25>
identifier 'argv'	 [LeadingSpace]	Loc=<main.m:14:27>
l_square '['		Loc=<main.m:14:31>
r_square '] '		Loc=<main.m:14:32>
r_paren ') '		Loc=<main.m:14:33>
l_brace '{'	 [LeadingSpace]	Loc=<main.m:14:35>

...
Copy the code

In this step, the code in the source file is converted into a special markup stream. The source code is divided into characters and words one by one. The corresponding source file and the specific line number are marked in the Loc at the end of the line, which is convenient to locate the problem when an error is reported

Syntax analysis

Run the clang-xclang-ast – dump-fsyntax-only main.m command to analyze the syntax. The result is as follows

. | `-PointerType 0x7f9824831b10'char *'
|   `-BuiltinType 0x7f9824830ca0 'char'
|-TypedefDecl 0x7f9825006458 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list 'struct __va_list_tag [1]'
| `-ConstantArrayType 0x7f9825006400 'struct __va_list_tag [1]' 1
|   `-RecordType 0x7f9825006280 'struct __va_list_tag'
|     `-Record 0x7f9825006200 '__va_list_tag'
|-ObjCInterfaceDecl 0x7f98250064a8 <./AppDelegate.h:11:1, line:14:2> line:11:12 AppDelegate
|-FunctionDecl 0x7f98250067e0 <main.m:14:1, line:23:1> line:14:5 main 'int (int, char **)'
| |-ParmVarDecl 0x7f98250065b8 <col:10, col:14> col:14 argc 'int'
| |-ParmVarDecl 0x7f98250066d0 <col:20, col:32> col:27 argv 'char **':'char **'
| `-CompoundStmt 0x7f9825006f28 <col:35, line:23:1>
|   |-ObjCAutoreleasePoolStmt 0x7f9825006ee0 <line:15:5, line:21:5>
|   | `-CompoundStmt 0x7f9825006eb8 <line:15:22, line:21:5>
|   |   |-DeclStmt 0x7f9825006960 <line:16:9, col:32>
|   |   | `-VarDecl 0x7f98250068e0 <col:9, line:12:21> line:16:13 used eight 'int' cinit
|   |   |   `-IntegerLiteral 0x7f9825006940 <line:12:21> 'int' 8
|   |   |-DeclStmt 0x7f9825006a10 <line:17:9, col:20>
|   |   | `-VarDecl 0x7f9825006990 <col:9, col:19> col:13 used six 'int' cinit
|   |   |   `-IntegerLiteral 0x7f98250069f0 <col:19> 'int' 6
|   |   `-DeclStmt 0x7f9825006b30 <line:19:9, col:31>
|   |     `-VarDecl 0x7f9825006a40 <col:9, col:28> col:13 used rank 'int' cinit
|   |       `-BinaryOperator 0x7f9825006b10 <col:20, col:28> 'int' '+'
|   |         |-ImplicitCastExpr 0x7f9825006ae0 <col:20> 'int' <LValueToRValue>
|   |         | `-DeclRefExpr 0x7f9825006aa0 <col:20> 'int' lvalue Var 0x7f98250068e0 'eight' 'int'
|   |         `-ImplicitCastExpr 0x7f9825006af8 <col:28> 'int' <LValueToRValue>
|   |           `-DeclRefExpr 0x7f9825006ac0 <col:28> 'int' lvalue Var 0x7f9825006990 'six' 'int'
|   `-ReturnStmt 0x7f9825006f18 <line:22:5, col:12>
|     `-IntegerLiteral 0x7f9825006ef8 <col:12> 'int' 0
`-FunctionDecl 0x7f9825006bd0 <line:20:9> col:9 implicit used NSLog 'void (id, ...) ' extern
  |-ParmVarDecl 0x7f9825006c68 <<invalid sloc>> <invalid sloc> 'id':'id'
  `-FormatAttr 0x7f9825006cd0 <col:9> Implicit NSString 1
  
...
Copy the code

This step parses the stream of tokens generated by lexical analysis into an abstract syntax tree — AST, in which each node is also marked for its position in the source code.

Static analysis

After converting the source code into an abstract syntax tree, the compiler can parse the tree. Static analysis improves code quality by checking for errors such as methods that are called but not defined, variables that are defined but not used, etc. You can also use Xcode’s own static analysis tool (Product -> Analyze).

Type checking

Types are generally divided into two categories: dynamic and static. Dynamic checks are performed at runtime, and static checks are performed at compile time. In the past, you could write code that sent any message to any object, and only at runtime would the object be checked to see if it could respond to those messages. Because this type of checking is done only at run time, it is called dynamic typing.

As for static typing, this is checked at compile time. When using ARC in code, the compiler does a lot of type checking at compile time: the compiler needs to know which object to use and how to use it.

At this stage, Clang checks, most commonly to see if the program sends the right message to the right object and calls the normal function at the right value. If you send a Hello message to a simple NSObject* object, clang will report an error. Similarly, if you set a property to an object that doesn’t match its own type, the compiler will warn you that it might be used incorrectly.

Other analysis

Other analysis ObjCUnusedIVarsChecker. CPP is used to check whether there are defined, but have never used a variable. Objcselfinitchecker. CPP checks if [self initWith…] has been called before calling self in your initialization method. Or super init.

The resources

Clang static analysis

LLVM IRintermediate

Use the clang-O3 -s -emit- LLVM main.m -o main.ll command to generate LLVM intermediate IR(generate main.ll file). IR is the output input at the front end of the compilation process and at the back end.

; ModuleID = 'main.m'
source_filename = "main.m"
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
target triple = "X86_64 - apple - macosx10.13.0"

%struct.__NSConstantString_tag = type { i32*, i32, i8*, i64 }

@__CFConstantStringClassReference = external global [0 x i32]
@.str = private unnamed_addr constant [3 x i8] c"%d\00", section "__TEXT,__cstring,cstring_literals", align 1
@_unnamed_cfstring_ = private global %struct.__NSConstantString_tag { i32* getelementptr inbounds ([0 x i32], [0 x i32]* @__CFConstantStringClassReference, i32 0, i32 0), i32 1992, i8* getelementptr inbounds ([3 x i8], [3 x i8]* @.str, i32 0, i32 0), i64 2 }, section "__DATA,__cfstring", align 8

; Function Attrs: ssp uwtable
define i32 @main(i32, i8** nocapture readnone) local_unnamed_addr # 0 {
  %3 = tail call i8* @objc_autoreleasePoolPush() # 2
  notail call void (i8*, ...) @NSLog(i8* bitcast (%struct.__NSConstantString_tag* @_unnamed_cfstring_ to i8*), i32 1)
  tail call void @objc_autoreleasePoolPop(i8* %3)
  ret i32 0
}

declare i8* @objc_autoreleasePoolPush() local_unnamed_addr

declare void @NSLog(i8*, ...) local_unnamed_addr # 1

declare void @objc_autoreleasePoolPop(i8*) local_unnamed_addr

attributes #0 = { ssp uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="penryn" "Target - the features" = "+ cx16, + FXSR, + MMX and sse +, + sse2, + sse3, + sse4.1, + ssse3, + x87" "unsafe - the fp - math" = "false" "use-soft-float"="false" }
attributes #1 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="penryn" "Target - the features" = "+ cx16, + FXSR, + MMX and sse +, + sse2, + sse3, + sse4.1, + ssse3, + x87" "unsafe - the fp - math" = "false" "use-soft-float"="false" }
attributes #2 = { nounwind }! llvm.module.flags = ! {! 0,! 1,! 2,! 3,! 4,! 5,! 6}! llvm.ident = ! {! 7}! 0 =! {i32 1, !"Objective-C Version", i32 2} ! 1 =! {i32 1, !"Objective-C Image Info Version", i32 0} ! 2 =! {i32 1, !"Objective-C Image Info Section",!"__DATA,__objc_imageinfo,regular,no_dead_strip"}
!3 = !{i32 4, !"Objective-C Garbage Collection", i32 0} ! 4 =! {i32 1, !"Objective-C Class Properties", i32 64} ! 5 =! {i32 1, !"wchar_size", i32 4} ! 6 =! {i32 7, !"PIC Level", i32 2} ! 7 =! {!"Apple LLVM version 9.1.0 (clang-902.0.39.2)"}

Copy the code

LLVM optimization

Using the clang-emit – LLVM -c main.m -o main. BC command, LLVM is used to optimize the code.

  • For global variable optimization, circular optimization, tail recursive optimization and so on.
  • You can also set optimization levels of -01, -03, -0S in Xcode compilation Settings, and write your own Pass.
  • Pass is a node that optimizes LLVM, and each node does something that adds up to a complete optimization and transformation of LLVM.
  • If bitcode is enabled, Apple will make further optimization. If there is a new backend architecture, it can still be generated with this optimized Bitcode.

Generating assembly code

Using clang-s-fobjc-arc main.m -o main. S generates the corresponding assembly code

At this point, the compilation phase is complete, converting the written code into assembly code that the machine can recognize

assembly

An assembler converts assembly code into instructions that can be executed by a machine. Almost every assembly statement corresponds to a machine instruction. Therefore, the assembly process of the assembler is relatively simple compared with the compiler. It has no complex syntax, no semantics, and no instruction optimization. It is only translated according to the comparison table of the assembly instruction and the machine instruction.

Use clang-fmodules -c main.m -o main.o to generate the corresponding object file

Programs built using Xcode will find this file in the DerivedData directory, as shown below

link

Links are mainly divided into static links and dynamic links. Links in the compiler stage are static links, and relevant dynamic links will be explained in the next chapter of App startup

This stage is to link the object file generated in the previous stage with the referenced static library, and finally generate the executable file. The linker solves the link between the object file and the library.

Using Clang main.m to generate the executable, you can see that the executable is of the Mach-O type, which is the same type of executable on both MAC OS and iOS platforms.

At this point, the compilation process is complete and the executable file Mach -o is generated

What does the linker do at compile time?

The contents of the Mach-O file are mainly code and data: code is the definition of a function; Data is the definition of a global variable, including its initial value. Instances of both code and data need to be associated by symbols.

Why is that? Because the code in the Mach-O file, such as the sequence of machine instructions generated by if, for, and while, is storing the data to be manipulated somewhere, variable symbols need to be bound to the data’s storage address. The code you write also references other code, and the referenced function symbol needs to be bound to the address of that function.

The role of the linker is to perform tasks such as binding variables, function symbols, and their addresses. And when we talk about symbols, we can think of variable names and function names.

Why sign binding

  • If addresses and symbols are not bound, to let the machine know what memory address you are manipulating, you need to set the memory address for each instruction when you write the code.
  • Readability and maintainability will be poor, and the address will need to be maintained after modifying the code
  • You need to write multiple copies of code for different platforms, which could have been compiled in a high-level language
  • Equivalent to writing assembly directly

Why merge multiple Mach-O files in your project into one

Variables and interface functions between files in the project are interdependent, so you need to bind the symbols and addresses of multiple Mach-O files generated in the project through a linker.

Without this binding process, a mach-O file generated from a single file would not work properly. Because if the runtime encounters a call to a function implemented in another file, the address of the calling function will not be found and execution cannot continue.

In the process of linking multiple object files, the linker creates a symbol table that records all defined and undefined symbols. If the same symbols appear during linking, an error message “LD: dumplicate Symbols” will appear. If no symbols are found in another object file, the error message “Undefined Symbols” is displayed.

What are the main things the linker does with the code

  • Look in the project file for variables that are not defined in the object code file.
  • Scan the different files in the project, collect all symbol definitions and reference addresses, and place them in the global symbol table.
  • Calculate the length and position after merging, generate segments of the same type for merging, and establish binding.
  • Address relocation of variables in different files in the project.

How does the linker get rid of useless functions and keep mach-O size

When the linker collates function calls, it follows each reference from the main function and marks it as live. After following, functions that are not marked live are useless. The linker can then automatically remove unwanted code by turning on Dead Code stripping switches. Also, this switch is on by default.

Mach-OAnalysis of the

Mach-O is short for Mach Object file format, which is the MAC and iOS executable file format, Similar to Portable PE Format on Windows and Executable and Linking Format on Linux

Mach-o is the native executable format for binary files in OS X and is the preferred format for transferring code. The executable format determines the order in which the code and data in the binary file are read into memory. The order of code and data affects memory usage and paging activity, which directly affects program performance.

The Mach-O binaries are organized into segments. Each section contains one or more sections. The size of a segment is measured by the number of bytes of all the parts it contains, rounded to the next virtual memory page boundary. Therefore, a segment is always 4096 bytes or a multiple of 4 kilobytes, where 4096 bytes is the minimum size.

Common Mach-O files

1. Target file:.o

2. Library file:.a. dylib Framework

3. Executable file: dyld.dsym

Mach-o file format

MachO can be a multi-schema binary, called a “universal binary.”

Main architecture armv7, armv7s arm64, i386, x86_64, including most of the iPhone using arm64

Universal binary is a type of program code introduced by Apple. Binaries that work with multiple schemas at the same time

  • Optimal performance for multiple architectures simultaneously in the same package.
  • Because of the need to store multiple types of code, general-purpose binary applications are generally larger than single-platform binary applications.
  • However, because the two architectures share common non-execution resources, there are not twice as many as in a single version.
  • And because only a portion of the code is called during execution, no extra memory is required to run.

Mach-o file structure

Header

Header contains general information about the binary, byte order, schema type, number of load instructions, and so on. This allows you to quickly verify information such as whether the current file is 32-bit or 64-bit, the corresponding processor, and the file type

You can use otool -v -h a.out to view the structure, or use MachOView to view it directly

Load Commons

Load Commands is a table that contains many things. The content includes the location of the region, symbol table, dynamic symbol table, etc. This section follows the Header and is used to determine the distribution of memory when loading the Mach-O file

LC_LOAD_DYLINKER

LC_LOAD_DYLINKER This field specifies who loaded our MachO. In the next section, we’ll talk about how dyLD loads Mach-O

LC_LOAD_DYLIB

LC_LOAD_DYLIB this field marks the address of all dynamic libraries. Only if there is a mark in LC_LOAD_DYLIB, the dynamic libraries outside MachO (such as Framework) can be correctly referenced by DYLD, otherwise DyLD will not actively load

Data

Data is usually the largest part of the object file, containing seinterfaces specific Data, such as static C strings, OC methods with/without parameters, and C functions with/without parameters.

Contains each segment required for Load Commands, and each segment contains multiple sections. When running an executable, the virtual memory system maps segments to the process’s address space.

Use xcrun size -x -l -m a.out to view the segment content, or MachOView

The name of the meaning
Segment __PAGEZERO The size is 4GB. The first 4GB of the process address space is mapped as unreadable, unwritable, and unexecutable
Segment __TEXT Contains executable code, mapped read-only and executable.
Segment __DATA Contains data that will be changed, mapped in both read-write and non-executable ways.
Segment __LINKEDIT Contains metadata for methods and variables, code signatures, and more.

reference

  • Daming -iOS Master class
  • The story behind DYLD & Source code Analysis
  • IOS Compilation process
  • IOS reverse (5)- I wonder how MachO can claim to understand DYLD
  • In-depth understanding of the iOS App startup process
  • IOS learning in-depth understanding of the program compilation process