LLVM is a free software project that is a compiler infrastructure, written in C++, containing a series of modular compiler components and toolchains for developing compiler front and back ends. It is a program written for any programming language that uses virtual techniques to create compile-time, link-time, run-time, and “idle time” optimizations. It was originally implemented in C/C++, It currently supports ActionScript, Ada, D, Fortran, GLSL, Haskell, Java bytecode, Objective-C, Swift, Python, Ruby, Crystal, Rust, Scala, C# and more.
— Wikipedia
In 2000, Chris Lattner of the University of Illinois at Urbana-Champaign (UIUC), one of the world’s leading public research universities, tweeted Clattner_llvm (@clattner_llVM) developed a compiler development tool suite called Low Level Virtual Machine. Later, it covers an increasingly wide range and can be used for general compilers, JIT compilers, assemblers, debuggers, static analysis tools and a series of programming language related work. So the abbreviation LLVM became the official name. Chris Lattner later developed Clang, allowing LLVM to directly challenge GCC. In 2012, LLVM won the ACM Software System Award, along with UNIX, WWW, TCP/IP, Tex, JAVA, etc.
An in-depth look at Clang/LLVM for iOS
Chris Lattner, the creator of Swift, is a familiar name to iOS developers. LLVM, which he and his team developed, has become a critical underlying infrastructure for iOS and the entire macOS ecosystem. Although Lattner himself has gone to Google to work on ARTIFICIAL intelligence, it’s important for iOS developers to understand and master some of the basics of LLVM.
LLVM que
The official website for LLVM is llvm.org/. As you can see from the official website, LLVM is actually a collection of compiled components. And the Clang (pronounced krone) is the front end. The front end here is not a front end concept like HTML5. With that said, let’s briefly review the design of a traditional compiler.
Traditional compiler
Before LLVM, the GCC compiler was the most widely used, and GCC still plays an important role today.
Compiler Front End
The task of the front end of the compiler is to parse the source code, which includes the following three processes
- Lexical analysis
- Syntax analysis
- Semantic analysis
Check the source code for errors and build an Abstract Syntax Tree (AST). The LLVM front end also generates intermediate representation (IR) code.
The Optimizer Optimizer
The optimizer is responsible for various optimizations. Improve code runtime, such as eliminating redundant calculations.
Compiler Back End
Map code to the target instruction set. Generate machine language and perform machine-specific code optimizations. Some sources also refer to the back end of the compiler as a Code Generator.
As you can see from the above, the traditional compiler architecture is so coupled between the front end and the back end that supporting a new programming language, or a new target platform, can be a lot of work.
LLVM architecture
The most important thing that makes LLVM a mainstay of compilers is its use of a common code representation, called IR. With IR, LLVM can write a separate front end for any programming language and a separate back end for any hardware architecture.
LLVM Architecture Overview
Clang
Clang is a subproject of the LLVM project. It is a lightweight compiler based on the LLVM architecture that was created as an alternative to GCC to provide faster compilation times. It is the compiler responsible for compiling C, C++, and Objective-C, and it belongs to the compiler front end of the entire LLVM architecture. For us, studying Clang gives us a deeper understanding of the process from source code to assembly to machine code.
Clang’s website is clang.llvm.org/
Clang has the following advantages over GCC
- Fast compilation: On some platforms, Clang compiles significantly faster than GCC (OC compiles 3 times faster than GGC in Debug mode)
- Small footprint: The AST generated by Clang takes up about one-fifth of the memory of GCC
- Modular design: Clang uses a library-based modular design for easy IDE integration and reuse for other purposes
- Diagnostic information is readable: During compilation, Clang creates and retains a large amount of detailed metadata for debugging and error reporting
- The design is clear and simple, easy to understand, and easy to expand and enhance
That is, LLVM in a broad sense refers to the entire LLVM architecture, while LLVM in a narrow sense refers to the LLVM back end. The LLVM back end consists of code optimization (optimizer) and object code generation (back end).
The compilation process in iOS
Before exploring the compile process, let’s print out the compile phase of the source code with a command
Clang-ccc-print-phases File nameCopy the codeThe output is as follows
0: input, "main.m", objective-c
1: preprocessor, {0}, objective-c-cpp-output
2: compiler, {1}, ir
3: backend, {2}, assembler
4: assembler, {3}, object
5: linker, {4}, image
6: bind-arch, "x86_64", {5}, image
Copy the codeAs can be seen from the results, there are altogether 7 stages.
- Input: Indicates that the main. M file is entered in OC format
- Preprocessor: The preprocessing stage, which includes macro replacement and header file import
- Compiler: In the stage of compilation, lexical analysis, grammar analysis, semantic analysis, and finally generate IR
- Backend: LLVM is optimized one by one to generate assembly code.
- Assembler: assembler that generates an object file
- Linker: Link, link required dynamic and static libraries, generate executable files
- Bind-arch: schema binding that generates executable files for different schemas
Pretreatment stage
Clang -e File name -o Output file nameCopy the codeWe can use the clang command above to execute the preprocessing phase and output it to a file.
We can test it by using the following code in main.m
// main.m
#include <stdio.h>
#define C 30
int main(int argc, const char * argv[]) {
@autoreleasepool {
int a = 10;
int b = 20;
printf("%d", a + b + C);
}
return 0;
}
Copy the codeThen perform
clang -E main.m -i main2.m
Copy the codeCheck the contents of the main2.m file:
As you can see, the macro has been replaced and the stdio.h header has been imported.
If a command like typedef is also replaced during preprocessing, we can test it:
#include <stdio.h>
#define C 30
typedef int JH_INT_64;
int main(int argc, const char * argv[]) {
@autoreleasepool {
JH_INT_64 a = 10;
JH_INT_64 b = 20;
printf("%d", a + b + C);
}
return 0;
}
Copy the codeIt’s the same command
clang -E main.m -o main3.m
Copy the codeThe results are as follows
As you can see, a typedef is not replaced because it is just an alias and does not fall within the scope of macro definitions.
Compilation phase
After the pre-processing phase, the source file needs to be compiled to generate the corresponding intermediate code IR. But we know that actually the whole compilation stage is divided into two stages: lexical analysis and grammatical analysis.
Lexical analysis
After the preprocessing is complete, a lexical analysis is performed, and the code is sliced into tokens such as parentheses, equals signs, and strings.
The command for lexical analysis is as follows
Clang-fmodules-fsyntax-only - xclang-dump -tokens filenameCopy the codeLet’s test that out
clang -fmodules -fsyntax-only -Xclang -dump-tokens main.m
Copy the codeThe terminal output is as follows
annot_module_include '#include
#define C 30 typedef int JH_INT_64; int main(int argc, const char * argv[]) { @autoreleasepool { JH_INT_64 a '
Loc=<main.m:9:1>
typedef 'typedef' [StartOfLine] Loc=<main.m:12:1>
int 'int' [LeadingSpace] Loc=<main.m:12:9>
identifier 'JH_INT_64' [LeadingSpace] Loc=<main.m:12:13>
semi '; ' Loc=<main.m:12:22>
int 'int' [StartOfLine] Loc=<main.m:13:1>
identifier 'main' [LeadingSpace] Loc=<main.m:13:5>
l_paren '(' Loc=<main.m:13:9>
int 'int' Loc=<main.m:13:10>
identifier 'argc' [LeadingSpace] Loc=<main.m:13:14>
comma ', ' Loc=<main.m:13:18>
const 'const' [LeadingSpace] Loc=<main.m:13:20>
char 'char' [LeadingSpace] Loc=<main.m:13:26>
star The '*' [LeadingSpace] Loc=<main.m:13:31>
identifier 'argv' [LeadingSpace] Loc=<main.m:13:33>
l_square '[' Loc=<main.m:13:37>
r_square '] ' Loc=<main.m:13:38>
r_paren ') ' Loc=<main.m:13:39>
l_brace '{' [LeadingSpace] Loc=<main.m:13:41>
at The '@' [StartOfLine] [LeadingSpace] Loc=<main.m:14:5>
identifier 'autoreleasepool' Loc=<main.m:14:6>
l_brace '{' [LeadingSpace] Loc=<main.m:14:22>
identifier 'JH_INT_64' [StartOfLine] [LeadingSpace] Loc=<main.m:15:9>
identifier 'a' [LeadingSpace] Loc=<main.m:15:19>
equal '=' [LeadingSpace] Loc=<main.m:15:21>
numeric_constant '10' [LeadingSpace] Loc=<main.m:15:23>
semi '; ' Loc=<main.m:15:25>
identifier 'JH_INT_64' [StartOfLine] [LeadingSpace] Loc=<main.m:16:9>
identifier 'b' [LeadingSpace] Loc=<main.m:16:19>
equal '=' [LeadingSpace] Loc=<main.m:16:21>
numeric_constant '20' [LeadingSpace] Loc=<main.m:16:23>
semi '; ' Loc=<main.m:16:25>
identifier 'printf' [StartOfLine] [LeadingSpace] Loc=<main.m:17:9>
l_paren '(' Loc=<main.m:17:15>
string_literal '"%d"' Loc=<main.m:17:16>
comma ', ' Loc=<main.m:17:20>
identifier 'a' [LeadingSpace] Loc=<main.m:17:22>
plus '+' [LeadingSpace] Loc=<main.m:17:24>
identifier 'b' [LeadingSpace] Loc=<main.m:17:26>
plus '+' [LeadingSpace] Loc=<main.m:17:28>
numeric_constant '30' [LeadingSpace] Loc=<main.m:17:30 <Spelling=main.m:11:11>>
r_paren ') ' Loc=<main.m:17:31>
semi '; ' Loc=<main.m:17:32>
r_brace '} ' [StartOfLine] [LeadingSpace] Loc=<main.m:18:5>
return 'return' [StartOfLine] [LeadingSpace] Loc=<main.m:19:5>
numeric_constant '0' [LeadingSpace] Loc=<main.m:19:12>
semi '; ' Loc=<main.m:19:13>
r_brace '} ' [StartOfLine] Loc=<main.m:20:1>
eof ' ' Loc=<main.m:20:2>
Copy the codeAs you can see, the code in main.m is broken down into tokens one by one.
Syntax analysis
Lexical analysis is followed by grammatical analysis. Its job is to verify that the syntax is correct. On the basis of lexical analysis, word sequences are combined into various lexical phrases. Such as programs, statements, expressions, etc., and then form all the nodes into Abstract Syntax Tree (AST).
Parsing is named as follows
Clang-fmodules-fsyntax-only - xclang-ast -dump file nameCopy the codeLet’s test that out
clang -fmodules -fsyntax-only -Xclang -ast-dump main.m
Copy the codeThe terminal displays the following result
TranslationUnitDecl 0x7fa0aa017408 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x7fa0aa017ca0 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x7fa0aa0179a0 '__int128'
|-TypedefDecl 0x7fa0aa017d08 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x7fa0aa0179c0 'unsigned __int128'
|-TypedefDecl 0x7fa0aa017da0 <<invalid sloc>> <invalid sloc> implicit SEL 'SEL *'
| `-PointerType 0x7fa0aa017d60 'SEL *'
| `-BuiltinType 0x7fa0aa017c00 'SEL'
|-TypedefDecl 0x7fa0aa017e78 <<invalid sloc>> <invalid sloc> implicit id 'id'
| `-ObjCObjectPointerType 0x7fa0aa017e20 'id'
| `-ObjCObjectType 0x7fa0aa017df0 'id'
|-TypedefDecl 0x7fa0aa017f58 <<invalid sloc>> <invalid sloc> implicit Class 'Class'
| `-ObjCObjectPointerType 0x7fa0aa017f00 'Class'
| `-ObjCObjectType 0x7fa0aa017ed0 'Class'
|-ObjCInterfaceDecl 0x7fa0aa017fa8 <<invalid sloc>> <invalid sloc> implicit Protocol
|-TypedefDecl 0x7fa0aa0182e8 <<invalid sloc>> <invalid sloc> implicit __NSConstantString 'struct __NSConstantString_tag'
| `-RecordType 0x7fa0aa018100 'struct __NSConstantString_tag'
| `-Record 0x7fa0aa018070 '__NSConstantString_tag'
|-TypedefDecl 0x7fa0aa018380 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x7fa0aa018340 'char *'
| `-BuiltinType 0x7fa0aa0174a0 'char'
|-TypedefDecl 0x7fa0aa054e68 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list 'struct __va_list_tag [1]'
| `-ConstantArrayType 0x7fa0aa054e10 'struct __va_list_tag [1]' 1
| `-RecordType 0x7fa0aa054c90 'struct __va_list_tag'
| `-Record 0x7fa0aa054c00 '__va_list_tag'
|-ImportDecl 0x7fa0aa055670 <main.m:9:1> col:1 implicit Darwin.C.stdio
|-TypedefDecl 0x7fa0aa0556c0 <line:12:1, col:13> col:13 referenced JH_INT_64 'int'
| `-BuiltinType 0x7fa0aa017500 'int'
|-FunctionDecl 0x7fa0aa055980 <line:13:1, line:20:1> line:13:5 main 'int (int, const char **)'
| |-ParmVarDecl 0x7fa0aa055728 <col:10, col:14> col:14 argc 'int'
| |-ParmVarDecl 0x7fa0aa055840 <col:20, col:38> col:33 argv 'const char **':'const char **'
| `-CompoundStmt 0x7fa0aa1dc880 <col:41, line:20:1>
| |-ObjCAutoreleasePoolStmt 0x7fa0aa1dc838 <line:14:5, line:18:5>
| | `-CompoundStmt 0x7fa0aa1dc810 <line:14:22, line:18:5>
| | |-DeclStmt 0x7fa0aa055b60 <line:15:9, col:25>
| | | `-VarDecl 0x7fa0aa055ae0 <col:9, col:23> col:19 used a 'JH_INT_64':'int' cinit
| | | `-IntegerLiteral 0x7fa0aa055b40 <col:23> 'int' 10
| | |-DeclStmt 0x7fa0aa1dc5d0 <line:16:9, col:25>
| | | `-VarDecl 0x7fa0aa055b88 <col:9, col:23> col:19 used b 'JH_INT_64':'int' cinit
| | | `-IntegerLiteral 0x7fa0aa1dc200 <col:23> 'int' 20
| | `-CallExpr 0x7fa0aa1dc7b0 <line:17:9, col:31> 'int'
| | |-ImplicitCastExpr 0x7fa0aa1dc798 <col:9> 'int (*)(const char *, ...) ' <FunctionToPointerDecay>
| | | `-DeclRefExpr 0x7fa0aa1dc5e8 <col:9> 'int (const char *, ...) ' Function 0x7fa0aa1dc228 'printf' 'int (const char *, ...) '
| | |-ImplicitCastExpr 0x7fa0aa1dc7f8 <col:16> 'const char *' <NoOp>
| | | `-ImplicitCastExpr 0x7fa0aa1dc7e0 <col:16> 'char *' <ArrayToPointerDecay>
| | | `-StringLiteral 0x7fa0aa1dc648 <col:16> 'char [3]' lvalue "%d"
| | `-BinaryOperator 0x7fa0aa1dc748 <col:22, line:11:11> 'int' '+'
| | |-BinaryOperator 0x7fa0aa1dc708 <line:17:22, col:26> 'int' '+'
| | | |-ImplicitCastExpr 0x7fa0aa1dc6d8 <col:22> 'JH_INT_64':'int' <LValueToRValue>
| | | | `-DeclRefExpr 0x7fa0aa1dc668 <col:22> 'JH_INT_64':'int' lvalue Var 0x7fa0aa055ae0 'a' 'JH_INT_64':'int'
| | | `-ImplicitCastExpr 0x7fa0aa1dc6f0 <col:26> 'JH_INT_64':'int' <LValueToRValue>
| | | `-DeclRefExpr 0x7fa0aa1dc6a0 <col:26> 'JH_INT_64':'int' lvalue Var 0x7fa0aa055b88 'b' 'JH_INT_64':'int'
| | `-IntegerLiteral 0x7fa0aa1dc728 <line:11:11> 'int' 30
| `-ReturnStmt 0x7fa0aa1dc870 <line:19:5, col:12>
| `-IntegerLiteral 0x7fa0aa1dc850 <col:12> 'int' 0
`-<undeserialized declarations>
Copy the codeIf iOS header files are imported when parsing commands are executed, add the specified SDK path to avoid execution failures
The following command specifies the path to the iOS 13.2 emulator SDK.
clang -isysroot / Applications/Xcode. App/Contents/Developer/Platforms/iPhoneSimulator platform/Developer/SDKs/iPhoneSimulator13.2. The SDK -fmodules -fsyntax-only -Xclang -ast-dump main.mCopy the code
After the output of the AST abstract syntax tree, there is an obvious tree structure that is not difficult to read.
|-FunctionDecl 0x7fa0aa055980 <line:13:1, line:20:1> line:13:5 main 'int (int, const char **)'
Copy the code- At the beginning of
FunctionDeclRepresentation is a function declaration node. It is followed by a memory address, which is really the address of the function in the stack space. The contents inside Angle brackets indicate the starting range of main, starting at the first character on line 13 and ending at the first character on line 20. And then the backline:13:5Represents the parameter declaration area of the function starting with the fifth character on line 13.
| |-ParmVarDecl 0x7fa0aa055728 <col:10, col:14> col:14 argc 'int'
| |-ParmVarDecl 0x7fa0aa055840 <col:20, col:38> col:33 argv 'const char **':'const char **'
Copy the code- These two nodes are of type
ParmVarDecl, represents the argument node, where the first argument argc is of type int and the second argument argv is const char**.
| `-CallExpr 0x7fa0aa1dc7b0 <line:17:9, col:31> 'int'
Copy the codeCallExprCall a function. At the back of theintRepresents the return value of the calling function. This is actually the source codeprintfThe position of the function.
| | |-DeclStmt 0x7fa0aa055b60 <line:15:9, col:25>
| | | `-VarDecl 0x7fa0aa055ae0 <col:9, col:23> col:19 used a 'JH_INT_64':'int' cinit
| | | `-IntegerLiteral 0x7fa0aa055b40 <col:23> 'int' 10
| | |-DeclStmt 0x7fa0aa1dc5d0 <line:16:9, col:25>
| | | `-VarDecl 0x7fa0aa055b88 <col:9, col:23> col:19 used b 'JH_INT_64':'int' cinit
| | | `-IntegerLiteral 0x7fa0aa1dc200 <col:23> 'int' 20
Copy the codeDeclStmtWe can easily see that we have declared two variables of type int, one of 10 and one of 20.
| | |-ImplicitCastExpr 0x7fa0aa1dc798 <col:9> 'int (*)(const char *, ...) ' <FunctionToPointerDecay>
| | | `-DeclRefExpr 0x7fa0aa1dc5e8 <col:9> 'int (const char *, ...) ' Function 0x7fa0aa1dc228 'printf' 'int (const char *, ...) '
| | |-ImplicitCastExpr 0x7fa0aa1dc7f8 <col:16> 'const char *' <NoOp>
| | | `-ImplicitCastExpr 0x7fa0aa1dc7e0 <col:16> 'char *' <ArrayToPointerDecay>
| | | `-StringLiteral 0x7fa0aa1dc648 <col:16> 'char [3]' lvalue "%d"
Copy the codeImplicitCastExprThe printf function also has implicit conversion operations:int (*)(const char *, ...).- The last line is obviously at the beginning of printf
%dTo receive a value of type int.
| | `-BinaryOperator 0x7fa0aa1dc748 <col:22, line:11:11> 'int' '+'
| | |-BinaryOperator 0x7fa0aa1dc708 <line:17:22, col:26> 'int' '+'
| | | |-ImplicitCastExpr 0x7fa0aa1dc6d8 <col:22> 'JH_INT_64':'int' <LValueToRValue>
| | | | `-DeclRefExpr 0x7fa0aa1dc668 <col:22> 'JH_INT_64':'int' lvalue Var 0x7fa0aa055ae0 'a' 'JH_INT_64':'int'
| | | `-ImplicitCastExpr 0x7fa0aa1dc6f0 <col:26> 'JH_INT_64':'int' <LValueToRValue>
| | | `-DeclRefExpr 0x7fa0aa1dc6a0 <col:26> 'JH_INT_64':'int' lvalue Var 0x7fa0aa055b88 'b' 'JH_INT_64':'int'
| | `-IntegerLiteral 0x7fa0aa1dc728 <line:11:11> 'int' 30
Copy the codeBinaryOperatorRepresents a binary operator. The way to read this is to look at the leaves first, so it translates to the result of (a + b) + 30 as an input to the printf function.
| `-ReturnStmt 0x7fa0aa1dc870 <line:19:5, col:12>
| `-IntegerLiteral 0x7fa0aa1dc850 <col:12> 'int' 0
Copy the codeReturnStmtThe function returns a declaration node. Shows that the result returned is int type 0.
Generate intermediate code IR
After completing the above steps, the intermediate code IR is generated, and the code generator translates the syntax tree from the top down to LLVM IR step by step.
LLVM IR has three forms, but they are essentially equivalent, just as water can be gas, liquid and solid.
- Text: An easy-to-read text format similar to assembly language, extended to LL.
// Generate IR clang-s-fobjc-ARC-emit - LLVM file name in text formatCopy the code- Memory: indicates the memory format
- Bitcode: binary format with extension.bc
// Generate bitcode IR clang-c-fobjc-ARC-emit - LLVM file nameCopy the codeIR basic syntax:
@ global id % Local ID ALloca open space align memory align I32 32 bit, 4 bytes Store write memory Load read data call call function RET return
So let’s actually test it out
// main.m
#include <stdio.h>
int test(int a, int b) {
return a + b + 3;
}
int main(int argc, const char* argv[]) {
int a = test(1.2);
printf("%d", a);
return 0;
}
Copy the codeThe main.m file looks like this, and then we execute it on the terminal
clang -S -fobjc-arc -emit-llvm main.m
Copy the codeA main.ll file is then generated, with some code captured as follows
; Function Attrs: noinline nounwind optnone ssp uwtable
define i32 @test(i32, i32) #0{%3 = alloca i32, align 4
%4 = alloca i32, align 4
store i32 %0, i32* %3, align 4
store i32 %1, i32* %4, align 4
%5 = load i32, i32* %3, align 4
%6 = load i32, i32* %4, align 4
%7 = add nsw i32 %5%,6
%8 = add nsw i32 %7.3
ret i32 %8}; Function Attrs: noinline optnone ssp uwtable define i32 @main(i32, i8**) #1{%3 = alloca i32, align 4
%4 = alloca i32, align 4
%5 = alloca i8**, align 8
%6 = alloca i32, align 4
store i32 0, i32* %3, align 4
store i32 %0, i32* %4, align 4
store i8** %1, i8*** %5, align 8
%7 = call i32 @test(i32 1, i32 2)
store i32 %7, i32* %6, align 4
%8 = load i32, i32* %6, align 4
%9 = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @.str, i32 0, i32 0), i32 %8)
ret i32 0
}
Copy the codeIf we look at the main method first, we can see that there are two calls in the main method, which correspond to the test() method and print method respectively. Then we look at the performance of test method in IR:
// IR
define i32 @test(i32, i32)
// main.m
int test(int a, int b)
Copy the code- This row is the same thing as
int test(int a, int b). The first i32 represents the return value of the function, and the second and third i32 represent the function arguments. There are actually two hidden identifiers %0 and %1 for parameters A and B, respectively.
%3 = alloca i32, align 4
%4 = alloca i32, align 4
Copy the code- Allocate 4 bytes of memory space, aligned with 4 bytes, and return to %3 for identification
- Allocate 4 bytes of memory, align with 4 bytes, and return to %4 for identification
store i32 %0, i32* %3, align 4
store i32 %1, i32* %4, align 4
Copy the code- Extract the contents of the %0 identifier into the memory space of the %3 identifier, aligned with 4 bytes
- Extract the contents of the %1 identifier into the memory space of the %4 identifier, aligned with 4 bytes
%5 = load i32, i32* %3, align 4
%6 = load i32, i32* %4, align 4
Copy the code- Extract the contents of the %3 identifier into the memory space of the %5 identifier, aligned with 4 bytes
- Extract the contents of the %4 identifier into the memory space of the %6 identifier, aligned with 4 bytes
%7 = add nsw i32 %5%,6
%8 = add nsw i32 %7.3
Copy the code- Add the %5 and %6 identifiers and assign the result to the %7 identifiers
- Add the %7 identifier to 3 and assign the result to the %8 identifier
ret i32 %8
Copy the code- Returns the contents of the %8 identifier
Return a + b + 3; return a + b + 3; In the IR intermediate code, however, this line of code is translated into more steps. In fact, this is because optimization is not enabled by default when executing the command that generates IR code. We can check the corresponding optimization level in Debug and Release mode in XCode:
Optimization levels from low to high are:
- None: Default option for Debug mode, not optimized. Identifier for
-O0. - Fast: the identifier is
-O, O1. - Faster: the identifier is
-O2. - Fastest: identifier is
-O3. - Fastest, Smallest: Identifier is
-Os. - Slowest, Aggressive Optimizations: Identifier is
-Ofast. - Smallest, Aggressive Size Optimizations: The identifier is
-Oz.
So, for example, we’re using the Smallest optimization strategy, which allows us to enter this on the terminal
clang -Os -S -fobjc-arc -emit-llvm main.m -o main1.ll
Copy the codePart of main1.ll is captured as follows
; Function Attrs: norecurse nounwind optsize readnone ssp uwtable
define i32 @test(i32, i32) local_unnamed_addr #0{%3 = add i32 %0.3
%4 = add i32 %3%,1
ret i32 %4}; Function Attrs: nounwind optsize ssp uwtable define i32 @main(i32, i8** nocapture readnone) local_unnamed_addr #1{%3 = tail call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @.str, i64 0, i64 0), i32 6) #3, !clang.arc.no_objc_arc_exceptions !9
ret i32 0
}
Copy the codeYou can see that the lines of IR code have been greatly reduced. And in the main method, directly calculate the test function execution result is 6, and omit the call to test function.
Generating assembly code
Assembly code is generated from the final BC or LL intermediate code
Clang-s -fobjc-arc main.bc -o main.s // BC -clang-s -fobjc-arc main.bc -o main.s // BC -clang-s -fobjc-arc main.bc -o main.sCopy the codeOptimization strategies can also be used to generate assembly code
Clang-os-s-fobjc-arc intermediate code IR-O assembly source fileCopy the codeLet’s test that out
clang -Os -S -fobjc-arc main.ll -o main.s
Copy the codeThe output assembly source file main.s contains the following contents:
.section __TEXT,__text,regular,pure_instructions
.build_version macos, 10, 15 sdk_version 10, 15
.globl _test ## -- Begin function test
.p2align 4, 0x90
_test: ## @test
.cfi_startproc
## %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
movl %edi, -4(%rbp)
movl %esi, -8(%rbp)
movl -4(%rbp), %esi
addl -8(%rbp), %esi
addl $3, %esi
movl %esi, %eax
popq %rbp
retq
.cfi_endproc
## -- End function
.globl _main ## -- Begin function main
.p2align 4, 0x90
_main: ## @main
.cfi_startproc
## %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
subq $32, %rsp
movl $0, -4(%rbp)
movl %edi, -8(%rbp)
movq %rsi, -16(%rbp)
movl $1, %edi
movl $2, %esi
callq _test
movl %eax, -20(%rbp)
movl -20(%rbp), %esi
leaq L_.str(%rip), %rdi
movb $0, %al
callq _printf
xorl %esi, %esi
movl %eax, -24(%rbp) ## 4-byte Spill
movl %esi, %eax
addq $32, %rsp
popq %rbp
retq
.cfi_endproc
## -- End function
.section __TEXT,__cstring,cstring_literals
L_.str: ## @.str
.asciz "%d"
.section __DATA,__objc_imageinfo,regular,no_dead_strip
L_OBJC_IMAGE_INFO:
.long 0
.long 64
.subsections_via_symbols
Copy the codeGenerate object file
The generation of the object file is that the assembler takes the assembly code as input, converts the assembly code into machine code, and finally outputs the object file. The command is as follows:
Clang-fmodules -c assembler source file -o target fileCopy the codeLet’s test that out
clang -fmodules -c main.s -o main.o
Copy the codeAfter the object file is generated, we can use the nm command to view symbols
O // Output (undefined) external _printf 0000000000000000 (__TEXT, __TEXT) External _test 0000000000000020 (__TEXT,__text) external _mainCopy the code_prinf is a undefined external symbol, where undefined means that the _prinf symbol is temporarily not found in the current file, and external means that the symbol is externally accessible.
Generate an executable file
The linker links the compiled.o files with library files (dynamic library.dylib, static library.a) to produce a Mach-o executable.
The command is
Clang Target file name -o Executable file nameCopy the codeLet’s test that out
clang main.o -o main
Copy the codeNext, look at symbols in the executable
xcrun nm -nm main
(undefined) external _printf (from libSystem)
(undefined) external dyld_stub_binder (from libSystem)
0000000100000000 (__TEXT,__text) [referenced dynamically] external __mh_execute_header
0000000100000f20 (__TEXT,__text) external _test
0000000100000f40 (__TEXT,__text) external _main
0000000100002008 (__DATA,__data) non-external __dyld_private
Copy the codeYou can see that the symbol of the printf function is loaded from the libSystem library, indicating that the system function is bound by dyld at run time.
Compile the LLVM project
LLVM download
As a result of the domestic network limited, it is best to use the image below to download source mirrors.tuna.tsinghua.edu.cn/help/llvm/ LLVM
- Download the LLVM project
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/llvm.git
Copy the code- Download Clang in the tools directory of LLVM
cd llvm/tools
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/clang.git
Copy the code- Download compiler-rt, libcxx, libcxxabi in the LLVM projects directory
cd ../projects
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/compiler-rt.g
it
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/libcxx.git
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/libcxxabi.git
Copy the code- Install the Extra tool under tools in Clang
cd ../tools/clang/tools
git clone https://mirrors.tuna.tsinghua.edu.cn/git/llvm/clang-tools-e
xtra.git
Copy the codeThe LLVM compiler
Since the latest LLVM only supports cmake compilation, we also need to install cmake
Install cmake
- Check brew to see if cmake is installed and skip this step if so
brew list
Copy the code- Install cmake via BREW
brew install cmake
Copy the codeCompile LLVM with Xcode
- Cmake compiles into Xcode projects
mkdir build_xcode
cdbuild_code cmake -G Xcode .. /llvmCopy the code- Compile Clang using Xcode
-
Select Automatic creation of Schemes
-
Compile, select ALL_BUILD Scheme to compile, and it is expected to take more than 1 hour
-
A preliminary study of Clang plug-in
Once compiled, we can customize our own plug-in based on Clang.
Clang provides three different ways to write:
- LibClang: stable high-level C abstract interface
Benefits: 1. You can interact with clang in languages other than C++ 2. 3. Provide powerful high-level abstractions such as iterating AST over CURSOR, & don't need to learn Clang's AST details. Disadvantages: Unable to fully control clang ASTCopy the codePs: The C&Python API is officially available, and there is a Clangkit in OC form
- Clang Plugins: The Clang plug-in allows you to add and run other operations on the AST as part of compilation. Plug-ins are dynamic libraries loaded by the compiler at run time that are easy to integrate into the build environment.
Using the Clang plug-in: 1. If any of the dependencies change, you need your tool to rerun 2. Want your tool to be able to make or break builds 3. Need full control of Clang AST without using Clang plugins: 1. You want to run tools outside of your build environment 2. You want full control over Clang Settings, including the mapping of in-memory virtual files 3. You need to run a specific subset of files in your project that are independent of any changes that trigger the rebuildCopy the codePs: Clang plugins are the way to go when you need warnings or errors tailored to a particular format for your project, or when you need to create additional build artifacts from a build step.
- LibTooling is a C++ interface designed to write standalone tools and integrate them into services running clang tools.
Using LibTooling: 1. Desired to run tool 2 on a single file or a specific subset of files independent of the build system. LibTooling: 1 was used to share code with the Clang plug-in. You want to run as part of a build triggered by a dependency change. 2. You want a stable interface so that code 3 does not need to be changed when the AST API changes. Want to use advanced abstractions like cursor 4. Don't want to write your tools in C ++Copy the codePs: Select libTooling when you need to write a simple syntax checker or a refactoring tool
What we need now is the ability to check at compile time that the property identifier should be set to copy but is not, and then give a warning. So our best solution is the clang plugin. Let’s see how it works:
Once the dynamic library is loaded, we can get our custom pluginAction(a subclass of FrontendAction).
Then, after CompileInstance is initialized, the pluginAction member functions (BeginSourceFile, Excute, EndSourceFile) are called in sequence.
CreateConsumer creates our custom consumer to get the syntax tree information, executes the ExecuteAction function to enter the ParseAST analysis process, and calls our custom ASTConsumer to handle. Match the AST Notes that you want to check for the operation through the RecursiveASTVisitor or ASTMatcher.
Create a Diagnosis to warn or report an error if the spec is not met, and you can create a FixHint to provide the ability to fix it. Information such as source location & global identifier is obtained through ASTContext and its associated SourceManager.
ASTMatcher is recommended for the above ParseAST phase, which allows simple, accurate and efficient matching to AST Notes.
Create the plug-in
- in
/llvm/tools/clang/toolsCreate a folder for the plug-in
- Modify the
/llvm/tools/clang/toolsThe newCMakeLists.txtFile, add a lineAdd_clang_subdirectory (plug-in folder name)
- In the JHPlugin directory, create a new source file named jhpulgin. CPP and a new text file named cmakelists.txt.
Enter the following contents in the cmakelists. TXT file
add_llvm_library( JHPlugin MODULE BUILDTREE_ONLY
JHPlugin.cpp
)
Copy the code-
To regenerate the Xcode project using cmake, execute cmake -g Xcode in build_xcode.. /llvm
-
Finally, in the LLVM Xcode project, you can see that Loadable Modules has its own Plugin directory.
Writing a plug-in
1. Customize an Action that inherits from PluginASTAction
/ / PluginASTAction inheritance
class JHASTAction : public PluginASTAction {
public:
unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &Compiler, StringRef InFile);
bool ParseArgs(const CompilerInstance &CI, const std: :vector<std: :string> &args);
};
Copy the code2. Register the plug-in
// Register the plug-in
static FrontendPluginRegistry::Add<JHPlugin::JHASTAction>
X("JHPlugin"."This is the description of the plugin");
Copy the code2. Customize a consumer that inherits from ASTConsumer
// Customize ASTConsumer
class JHConsumer : public ASTConsumer {
}
Copy the codeThe ASTConsumer class is an abstract class that resolves nodes in the AST syntax tree. By subclassing this abstract class, we can override the methods provided by the ASTConsumer class to resolve nodes:
// HandleTopLevelDecl - Handle the specified top-level declaration.
/// This iscalled by the parser to process every top-level Decl*.
///
/// \returns true to continue parsing, or false to abort parsing.
// Parse a top-level declaration node
virtual bool HandleTopLevelDecl(DeclGroupRef D);
/// HandleTranslationUnit - This method is called when the ASTs for entire
/// translation unit have been parsed.\
// is called after parsing the entire file
virtual void HandleTranslationUnit(ASTContext &Ctx) {}
Copy the codeWe added the following two methods to JHConsumer:
// Parse the top-level node
bool HandleTopLevelDecl(DeclGroupRef D) {
cout << "Parsing..." << endl;
return true;
}
// is called after parsing the entire file
void HandleTranslationUnit(ASTContext &Ctx) {
cout << "File parsed!" << endl;
matcher.matchAST(Ctx);
}
Copy the codeThen we compile the current plug-in, and we test it with the compiled Clang and JHPlugin. Let’s create a new main.m file on our desktop, with the following contents:
int test(int a, int b) {
return a + b;
}
int test2(int a, int b, int c) {
return a + b + c;
}
int main(a) {
test(1.2);
test2(1.2.3);
return 0;
}
Copy the codeThen run the following command on the terminal:
/ Users/leejunhui/Documents/iOS/OpenSourceApple LLVM source/build_xcode/Debug/bin/clang - isysroot / Applications/Xcode. App/Contents/Developer/Platforms/iPhoneSimulator platform/Developer/SDKs/iPhoneSimulator13.2. The SDK - Xclang - load - Xclang/Users/leejunhui/Documents/iOS/OpenSourceApple LLVM source/build_xcode/Debug/lib/JHPlugin dylib - Xclang -add-plugin -Xclang JHPlugin -c main.mCopy the codeBreak down the command:
Self-compiled clang file path -isysroot iOS SDK path - Xclang-load Xclang plug-in dylib path - xclang-add-plugin -Xclang plug-in name -c source fileCopy the codeThe final output is as follows:
Parsing... Parsing... Parsing... File parsing complete!Copy the codeThe three top-level nodes in main.m and the entire file are parsed.
3. Final code
#include <iostream>
#include "clang/AST/AST.h"
#include "clang/AST/DeclObjC.h"
#include "clang/AST/ASTConsumer.h"
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
#include "clang/Frontend/FrontendPluginRegistry.h"
using namespace clang;
using namespace std;
using namespace llvm;
using namespace clang::ast_matchers;
namespace JHPlugin {
class JHMatchCallback: public MatchFinder::MatchCallback {
private:
CompilerInstance &CI;
// Check if it is your own file
bool isUserSourceCode(const string filename) {
if (filename.empty()) return false;
// Source code that is not in Xcode is considered user source
if (filename.find("/Applications/Xcode.app/") = =0) return false;
return true;
}
// Check whether copy should be used.
bool isShouldUseCopy(const string typeStr) {
if (typeStr.find("NSString") != string::npos ||
typeStr.find("NSArray") != string::npos ||
typeStr.find("NSDictionary") != string::npos/ *... * /) {
return true;
}
return false;
}
public:
JHMatchCallback(CompilerInstance &CI):CI(CI){}
void run(const MatchFinder::MatchResult &Result){
// Get the node from the result.
const ObjCPropertyDecl *propertyDecl = Result.Nodes.getNodeAs<ObjCPropertyDecl>("objcPropertyDecl");
// Get the file name
string filename = CI.getSourceManager().getFilename(propertyDecl->getSourceRange().getBegin()).str();
if (propertyDecl && isUserSourceCode(filename)) {// If the node has a value and is a user file
// Get the type of attribute
string typeStr = propertyDecl->getType().getAsString();
// Get the description of the node
ObjCPropertyDecl::PropertyAttributeKind attrKind = propertyDecl->getPropertyAttributes();
// Check whether Copy should be used
if(isShouldUseCopy(typeStr) && ! (attrKind & ObjCPropertyDecl::OBJC_PR_copy)) {cout<<typeStr<<"Copy should be used instead of copy, warning!"<<endl;
// Diagnostic engine
DiagnosticsEngine &diag = CI.getDiagnostics();
/ / the Report Report
diag.Report(propertyDecl->getBeginLoc(),diag.getCustomDiagID(DiagnosticsEngine::Warning, "Copy is recommended for %0"))<<typeStr; }}}};// Custom JHConsumer
class JHConsumer: public ASTConsumer {
private:
MatchFinder matcher;
JHMatchCallback callback;
public:
JHConsumer(CompilerInstance &CI):callback(CI){
// Add a MatchFinder to the objcPropertyDecl node
// The callback is in JHMatchCallback's run method.
matcher.addMatcher(objcPropertyDecl().bind("objcPropertyDecl"),&callback);
}
// called after the entire file has been parsed
void HandleTranslationUnit(ASTContext &context) {
cout<<"That's it!"<<endl; matcher.matchAST(context); }};// Implement our custom Action by inheriting PluginASTAction
class JHASTAction: public PluginASTAction {
public:
bool ParseArgs(const CompilerInstance &CI,const vector<string> &arg){
return true;
}
unique_ptr <ASTConsumer> CreateASTConsumer(CompilerInstance &CI, StringRef InFile) {
return unique_ptr <JHConsumer> (newJHConsumer(CI)); }}; }// Register the plug-in
static FrontendPluginRegistry::Add<JHPlugin::JHASTAction> X("JHPlugin"."This is the description of the plugin");
Copy the codeWe declare the following four properties in the ViewController:
@interface ViewController(a)
@property (nonatomic.strong) NSArray *arr;
@property (nonatomic.strong) NSString *str;
@property (nonatomic.copy) NSArray *arr1;
@property (nonatomic.copy) NSString *str1;
@end
Copy the codeThen run it:
/ Users/leejunhui/Documents/iOS/OpenSourceApple LLVM source/build_xcode/Debug/bin/clang - isysroot / Applications/Xcode. App/Contents/Developer/Platforms/iPhoneSimulator platform/Developer/SDKs/iPhoneSimulator13.2. The SDK - Xclang - load - Xclang/Users/leejunhui/Documents/iOS/OpenSourceApple LLVM source/build_xcode/Debug/lib/JHPlugin dylib - Xclang -add-plugin -Xclang JHPlugin -c ViewController.mCopy the codeThe output is as follows:
That's it! NSArray * should be decorated with copy instead of copy, warning! Viewcontroller.m :12:1: Warning: NSArray * Copy @property (nonatomic, strong) NSArray *arr; ^ NSString * should be modified with copy instead of copy, warning! Viewcontroller.m :13:1: warning: Copy @property (nonatomic, strong) NSString * STR; ^ 2 warnings generated.Copy the codeYou can see that the last two properties that should use the copy modifier are detected and warned because copy is not used.
Plug-ins integrate into Xcode
The way we’ve done it above is to view the plug-in execution on the terminal, but ideally it would be integrated into Xcode itself. Open the test project and add the following content to Build Settings -> Other C Flags:
- xclang-load Xclang (.dylib) dynamic library path - xclang-add-plugin -Xclang plug-in nameCopy the code
At this point, if we run Command + B to compile the project, we will get the following error message
The solution is to add two Settings, as shown below
- CC corresponds to the absolute path of your own compiled clang
- CXX corresponds to the absolute path of the clang++ compiler itself
Then modify one more setting, as shown below
Finally, we Command + B to compile again. Note that this time the compilation will take a bit longer because Xcode has to load the clang we compiled ourselves. After loading, we can see the following prompt
conclusion
The resources
In-depth analysis of iOS compiler Clang/LLVM – Daming
Develop a Clang Plugin efficiently