preface
Learning is like rowing upstream; not to advance is to drop back. Mutual encouragement!!
Today I would like to share with you A Swift practice of station A.
After continuous iteration, Swift has now become the preferred development language of iOS and even Apple platform. Station A has also been fully involved in Swift wave, enjoying the comfortable and efficient development experience brought by Swift language. Swift Practice of Station A — Part I introduces the technical background of Swift, the evolution process of Swift architecture and the selection of technologies such as SwiftUI and Combine, the latest framework. The original | address
How to go
Google I/O, which ended yesterday, is reminiscent of Kotlin’s popular search three years ago, when Google I/O officially announced Kotlin as the preferred language for Android development over Java. The power of evolution is due to apple’s replacement of Objective-C by Swift in 2014 as the preferred development language for iOS and apple’s entire platform, which has increased the enthusiasm of iOS developers. The previous part introduced Swift’s technical background and how to choose a development framework. The next part will introduce how most projects with OC as the main body dance with Swift and how to solve engineering problems with Swift dynamics.
However, how does mixed development get started? Are there any preconditions?
precondition
Mashup is essentially a declaration of OC syntax generated by a compiler to a declaration of Swift syntax so that Swift can call the OC interface directly from the generated declaration. Conversely, OC calls to the Swift interface can use the same method to generate the Swift syntax declaration into OC syntax header files. The compilation tools generated by these transformations are integrated into the development tool Xcode.
Xcode is simply a tool for executing multiple command lines, such as Clang, LD, etc. The Xcode and Project files contain the parameters of these commands and the order in which they are executed, as well as the files to be compiled and their dependencies. Llbuild [1] is a low-level build system that executes commands sequentially according to the configuration in Xcode Project. The command line tool parameters are set in Xcode’s Build Settings. Set the Always Embed Swift Standard Libraries in Build Settings to YES, and then bridge the file. That is, productname-bridge-header. h imports the OC classes that need to be exposed to Swift. If the OC to be invoked by Swift is in a different Project, set the Project of OC to Module, the cbmMODULE to YES, and import the Header file of the Module into the Umbrella Header of the OC Modulemap file.
How do I set up CocoaPods
The Swift Pod Podspec needs to specify its dependency on OC Pod. In the project Podfile, the OC Pod is followed by :modular_headers => true. Turning on Modular Header converts Pod to Module. So what does CocoaPods do? Run Pod Install — Verbose to see that when generating Pod Targets, CocoaPods generates Module Map File and Umbrella Header. Each project setting can be very strange, and CocoaPods uses its own DSL configuration to set these compilation parameters, so it’s important to understand some of the compilation parameters and concepts of the composite Settings:
- For the chDEFINES Module, set the parameter to YES.
- Module Map File Indicates the Module Map path.
- Header Search Paths represent the OC Header file Paths defined by the Module Map.
- The default setting for Product Module Name is the same as Target Name.
- Framework Search Paths sets the Framework dependent Search Paths.
- Other C Flags can be used to configure dependencies on Other Module file paths.
- Other Swift Flags Specifies the path to its Module Map file.
The main components of CocoaPods are CLAide[2] for parsing commands, parsing Pod description files, Examples include Podfile, podfile. lock, and PodSpec files cocoapods-core [3], Co** Coapods-downloader [4], Molinillo[5], And xcodeProj for creating and editing.xcodeProj and.xcworkspace files for Xcode [6]. After Pod Install is executed, the component invokes the process and configures the Module’s process location as shown below:
After the above work, if we want to use the library FMDB developed by OC in Swift, we can Import it directly with the following code
import UIKit
import FMDB
class SwiftTestClass: NSObject {
var db:FMDB.FMDatabase?
override init() {
super.init()
self.db = FMDB.FMDatabase(path: "dbname")
print("init ok")
}
}
Copy the codeAs you can see, after Import FMDB dumps FMDB’s Module, the interface can still be invoked directly using Swift syntax.
Note here that the Pod that Module depends on also needs to be Module. Therefore, the Module needs to be transformed from bottom to top. In addition, if a Header file is in Umbrella Header after Module is enabled, other pods that contain the Header file also need to open Module.
Why Module?
Before the Module can be used, developers need to preprocess the C compiler processing class header file to be imported, find which other imported headers are in the header file, and recurse until they find all of them. However, there are many problems with preprocessing. First, compilation is highly complex and time-consuming, because each compilable file will be separately compiled and preprocessed, so the recursive search for imported header files will be repeated many times in the preprocessing process, especially when header files with deep inclusion relationship are imported by many. M files. Second, there will be macro definition conflicts need to reorder and resolve dependency issues.
Module is relatively simple, its header only needs to be parsed once, so the complexity of compiling is exponentially lower, and the way the compiler handles Module is completely different from the way C preprocesses it. If a Module needs to be imported during compilation, the compiler will check the Module Cache first. If there is a corresponding binary file, the Module will be loaded directly. If there is no binary file, the Module will be parsed. Reparsing a compiled Module will only happen if any of the header files contained in the header file change, or if an update is made depending on another Module. For example:
#import <FMDB/FMDatabase.h>
Copy the codeClang will look for fmDatabase. h from the Headers directory of fMDB. framework and module.modulemap from the Modules directory of FMDB.framework. Analyze module. moduleMap to determine if FMDatabase.h is part of the module. The Module Map defines the relationship between modules and header files. The contents of the module.modulemap of fMDB. framework are as follows:
framework module FMDB {
umbrella header "FMDB-umbrella.h"
export *
module * { export * }
}
Copy the codeTo determine whether fmDatabase. h is a part of Module, look for fmDatabase. h in the Umbrella Header file in module.modulemap. Go to the Headers directory and check for fmDB-umbrella. H:
#ifdef __OBJC __#import <UIKit/UIKit.h> #else #ifndef FOUNDATION_EXPORT #if defined(__cplusplus) #define FOUNDATION_EXPORT extern "C" #else #define FOUNDATION_EXPORT extern #endif #endif #endif #import "FMDatabase.h" #import "FMDatabaseAdditions.h" #import "FMDatabasePool.h" #import "FMDatabaseQueue.h" #import "FMDB.h" #import "FMResultSet.h" FOUNDATION_EXPORT double FMDBVersionNumber; FOUNDATION_EXPORT const unsigned char FMDBVersionString[];Copy the codeAs you can see in the code above, FMDatabase.h is already included in the file, so Clang will import FMDB as Module. An Umbrella framework is an encapsulation of a framework to hide complex dependencies between frameworks. Build the Module will be deposited to the ~ / Library/Developer/Xcode/DerivedData/ModuleCache noindex/below this directory.
Clang compiles a single OC file by importing a header file, whereas Swift does not have a header file, so the Swift compiler Swiftc needs to look for declarations before generating interfaces. In addition, Swiftc also looks for OC declarations in declarations exposed in Module Map files and Umbrella Header files.
Support for Swift 5.1 plus Module Stability and Library Evolution is required if the project is to build binary libraries.
Name Mangling
After finding the OC statement, Swiftc needs to conduct the Name Mangling. The Name Mangling is used both to prevent Name collisions like C++ and to Swift stylize renaming of OC interface names. If you are not satisfied with the Name Mangling naming, you can go back to the OC source code and redefine the Name you want to use in Swift with NS_SWIFT_NAME.
Swiftc Name Mangling generates more information than C and C++ Name Mangling, such as the following code:
public func int2string(number: Int) -> String {
return "(number)"
}
Copy the codeAfter Swiftc is compiled, run the nm-g command to view the following information:
0000000000003de0 T _$s8demotest10int2string6numberSSSi_tF
Copy the codeAs shown above, $s in the message represents global, 8Demotest’s demotest is the Module name, and 8 is the length of the Module name. Int2string is the function name, 10 is the length of the class name, and 6number is the parameter name. SS indicates that the parameter type is Int. Si represents a String, and _tF indicates that the preceding Si is a return type.
Next, compare the compilation process of Clang and Swiftc. First, the compilation process of Clang is shown below
As for how OC calls Swift interface, Swiftc will generate a header file. If there is a Public declaration in the code, Swiftmodule will be generated according to the file first, and Swiftmodule will be merged after the file link, and finally generated into a header file as a whole. The process is shown in the figure below:
Why can THE OC interface be tuned?
Swift code can call the OC interface because the OC interface is automatically generated by the compiler as a Swift syntax interface file. In Xcode, click on the Related Items in the upper left corner of the OC header file and select Generated Interface to view the Generated Swift version Interface file. The automatically converted Swift interface files can be directly called by Swift. During the conversion process, the compiler will convert NSString, the basic OC library, into the corresponding Swift libraries such as String and Date. OC’s initialization methods are also converted to Swift’s constructor methods. Error handling is also converted to Swift style. Here are the types for OC and Swift conversions:
Apple also provides macros to help generate a nice Swift interface.
As we all know, OC has been missing non-null type information, which can be wrapped in NS_ASSUME_NONNULL_BEGIN and NS_ASSUME_NONNULL_END so that non-null is not specified one by one. The NS_DESIGNATED_INITIALIZER macro can be used to designate an initializer, leaving the NS_DESIGNATED_INITIALIZER macro as Convenience. NS_SWIFT_NAME is used to rename names used in Swift, and NS_REFINED_FOR_SWIFT resolves data inconsistencies.
Access to the Core Foundation type is inevitable during iOS development. Once the Core Fundation framework is imported into the Swift hybrid environment, its type is automatically converted to Swift. Swift also automatically manages the memory of Annotated Core Foundation objects, rather than manually calling CFRetain, CFRelease, or CFAutorelease as in OC. Unannotated objects are wrapped in an Unmanaged structure, as in this code:
CFStringRef showTwoString(CFStringRef s1, CFStringRef s2)
Copy the codeTo Swift:
func showTwoString(_: CFString!, _: CFString!) -> Unmanaged<CFString>! {
// ...
}
Copy the codeAs shown in the code above, the Core Fundation name is converted without the suffix Ref, because in Swift all classes are reference types and the Ref suffix is redundant. The Unmanaged structure above has two methods, takeUnretainedValue() and takeRetainedValue(), both of which are used to return the original unwrapped type of the object. If the object has no Retain before, use takeUnretainedValue() and takeRetainedValue() if it has already retained.
Call C Variadic functions such as vasprintf(:::) in Swift with getVaList(::) or withVaList(::).
Call pointer parameter C function, and Swift mapping as shown below:
Module written by Swift can also be called by OC. However, such calls have many limitations, because there are many types in Swift that cannot be used by OC, such as enums defined in Swift, structs defined by Swift, top-level functions defined by Swift, global variables, Typealiases, Nested types. Swift has also become less Swift. \
Even with the implementation of mash-up, there are a number of challenges developers face. Because many problems in OC era, such as Hook, traceless burial point, can be easily implemented when OC runs, while Swift lacks natural support. Here’s a look at the dynamics of Swift and how we should use it until it’s official.
dynamic
Swift has its own runtime for dealing with the pure Swift language, but the Swift core team does not do dynamic features for the problem of “this runtime does not provide an interface to access”. It is because supporting dynamic features requires dealing with the effects of dynamic calls to the Virtual Method Table on SIL function optimization. For example, classes that are not overridden will automatically be optimized to static calls, which takes a lot of time. There are higher priorities at this stage, such as concurrency models, system programming, static analysis to support type states, and so on. Therefore, some people choose to implement a Swift runtime themselves, making Swift code dynamic. Jordan Rose[7] implemented a simplified version of Swift [8] Runtime, and a more rigorous Runtime implementation can refer to Echo[9] and Runtime[10].
One might ask, isn’t SwiftUI Preview the typical run-time replacement method? How did he do it? Instead, he uses the @_dynamicreplacement property, which is an internally used property that can be held directly for method replacement.
@_dynamicReplacement(for: runSomething())
static func _replaceRunSomething() -> String {
"replaced"
}
Copy the codeIf you want to put the above code into a library and load the library at runtime for runtime method substitution, you can do this:
runSomething()
let file = URL(fileURLWithPath: "/path/of/replaceLib.dylib")
guard let handle = dlopen(file.path, RTLD_NOW) else {
fatalError("oops dlopen failed")
}
runSomething()
Copy the codeIs there another way to do run-time method substitution besides this method?
Method substitution of a value type
The AnyClass and class_getSuperclass methods look at the inheritance chain of Swift objects. Swift classes that do not inherit NSObject have an implicit Super Class with a generated prefix SwiftObject, Such as _TtCs12_SwiftObject. Swift is an objC runtime type that implements NSObject and cannot interact with OC. But if you inherit NSObject you can interact with OC.
If a method or property declares @objc Dynamic, it can be called at run time on a Swift object dynamically by using AnyObject’s Perform method to Perform the method or property name passed in NSSelectorFromString.
Value types in Swift, such as Struct, Enum, and Array, can be converted to the corresponding OC object Type by Type Casting following the _ObjectiveCBridgeable protocol. For example, if you want to view the class inheritance relationship of Array, the code looks like this:
func classes(of cls: AnyClass) -> [AnyClass] {`` var clses:[AnyClass] = [] `` var cls: AnyClass? = cls `` while let _cls = cls { `` clses.append(_cls) `` cls = class_getSuperclass(_cls) `` } `` return clses ``}`` let arrays = ["jone", "rose", "park"]`` print(classes(of: object_getClass(arrays)!) )`` // [Swift.__SwiftDeferredNSArray, Swift.__SwiftNativeNSArrayWithContiguousStorage, Swift.__SwiftNativeNSArray, __SwiftNativeNSArrayBase, NSArray, NSObject]`Copy the codeAs the code above shows, Swift’s Array is ultimately inherited from NSObject, as are the other value types. As you can see, all Swift types are compatible with the OBJC runtime. So you can add objC runtime methods to these value types as follows:
Struct structWithDynamic {public var STR: String public func show(_ STR: String) String) -> String { print("Say (str)") return str } internal func showDynamic(_ obj: AnyObject, str: String) -> String { return show(str) } } let structValue = structWithDynamic(str: "Hi!" // Add Objc runtime method to structValue @convention(block)(AnyObject, String) -> String = structValue.showDynamic let imp = imp_implementationWithBlock(unsafeBitCast(block, to: AnyObject.self)) let dycls: AnyClass = object_getClass(structValue)! class_addMethod(dycls, NSSelectorFromString("objcShow:"), imp, "@24@0:8@16") // perform(NSSelectorFromString("objcShow:"), with: String("Bye!" ))!Copy the codeAs shown in the code above, fetching the function closure can be called by converting @convertion(block) to the C function Call Convention, or C functions can execute the pointer directly. The Memory Dump tool allows you to view the Memory structure of the Swift function and parse out symbolic information DL_Info. Memory Dump tools include Mikeash’s MemoryDumper2 [11]. For source code interpretation, refer to Swift Memory Dumping[12]. For a reverse view of the memory layout, see Preliminary Swift Runtime: Implementing a Packet Capture Tool for Alamofire using Frida [13]
Class method substitution
To replace a class method at runtime, pass the imp_implementationWithBlock method as AnyObject, return an IMP, and use class_getInstanceMethod to get the original method of the instance. Method substitution is performed via class_replaceMethod. For the complete code, see InterposeKit[14], as well as SwiftHook[15], a library for method substitution using libffi.
In addition, it is difficult to change the function pointing position by obtaining the function address in Swift because the NSInvocation is not available, so Hook Swift via THE C function. In Swift AnyClass there is an OC-like layout that records data pointing to classes and class member functions so that assembly can be used to do the function pointer substitution thing. The idea is: save the register, call a new function, then restore the register, restore the function. For details, please refer to SwiftTrace[16].
Insert the pile
The key step is to use DYLD_INSERT_LIBRARIES to intercept Swiftc at compile time. Commandline.arguments can get Swiftc execution arguments to find the Swift file to compile. Apple’s SwiftSyntax[17] source code parsing, generation, and transformation tool detects all methods and inserts specific methods to replace the logic code. After modification, -output-file-map is used to obtain the address of Mach-o to overwrite the previous product. Using the self. OriginalImplementation (…). Call the original implementation as a closure passed to the execute(Arguments :originalImpl:) method.
ClassContextDescriptorBuilder
The Swift runtime reserves Metadata information for each type. Metadata is statically generated by the compiler so that debugging with Metadata can discover type information. Metadata offset -1 is a pointer to the Witness Table, which provides the values of the allocate, copy, and destroy types. The Witness Table also records other attributes, such as type size, alignment, and Stride. The Metadata offset 0 is the Kind field, which describes the types that Metadata describes, such as Class, Struct, Enum, Optional, Opaque, Tuple, Function, and Protocol. Specific details of these types of Metadata is the Type of Metadata, official document [18] code description can include/swift/ABI/MetadataValues. See h [19]. For example, Metadata may contain more methods than the actual code because the compiler automatically generates methods whose Kind is described in the MethodDescriptorFlags class Kind:
enum class Kind { Method, Init, Getter, Setter, ModifyCoroutine, ReadCoroutine, }; `Copy the codeAs you can see, getters, setters, ModifyCoroutine and ReadCoroutine types for thread-specific reads and writes are generated automatically. \
/lib/IRGen/ genmeta.cpp [20]
- ClassContextDescriptorBuilder this Class is used to generate a Class of memory structure, it is inherited from TypeContextDescriptorBuilderBase.
- Enum, Struct type of memory structure Builder base classes are inherited from ContextDescriptorBuilderBase TypeContextDescriptorBuilderBase.
- ContextDescriptorBuilderBase is the foundation of the base class, the Module, the Extension, Anonymous, Protocol, Opaque Type, Generic are inherit from it.
- Struct Metadata and Enum Metadata shared memory layout, Struct can have multiple Pointers to Type Context Descriptor.
Memory layout refers to the use of a Struct or Tuple to determine how to arrange fields in memory based on the size and alignment of each field. In this process, not only the offset of each field, but also the size and alignment of the Struct or Tuple as a whole are described. The following is the code for GenMeta’s in-memory methods related to Class types:
/ / the layout of the bottom of the base class ContextDescriptorBuilderBase method void layout () {asImpl () addFlags (); asImpl().addParent(); } / / TypeContextDescriptorBuilderBase layout method of void layout () {asImpl () computeIdentity (); super::layout(); asImpl().addName(); asImpl().addAccessFunction(); asImpl().addReflectionFieldDescriptor(); asImpl().addLayoutInfo(); asImpl().addGenericSignature(); asImpl().maybeAddResilientSuperclass(); asImpl().maybeAddMetadataInitialization(); } / / ClassContextDescriptorBuilder layout method of void layout () {super: : layout (); addVTable(); addOverrideTable(); addObjCResilientClassStubInfo(); maybeAddCanonicalMetadataPrespecializations(); }Copy the codeDepending on the type of the Class can see Swift GenMeta memory layout is based on ContextDescriptorBuilderBase, TypeContextDescriptorBuilderBase to ClassContextDescriptorBu Ilder inherits layer upon layer, so the corresponding Class Type Nominal Type Descriptor can be described in the following C structure:
struct SwiftClassInfo { uint32_t flag; uint32_t parent; int32_t name; int32_t accessFunction; int32_t reflectionFieldDescriptor; . uint32_t vtable; uint32_t overrideTable; . };Copy the codeAs you can see from the code, the prefix of add is the added offset record, and addParent after addFlags is the next offset record. FieldDescriptor ReflectionFieldDescriptor is apple instead on the change of version 5.0 of the Metadata, the official Mirror reflection is still not perfect, some information also can’t provide, so added some reflection information in the Metadata.
The OC dynamic call method takes _cmd as the first argument, Self as the second argument, followed by a mutable argument list. Dynamic scheduling can add classes, variables, and methods at run time. In Swift, the dynamic call method is based on VTable, so the method cannot be searched dynamically at runtime. The address is statically written in VTable at compile time, which cannot be changed at runtime. You can use the static address call or DLSYM to search the name.
The address of the VTable is after the TypeContextDescriptor, the OverrideTable is after the VTable, and there are three fields to describe it, the first is which class is being overwritten, the second is the function being overwritten, and the third is the address of the function being overwritten. Therefore, the function pointer before and after rewriting can be found through OverrideTable, so that there is a chance to find the corresponding function in VTable to replace the function pointer, so as to achieve Hook effect. Note that the function address of VTable may be cleared or called using a direct address when the Swift compiler is set up for optimization, and method substitution cannot be performed through VTable if either of these situations occurs.
So any other ideas?
Mach_override
Mach_override [22], written using Wolf Rentzsch[21], is another way to add a JMP to the assembly of the original function, jump to the custom function, and then jump back to the original function. The three arguments to Mach_override_ptr are, first, to override the pointer to the function; Second, to override the pointer to the function; 3. The parameter can be set to the address of the pointer of the original function. After Mach_override_ptr returns successfully, the original function can be called. Mach_override allocates a virtual memory page and makes it writable and executable. Note that the Mach_override_ptr initial function is the same as the reentrant function pointer, and when called, the reentrant function will call the replacement function instead of the original function. How to use Mach_override in Swift can be referred to SwiftOverride[22].
conclusion
From the introduction of the previous part, you must have known what A station has done to embrace Swift. Based on the love and practice of some architects of STATION A and Kuaishou master station for Swift, the development experience of station A has been transformed.
In order to enable OC developers to master Swift and develop in A more “Swift” way, station A organized more than ten times of training and sharing within Swift group, and standardized Swift code style and static inspection process. In view of the pain points in the development experience, station A started the construction of the optimization, component and binary of the mixed programming project in the first half of 2020. After the layered design is completed, the Module is gradually decouped to the corresponding layer, and the language difference of Module API can be smoothen with the help of LLVM Module, so as to replace the bridge between Swift and Objective-C in the main project, and the Module problem is repaired for the base library of 10+ A station and middle station. And based on the master station binary scheme (GUNDAM) to improve the support for Swift and mixed programming. The XCFramework evolved from Swift ABI Stability to Module Stability. WWDC Session[23] is a good explanation of the principle of XCFramework. The representation XCFramework format also has good support for Objective-C/C/C++. Currently, the binary rate of components is about 80%, and about 50% of components have completed LLVM Module, increasing the build time by more than 60%. With the gradual embodiment of the advantages of Swift and the promotion of the Swift capacity construction of the team, more engineers in Station A tend to use Swift for business development, and the “acceleration” brought by Swift also makes the technical team feel A strong sense of “pushing back”.
Of course, station A has also encountered some Swift bugs, such as the module name and class name of the same Bug and Issue[24] after packaging RxSwift5, RxSwift6 solved this problem by avoiding the use of Typealias type curve, which has been officially marked as “solved”. Later versions work normally. There are also two unresolved issues. One is the Ambiguous Type Name Error in the Interface of the Module, refer to Issue[25]; The other one is generated after Import. Swiftinterface error, see Issue[26] on the website.
Finally, Swift development is not easy. Don’t be fooled by Swift’s concise syntax, the various size bracket combinations can confuse developers, and some features can make intuitive understanding difficult, such as the following code:
`let str:String! = "Hi"
let strCopy = str
Copy the codeDue to the nature of Swift derivation, strCopy is automatically inferred to a non-optional String after the exclamation symbol is added to the STR type. In fact, according to the official document [27], strCopy does not specify the type directly. That is, when STR is implicitly optional, it is String followed by an exclamation point, which is an implicit unpacking optional String and cannot be derived from the non-optional String. So Swift uses strCopy first as a plain optional value, which is very different from intuitive.
We thought Swift would be easier to Learn once ABI 5.0 was stable, but heavyweight frameworks like the new SwiftUI and Combine need to be worked on. Write Swift, Learn Every Year. Swift keeps learning from other languages. Are you ready for async/await? To use it, we need to see if the latest version of our APP supports these new features.
It was not easy, but it kept pace with The Times for stability and efficiency.
The iOS data | address
Recommended reading: Automatic compilation for iOS. Simplify testing and development
Recommended reading: Preliminary to Swift Concurrency