Summary of basic principles of iOS

This paper mainly explains the principle and realization of two kinds of wild pointer detection

Technical point: wild pointer detection

The main purpose of this paper is to understand the formation process of wild Pointers and how to detect wild Pointers

primers

Before introducing wild Pointers, let’s talk about the current types of exception handling, with a link to the Apple website.)

Exception types

Exceptions can be broadly divided into two categories:

  • 1.Software exception: mainly from kill(), pthread_kill(), NSException not caught in iOS, absort, etc
  • 2,Hardware abnormal: Hardware signals start with processor traps and are platform dependent. Wild pointer crashes are mostly hardware exceptions

When handling exceptions, however, you need to focus on two concepts

  • The Mach abnormal:The Mach layercapture
  • UNIX signal:The BSD layerTo obtain

The POSIX API in iOS is implemented through the BSD layer on top of Mach, as shown in the figure below

  • MachAccent is a Unix compatible system inspired by Accent.
  • BSDLayers are built on top of Mach and are an integral part of XNU. BSD is responsible for providing a reliable, modern API
  • POSIXPortable Operating System Interface

So, in summary, There is a correspondence between Mach exceptions and UNIX signals

  • 1. Hardware exception flow: Hardware exception -> Mach exception -> UNIX signal
  • 2. Software exception process: Software exception -> UNIX signal

Conversion of Mach exceptions to UNIX signals

Below is the code for converting Mach exceptions to UNIX signals, from BSD/Uxkern /ux_exception.c in XNU

switch(exception) {
case EXC_BAD_ACCESS:
    if (code == KERN_INVALID_ADDRESS)
        *ux_signal = SIGSEGV;
    else
        *ux_signal = SIGBUS;
    break;

case EXC_BAD_INSTRUCTION:
    *ux_signal = SIGILL;
    break;

case EXC_ARITHMETIC:
    *ux_signal = SIGFPE;
    break;

case EXC_EMULATION:
    *ux_signal = SIGEMT;
    break;

case EXC_SOFTWARE:
    switch (code) {

    case EXC_UNIX_BAD_SYSCALL:
    *ux_signal = SIGSYS;
    break;
    case EXC_UNIX_BAD_PIPE:
    *ux_signal = SIGPIPE;
    break;
    case EXC_UNIX_ABORT:
    *ux_signal = SIGABRT;
    break;
    case EXC_SOFT_SIGNAL:
    *ux_signal = SIGKILL;
    break;
    }
    break;

case EXC_BREAKPOINT:
    *ux_signal = SIGTRAP;
    break;
}
Copy the code

The corresponding relationships are summarized into a table, as shown below

  • Among these Mach exceptions are the following
The Mach abnormal instructions
EXC_BAD_ACCESS Inaccessible memory
EXC_BAD_INSTRUCTION An illegal or undefined instruction or operand
EXC_ARITHMETIC Arithmetic exceptions (such as dividing by 0). IOS is disabled by default, so we generally don’t encounter it
EXC_EMULATION Execute the instructions intended to support emulation
EXC_SOFTWARE Generally, we will not see this type of exception generated by software in Crash logs, but EXC_CRASH in Apple logs
EXC_BREAKPOINT Trace or breakpoint
EXC_SYSCALL UNIX system call
EXC_MACH_SYSCALL Mach system call
  • There are several types of UNIX signals
UNIX signal instructions
SIGSEGV Segmentation fault. Access unallocated memory and write to memory that has no write permission.
SIGBUS Bus error. Such as memory address alignment, incorrect memory type access, etc.
SIGILL An invalid instruction was executed, usually due to an error in the executable file
SIGFPE Fatal arithmetic operations. Such as value overflow, NaN values, and so on.
SIGABRT The call abort() is generated and sent via pthread_kill().
SIGPIPE Pipe burst. Typically generated by interprocess communication. For example, two processes that communicate using FIFO(pipe) will receive a SIGPIPE signal if they write to the pipe before the read pipe is opened or terminates unexpectedly. According to Apple’s documentation, this signal can be ignored.
SIGSYS The system call is abnormal.
SIGKILL This signal indicates that the system aborts the process. The crash report contains a code that represents the cause of the abort. Exit (), kill(9) and other function calls. An iOS process is killed, for example, the watchDog process.
SIGTRAP Breakpoint instruction or other trap instruction is generated.

Wild pointer

The object pointed to is released or reclaimed, but the pointer is not modified so that it still points to the reclaimed memory address. This pointer is the wild pointer

Wild pointer classification

This refers to the summary of Tencent Bugly team, which can be roughly divided into two categories

  • Memory is not overwritten
  • Memory overwritten

As shown in the figure below

Why so many OC field Pointers crash? Before the app is released, we usually go through several rounds of self-testing, inner testing, gray scale testing, etc. Generally speaking, most crashes should be covered. However, due to the randomness of wild Pointers, crashes often occur online rather than in the test, which is very fatal to the app experience

The randomness of wild Pointers can be roughly divided into two categories:

  • 1, can not run into the error of the logic, the execution of the error code, this can be throughImprove test scenario coverageTo solve the
  • 2, run into the faulty logic, but the address to which the wild pointer points does not necessarily cause crash, because:Wild pointerIts essence is a directionObjects that have been deletedorLimited memory areathePointer to the. hereOC wild pointerRefers to theA wild pointer caused by the pointer not being null after the OC object is released. The reason I don’t have to do this is becausedeallocIt just tells the system, this pieceI don't use the memory anymore, and the system doesn't make it inaccessible

Wild pointer solution idea

There are two main solutions in Xcode:

1, Malloc Scribble, its official description is as follows: when allocating memory alloc, fill the memory with 0xAA, release the memory dealloc fill the memory with 0x55.

Zombie Objects: An object that has been dereferenced, released, and still receives messages is called a Zombie object. The whole point of this solution is to turn all the freed objects into zombie objects

Compare the two schemes

  • 1.The zombie objectCompared with theMalloc Scribble.You don't have to worry about crashingAs long as the wild pointer points to a zombie object, a second visit to the wild pointer will definitely crash
  • 2. Zombie object this way,Not as extensive as Malloc ScribbleC functions can also be included in it by hook free method

1, Malloc Scribble

An exception occurs when the object memory is filled with 0xAA or 0x55

  • Application memoryallocIs filled in memory0xAA.
  • Free memorydeallocFill in memory0x55.

The above application and release fills correspond to the following two cases respectively

  • Request: Access directly without initialization
  • Release: Access after release

To sum up, our solution for wild Pointers is to fill 0x55 when the object is released. About the release of the object process can refer to memory management (1) of this article TaggedPointer/retain/release/dealloc/retainCount layer analysis

Wild pointer detection implementation 1

This realization is mainly based on the Tencent Bugly engineer: Chen Qifeng share, in its code the main idea is

  • 1. Replace the free Method of C function with the self-defined safe_free Method through Fishhook, similar to Method Swizzling

  • 2. In the safe_free method, fill the memory of the released variables with 0x55, so that the released variables cannot be accessed, so that the crash of some wild Pointers is changed from unnecessary to mandatory.

    • In order toPrevents memory filled with 0x55 from being filled with new data content, so that the wild pointer crash becomes unnecessary, the strategy adopted here is,Safe_free does not free this memory, but keeps it for itselfThat is, free is not actually called in the safe_free method.
    • At the same time in order toThis prevents excessive system memory consumption(because to preserve memory), need to be inIf the reserved memory is greater than a certain value, part of the memory is releasedTo prevent being killed by the system while receivingSystem Memory WarningWhen, also needFree some memory

  • 3. When crash occurs, the crash information obtained is limited, which is not conducive to troubleshooting, so proxy class (namely, subclass inherited from NSProxy) is adopted here to rewrite the three methods of message forwarding (see this article iOS- Underlying Principles 14: Dynamic method of message flow analysis resolution & message forwarding), and instance method of NSObject to get exception information. However, there is another problem, that is, NSProxy can only act as the proxy of OC objects, so it is necessary to add object type judgment in safe_FREE

The following is the complete wild pointer detection implementation code

  • The introduction of fishhook

  • Implement a proxy subclass of NSProxy
<! H --> @proxy: NSProxy @property (nonatomic, assign) Class originClass; @end <! M --> #import "mizombieproxy. h" @implementation MIZombieProxy - (BOOL)respondsToSelector:(SEL)aSelector{ return [self.originClass instancesRespondToSelector:aSelector]; } - (NSMethodSignature *)methodSignatureForSelector:(SEL)sel{ return [self.originClass instanceMethodSignatureForSelector:sel]; } - (void)forwardInvocation: (NSInvocation *)invocation { [self _throwMessageSentExceptionWithSelector: invocation.selector]; } #define MIZombieThrowMesssageSentException() [self _throwMessageSentExceptionWithSelector: _cmd] - (Class)class{ MIZombieThrowMesssageSentException(); return nil; } - (BOOL)isEqual:(id)object{ MIZombieThrowMesssageSentException(); return NO; } - (NSUInteger)hash{ MIZombieThrowMesssageSentException(); return 0; } - (id)self{ MIZombieThrowMesssageSentException(); return nil; } - (BOOL)isKindOfClass:(Class)aClass{ MIZombieThrowMesssageSentException(); return NO; } - (BOOL)isMemberOfClass:(Class)aClass{ MIZombieThrowMesssageSentException(); return NO; } - (BOOL)conformsToProtocol:(Protocol *)aProtocol{ MIZombieThrowMesssageSentException(); return NO; } - (BOOL)isProxy{ MIZombieThrowMesssageSentException(); return NO; } - (NSString *)description{ MIZombieThrowMesssageSentException(); return nil; } #pragma mark - MRC - (instancetype)retain{ MIZombieThrowMesssageSentException(); return nil; } - (oneway void)release{ MIZombieThrowMesssageSentException(); } - (void)dealloc { MIZombieThrowMesssageSentException(); [super dealloc]; } - (NSUInteger)retainCount{ MIZombieThrowMesssageSentException(); return 0; } - (struct _NSZone *)zone{ MIZombieThrowMesssageSentException(); return nil; } #pragma mark - private - (void)_throwMessageSentExceptionWithSelector:(SEL)selector{ @throw [NSException exceptionWithName:NSInternalInconsistencyException reason:[NSString stringWithFormat:@"(-[%@ %@]) was sent to a zombie object at address: %p", NSStringFromClass(self.originClass),NSStringFromSelector(selector), self] userInfo:nil]; } @endCopy the code
  • The concrete implementation of hook free method
<! MISafeFree. H --> @interface MISafeFree: NSObject void free_safe_mem(size_t freeNum); @end <! M --> #import "MISafeFree. H "#import "queue.h" #import "fishhook. H" #import "mizombieproxy.h" #import <dlfcn.h> #import <objc/runtime.h> #import <malloc/malloc.h> Static size_t kMIZombieSize; static size_t kMIZombieSize; Static void(* orig_free)(void *p); Static CFMutableSetRef registeredClasses = nil; */ struct DSQueue *_unfreeQueue = NULL; / struct DSQueue *_unfreeQueue = NULL; Int unfreeSize = 0; #define MAX_STEAL_MEM_SIZE 1024*1024*100 #define MAX_STEAL_MEM_NUM 1024*1024*10 #define BATCH_FREE_NUM 100 @implementation MISafeFree #pragma mark - Public Method Void free_safe_mem(size_t freeNum){#ifdef DEBUG // Get the queue length; size_t count = ds_queue_length(_unfreeQueue); FreeNum = freeNum > count? count : freeNum; For (int I = 0; i < freeNum; Void *unfreePoint = ds_queue_get(_unfreeQueue); Size_t memSize = malloc_size(unfreePoint); __sync_fetch_and_sub(&unfreeSize, (int)memSize); / / release orig_free (unfreePoint); } #endif } #pragma mark - Life Circle + (void)load{ #ifdef DEBUG loadZombieProxyClass(); init_safe_free(); #endif} #pragma mark - Private Method void safe_free(void* p){int unFreeCount = ds_queue_length(_unfreeQueue); / / reserved memory is greater than a certain value, release of part of the if (unFreeCount > MAX_STEAL_MEM_NUM * 0.9 | | unfreeSize > MAX_STEAL_MEM_SIZE) { free_safe_mem(BATCH_FREE_NUM); Size_t memSize = malloc_size(p); If (memSize > kMIZombieSize) {obj = (id)p; // Get the pointer to the original Class Class origClass = object_getClass(obj); Char *type = @encode(typeof(obj)); /* -cfsetContainsValue -cfsetContainsValue -cfsetContainsValue -cfsetContainsValue 0x55 */ if (STRCMP ("@", type) == 0 && CFSetContainsValue(registeredClasses, OrigClass)) {// memory is filled with 0x55 memset(obj, 0x55, memSize); Memcpy (obj, &kMIZombieIsa, sizeof(void*)); // Set the specified object_setClass(obj, [MIZombieProxy class]) for obj; // Keep the obj class ((MIZombieProxy*)obj). OriginClass = origClass; __sync_fetch_and_add(&unfreeSize, (int)memSize); __sync_fetch_and_add(&unfreeSize, (int)memSize) Ds_queue_put (_unfreeQueue, p); }else{ orig_free(p); } }else{ orig_free(p); }} // loadZombieProxyClass(){registeredClasses = CFSetCreateMutable(NULL, 0, NULL); Unsigned int count = 0; // Get all registered classes Class *classes = objc_copyClassList(&count); For (int I = 0; i < count; i++) { CFSetAddValue(registeredClasses, (__bridge const void *)(classes[i])); } // Free temporary variable memory free(classes); classes = NULL; kMIZombieIsa = objc_getClass("MIZombieProxy"); kMIZombieSize = class_getInstanceSize(kMIZombieIsa); } // initialize and free symbol rebind bool init_safe_free(){// Initialize the queue used to hold memory _unfreeQueue = ds_queue_create(MAX_STEAL_MEM_NUM); Orig_free = (void(*)(void*))dlsym(RTLD_DEFAULT, "free"); Struct rebinding {const char *name; // Target symbol name void *replacement; // The symbol value to be replaced (address value) void **replace; // store the original symbol value (address value)}; - Parameter 2: Rebindings_nel describes the length of the array */ / rebind the free symbol, Make it point to the custom safe_free function rebind_symbols((struct rebinding[]){{"free", (void*)safe_free}}, 1); return true; } @endCopy the code
  • test
- (void)viewDidLoad { [super viewDidLoad]; id obj = [[NSObject alloc] init]; self.assignObj = obj; // [MIZombieSniffer installSniffer]; } - (IBAction)mallocScribbleAction:(id)sender { UIView* testObj = [[UIView alloc] init]; [testObj release]; for (int i = 0; i < 10; I++) {UIView* testView = [[UIView alloc] initWithFrame:CGRectMake(0,200,CGRectGetWidth(sell.view.bounds), 60)]; [self.view addSubview:testView]; } [testObj setNeedsLayout]; }Copy the code

The print result is as follows

[UIView setNeedsLayout] was sent to a zombie object at address:
Copy the code

2, Zombie Objects

The zombie object

  • Can be used to detect memory errors (EXC_BAD_ACCESS), which can capture any calls to interpret access to bad memory
  • Send a message to a zombie object, it can still respond, and then crash with an error log showing the class name and method called by the wild pointer object

First of all, let’s take a look at how Xcode Zombie Objects are implemented. For details, please refer to this article

  • fromdeallocIn the source code, we can see"Replaced by NSZombie", i.e.,Object to releaseWhen,NSZombie will do the replacement in dealloc, as shown below

  • So the pseudo code for zombie object generation is as follows
CLS = object_getClass(self); Const char *zombieClsName = "_NSZombie_" + clsName; const char *zombieClsName = "_NSZombie_" + clsName; ZombieCls = objc_lookUpClass(zombieClsName); zombieClsName = objc_lookUpClass(zombieClsName); if (! ZombieCls) {//5, get zombie object Class _NSZombie_ Class baseZombieCls = objc_lookUpClass(" _NSZombie_"); //6, Create zombieClsName class zombieCls = objc_duplicateClass(baseZombieCls, zombieClsName, 0); } // delete member variables and associated references from the object if the memory has not been freed. objc_destructInstance(self); // Change the isa pointer to objc_setClass(self, zombieCls);Copy the code

When the zombie object is accessed again, the message forwarding process starts, the zombie object access process starts, logs are generated, and crash occurs

  • So the zombie object triggers the flow pseudocode as follows
CLS = object_getClass(self); Const char *clsName = class_getName(CLS); _NSZombie_ if (string_has_prefix(clsName, Const char *originalClsName = substring_from(clsName, 10); Const char *selectorName = sel_getName(_cmd); If (' *** * - [%s %s]: message sent to deallocated instance %p", originalClsName, selectorName, self); // Abort ();Copy the code

To sum up, the idea behind this midfield pointer probe is to replace the dealloc method by calling objc_destructInstance to de-reference the object

Wild pointer detection implementation 2

This way of thinking is mainly the source of sindrilin source code, its main idea is:

  • Flow of wild pointer detection

    • 1. Enable wild pointer detection

    • 2. Set the callback block to monitor the wild pointer, print information in the block, or store the stack

    • 3. Check whether the wild pointer crashes

    • 4. Maximum memory footprint

    • 5, whether to record the dealloc call stack

    • 6. Monitoring policies

      • 1) Only custom objects are monitored
      • 2) Whitelist strategy
      • 3) Blacklist policy
      • 4) Monitor all objects

    • 7. Swap NSObject dealloc methods

  • Trigger wild pointer

    • 1. Start processing objects

    • 2. Whether the substitution conditions are met

      • 1) According to the monitoring policy, whether it belongs to the class to be detected
      • 2) Whether the space is enough

    • 3. If the conditions are met, the object is obtained and dereferenced. If not, the object is released normally, that is, the original dealloc method is called

    • 4. Populate the object with data

    • 5, assign the zombie object class pointer to replace ISA

    • Object + Dealloc call stack, saved in zombie object

    • 7. Clear memory and objects according to the situation

Through the implementation of zombie object detection ideas

  • 1. Pass the OCMehod SwizzlingExchange,The root classes NSObject and NSProxythedeallocMethods forCustom deallocmethods
  • 2, in order toAvoid overwriting wild Pointers after memory space is freedThe question passedDictionaries store freed objectsAnd set it to30 seconds later, call the dealloc method to release the objects stored in the dictionary to avoid memory enlargement
  • 3. To get more crash information, you also need to create a subclass of NSProxy

The specific implementation

  • 1. Create a subclass of NSProxy, whose implementation is the same as the one aboveMIZombieProxyIt’s exactly the same
  • 2, the concrete implementation of Hook dealloc function
<! H --> @interface MIZombieSniffer: NSObject /*! */ + (void)installSniffer; / *! * @method uninstallSnifier */ + (void)uninstallSnifier; / *! * @method appendIgnoreClass * add a whitelist Class */ + (void)appendIgnoreClass: (Class) CLS; @end <! M --> #import "mizombiesniffer. h" #import <objc/runtime.h> // typedef void (*MIDeallocPointer) (id objc); Static BOOL _enabled = NO; // static NSArray *_rootClasses = nil; Static NSDictionary<id, NSValue*> *_rootClassDeallocImps = nil; Static NSMutableSet *__mi_sniffer_white_lists(){static NSMutableSet *mi_sniffer_white_lists; // Singleton initialization whitelist static dispatch_once_t onceToken; dispatch_once(&onceToken, ^{ mi_sniffer_white_lists = [[NSMutableSet alloc] init]; }); return mi_sniffer_white_lists; } static inline void __mi_dealloc(__unsafe_unretained id obj){// Obtain the object's Class Class currentCls = [obj Class]; Class rootCls = currentCls; // Get non-Nsobject and NSProxy classes while (rootCls! = [NSObject class] && rootCls ! = [NSProxy class]) {// Get the parent class of rootCls and assign rootCls = class_getSuperclass(rootCls); } // get the class name NSString *clsName = NSStringFromClass(rootCls); Mideallocimp = NULL; midealLocimp = NULL; [[_rootClassDeallocImps objectForKey:clsName] getValue:&deallocImp]; if (deallocImp ! = NULL) { deallocImp(obj); }} static inline IMP __mi_swizzleMethodWithBlock(Method Method, /* imp_implementationWithBlock: Taking a block argument, copying it into the heap, and returning a trampoline can be used as an implementation of any of the classes' methods, */ IMP blockImp = imp_implementationWithBlock((__bridge ID _Nonnull)(block)); Method_setImplementation replaces method's IMP return method_setImplementation(method, blockImp); } @implementation MIZombieSniffer + (void)initialize {_rootClasses = [@[[NSObject class], [NSProxy class]] retain]; } #pragma mark - public + (void)installSniffer{ @synchronized (self) { if (! _enabled) {// Hook root class dealloc method [self _swizzleDealloc]; _enabled = YES; }}} + (void)uninstallSnifier{@synchronized (self) {if (_enabled) {self _unswizzleDealloc]; _enabled = NO; AppendIgnoreClass :(Class) CLS {@synchronized (self) {NSMutableSet *whiteList = __mi_sniffer_white_lists(); NSString *clsName = NSStringFromClass(cls); [clsName retain]; [whiteList addObject:clsName]; } } #pragma mark - private + (void)_swizzleDealloc{ static void *swizzledDeallocBlock = NULL; // define block as method IMP static dispatch_once_t onceToken; dispatch_once(&onceToken, ^{swizzledDeallocBlock = (__bridge void *)[^void(id obj) { NSString *clsName = NSStringFromClass(currentClass); If ([__mi_sniffer_white_lists() containsObject: ClsName]) {// If the object is whitelisted, the object __mi_dealloc(obj) is released directly; } else {// Modify the isa pointer to the object to MIZombieProxy /* valueWithBytes:objCType creates and returns an NSValue object containing the given value, which will be interpreted as a given NSObject type - parameter 1: ObjVal = [NSValue valueWithBytes: &obj objCType: */ NSValue *objVal = [NSValue valueWithBytes: &obj objCType: Object_setClass (obj, [MIZombieProxy]); // Set the specified class for obj. // Keep the object's original class ((MIZombieProxy *)obj). OriginClass = currentClass; // Set dealloc to be called 30 seconds later to release the stored object. Dispatch_after (dispatch_time(DISPATCH_TIME_NOW, (int64_t)(30 * NSEC_PER_SEC)), dispatch_get_main_queue(), DISPATCH_TIME_NOW (int64_t)(30 * NSEC_PER_SEC)) ^{__unsafe_unretained id deallocObj = nil; // Obtain the required dealloc object [objVal getValue: Object_setClass (deallocObj, currentClass); // Release __mi_dealloc(deallocObj);}} copy]; }); // The dealloc method is originalDeallocImp NSMutableDictionary *deallocImps = [NSMutableDictionary dictionary]; // Get dealloc Method oriMethod = class_getInstanceMethod([rootClass) class], NSSelectorFromString(@"dealloc")); IMP originalDeallocImp = __mi_swizzleMethodWithBlock(oriMethod, swizzledDeallocBlock); [deallocImps setObject: [NSValue valueWithBytes: &originalDeallocImp objCType: @encode(typeof(IMP))] forKey: NSStringFromClass(rootClass)]; } //_rootClassDeallocImps dictionary store after the IMP implementation _rootClassDeallocImps = [deallocImps copy]; } + (void) _unswizzleDealloc {/ / reduction dealloc exchange IMP [_rootClasses enumerateObjectsUsingBlock: ^ (Class rootClass, NSUInteger idx, BOOL * _Nonnull stop) { IMP originDeallocImp = NULL; NSString *clsName = NSStringFromClass(rootClass); [[_rootClassDeallocImps objectForKey:clsName] getValue:&originDeallocImp];  NSParameterAssert(originDeallocImp); OriMethod = class_getInstanceMethod([rootClass], NSSelectorFromString(@"dealloc")); // Implement dealloc method_setImplementation(oriMethod, originDeallocImp);}]; // release [_rootClassDeallocImps release]; _rootClassDeallocImps = nil; } @endCopy the code
  • 3, test,
@interface ViewController ()

@property (nonatomic, assign) id assignObj;

@end

@implementation ViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    
    id obj = [[NSObject alloc] init];
    self.assignObj = obj;
    
    [MIZombieSniffer installSniffer];
}
- (IBAction)zombieObjectAction:(id)sender {

    NSLog(@"%@", self.assignObj);
    
}
Copy the code

The crash information is displayed as follows

Refer to the article

  • Quality control – Wild pointer location
  • IOS wild pointer processing
  • IOS wild pointer location: Wild pointer sniffer
  • Summary of iOS wild pointer location
  • IOS Zombie Objects principles explore

supplement

Github source link