How is the extreme Crash rate lower than 0.01% achieved?

IOS mobile phone butler, which has been in the mobile Internet for many years, has covered privacy (encrypted photo album), security (harassment blocking, SMS filtering), tools (network detection, photo cleaning, minimalist reminder, etc.) and other aspects after continuous iteration and innovation, providing professional security services for tens of millions of users. But at the same time, the engineering code is getting bigger and bigger (nearly 300,000 lines), a little problem will affect a large number of users, so Hand pipe has been working hard on the quality, and the Crash rate is pursued to the extreme. In recent iterations, special analysis has been made on Crash, and the Crash rate has steadily decreased from 0.02% to 0.01%, and version 7.7.1 approaches 0.009%. This paper will analyze and summarize two types of typical Crash cases.

I. Case analysis

Crash mainly occurs in objective-C method calls or system method calls, so two typical cases in this article are developed for OC and C method calls:

1.1. Crash occurs in objc_msgSend

Crash stack looks like this! Sigma (゚д゚ LLL)

Figure 1

Yes, I look at this stack and I only see one line of the project’s code stack, which is main, but we can find a way to break into the problem by looking deeper into the message mechanism of Objective-C.

Crash types

First we see that this is a Crash of type SEGV_ACCERR, accessing the wrong address.

Secondly, by analyzing the objc_msgSend method in assembly code, we can know that the line objc_msgSend + 16 (figure 2 below) reads the isa pointer offset 0x10 of the current OC method (see the appendix for objc_msgSend link article recommended). Since the object has been released, reading the address results in an incorrect read address, which results in a wild pointer Crash.

Figure 2

Find register

So, we look at the values of the registers at Crash (see Figure 3), where x0 is the first argument to the function that crashed, and for objc_msgSend x0 is also the address to the object that crashed, x1 is the second argument to the function that crashed, In objc_msgSend, the object that represents Crash calls a selector. RDM is kind enough to query this selector as respondsToSelector:. If X1 is a method that we wrote in our project, it’s easy to analyze the problem, just go to the project code, locate the function and find the reason, but respondsToSelector: there are too many places to call, what do we do? We still have to dig deeper.

Since the respondsToSelector: argument is a selector, you can immediately locate the problem code just by finding out what the selector is (which corresponds to the symbol in the symbol table of the query x2). Unfortunately, x2 is not in the address range of any of the modules in the Binary Images in the Crash report.

Figure 3

We know that the LR register is the address of a function call above the current function. If we know how the LR register executes, we can further determine the problem. Fortunately, the value of LR is within the address range of the steward module in Binary Images (see Figure 3, lr is 0x000000010508be44). The scope of the butler module is 0x104C24000-0x1055AFFff), so search for the corresponding symbol of LR in the symbol table, and get the following information: (below the MQQABC symbol table file for your app, in xcode package submitted need to be preserved, corresponding to the XXX. App. DSYM/Contents/Resources/DWARF/XXX)

Figure 4.

Figure 1 at this point, we know that only the main information of the stack of Crash is in – [MQQAlertView didDismissWithButtonIndex:] line 530, the cause of the Crash is called respondsToSelector:, That’s pretty close to the answer, but MQQAlertView is a generic popover component of the housekeeper, so you still need to know which page caused the Crash.

Locating the problem page

The handler uses RDM’s Crash reporting component to report attachments when a Crash occurs and stores some critical information on attachments (current ViewController stack, last released ViewController, applicationState, etc.). These attachment information can be viewed on the RDM platform, so we checked the attachment information and found that the Crash occurred when the user quit A certain page.

Find the cause of the problem →→

At this point, the path of the Crash was clear: the user to enter A, page page. A pop up A pop-up window, playing before the popup window users quickly exit pages, exit the page didn’t turn off the pop-up window, the user then click the pop-up window, because the window is the delegate of A page, and A page has been released, so led to visit the wild pointer.

The cause of the problem is identified, the problem code is located accurately, and the problem is not difficult to fix.

Note: objc_msgSend + 16 is a typical Crash stack caused by a wild pointer. This kind of problem can be solved smoothly according to the above ideas.

1.2. Crash occurs in C functions

Difficult Crash usually key stack are left on the system function, it also the pot to jilt to the system for us to find a good excuse, but to find a way to solve the problem is the goal, after all, the system is unable to help you carry the pan ¯ \ _ (ツ) _ / ¯ the following example is combined with the Crash report the information provided by the analysis of the typical case to solve the problem:

Figure 5

Several key messages can be seen from the Crash report:

1) The Crash type is also an invalid address SEGV_ACCERR. The invalid address is 0x68

2) Crash occurs in child Thread (Thread 7)

3) Crash = flockfile + 24

We debug the flockFile function in Xcode and locate it at + 24 (the breakpoint in Figure 6 below).

Figure 6.

LDR x8, [x19, #0x68] LDR x8, [x19, #0x68] LDR x8, [x19, #0x68] X19 + 0x68 = 0x68; x19 = 0; Mov x19, x0, x0 = 0; mov x19, x0 = 0;

Void flockfile (FILE *);

So, the FILE * pointer here is empty. Combined with the code call in the butler project in the stack:

– [MQQCBKAsdfUpdater mgPchAsdfCfgFileWithOFP:pFP:toFP:result:error:]

As you can see, three file paths were passed in, so the problem must be that one of the files does not exist. So far, this is the information we can analyze from the Crash report. Combined with the engineering code, we can conclude that the problem code is initially executed in the main thread, and dispatch to the child thread in the middle (from the Crash report). The state between threads is not well controlled, which leads to the deletion of files during the switchover to the execution of the child thread, leading to the Crash.

Two, method summary

The above analysis is only a review of the process. Many details have been omitted and are supplemented in this section. Since the main purpose of Crash analysis is to find out what happens to the function call when Crash occurs, this section is divided into several parts:

1) Function call convention for ARM64

2) Commonly used assembly instructions

3) Features of Objective-C function calls

4) Find the symbol table

5) The key information of the Crash report

2.1. ARM64 function call convention

Since the current mainstream models are iPhone 5S and above, we will only introduce the ARM64 here.

2.1.1. Registers for the ARM64 instruction set

Figure 7 (from ARM64 Reference manual)

The ARM64 instruction set has 31 64-bit general-purpose integer registers: X0 to X30 (w0 to W30 means to fetch only the lower 32 bits of these registers)

X0 through X7 are used to pass arguments and return the result from the subfunction (usually via x0, or at the execution address of X8 if it is a large structure).

LR: The X30 register, also known as the link register, generally holds the return address of the previous call

FP: r29, bottom stack register

Plus a stack top register SP

Stack 2.1.2.

The stack extends from the high address to the low address, the bottom of the stack is the high address, the top of the stack is the low address

Fp points to the low, or high, address of the current stack frame

Sp points to the top of the current stack frame, that is, the low address

Figure 8 shows the stack frame of _funcA calling _funcB:

Figure 8 (from the tech blog)

The first three lines of _funcB are shown in the assembly code of Figure 8:

The STP instruction on line 1 is to store the bottom pointer fp and link register LR of _funcA to the address of sp-0x10 on the top of _funcA, and set sp to SP-0x10 (fp_B in the figure), so as to return _funcA from _funcB and restore the stack frame of _funcA

Line 2 assigns SP to FP, setting the bottom pointer of _funcB (fp_B in figure)

Line 3 sets sp to SP-0x30. This completes the call from _funcA to _funcB.

2.1.3. Case Analysis

Here is an example of the argument passing of a function

Figure 9.

There are two methods as shown in Figure 9. OC method is a button click event, and C method above is called after clicking. For debugging convenience, C method has 11 parameters, and the values of the parameters in this example are 1 to 11.

To see the assembly code for the call procedure, we need to set a breakpoint in – (IBAction)testCmethodCall1 (ID)sender, and then set Always Show Disaasembly in Xcode (see Figure 10). So you’re looking at assembly code during debugging

Figure 10.

We break to the OC method with the assembly code shown in Figure 11

Figure 11.

Function call state switching

Sub sp, sp, #0x40 set new stack top register (sp)

Line 2: STP x29, x30, [sp, #0x30] Save the bottom stack register (X29 or FP) and the link register (x30 or LR)

Line 3: add x29, sp, #0x30 set fp (x29) to sp + 0x30, that is, set the new bottom stack register

These three lines complete the state switch required by the system to invoke the button click event method

Prepare the arguments for the C function

The next step up to STR w13, [sp, #0x8] is to prepare the arguments for calling the C method, which is verbose because it is not optimized.

ORR w8, WZR, #0x1 is an or instruction that assigns register zero or the value of 1 to register w8, i.e. W8 = 1, and similarly assigns 2 through 11 to register w9-w10, w3-w7, and w11-w13, respectively

Stur x0, [x29, #-0x8] Save x0 to x29-0x8

Stur x1, [x29, #-0x10] Save x1 to x29 -0x10

STR x2, [sp, #0x18] save x2 on sp + 0x18

Mov x0, x8 assign x8 (ORR w8, WZR, #0x1) to x0

Mov, x1, x9, same thing, assign 2 to x1

Mov x2, x10, w8, w9, w10, w8, w9, w10, w8, w9, w10, w8, w9, w10

Pass parameters on the stack

STR w11, [sp] w11 = 9 (mov w11, #0x9

Mov w12, [sp, #0x4] mov w12 10 (mov w12, #0xa

STR w13, [sp, #0x8] stores w13 (mov w13, #0xb) at the stack top offset 0x8

Calling C functions

At this point, the input arguments are ready, and the next call to bl 0x104FC237c is to call the C function

Figure 12

Into the C function assembly code, let’s clear this paragraph C functions under the task is to: return a1 + a2 + a11, so should be in the OC function w0, w1, w13 (w13 stack [sp, # 0 by 8]) take out the value of the combined, the result to x0, and back again, so:

Sub sp, sp, #0x30 set the top pointer of the stack to sp-0x30, so that the previous w13 stack position becomes [sp, #0x38], So you can see that the last red circle in Figure 12, LDR w1, [sp, #0x38] is actually loading w1 with the previous value of W13

STR w0, [sp, #0x2c], STR w1, [sp, #0x28], [sp, #0x28], [sp, #0x28], [sp, #0x28], [sp, #0x28], [sp, #0x28], [sp, #0x28], [sp, #0x28]

Add w0, w0, w1 add w0, w1 to w0 (calculate a1 + a2)

LDR w1, [sp, #0x38

Add w0, w0, w1 add the values of w0 and w1 to w0 (calculate a1 + a2 + a11), and now save the computed results in w0.

From the above analysis process, we can see:

• Assembly code at the beginning of a subfunction adjusts the FP and SP Pointers

• Use the X0-X7 registers for less than 8 parameter passes

• More than 8 are passed using the stack

• The return value of a subfunction generally exists in X0

• Because the registers X0, X1, X29, x30, etc., have special meanings, sometimes the values of these registers are stored on the stack before they are used

2.2. Commonly used assembly instructions

Section 2.1 has already touched on several assembly instructions. Here are some commonly used assembly instructions:

Mov A, b so a is equal to b

LDR a, [b] loads the contents of the address where pointer B is located into register A

STR a, [b] stores register A at the address pointed to by pointer B

LDR a, [b, #0x10] loads the contents of register A from register B address +0x10

ldr a, [b, #0x10]! The exclamation mark means to load the contents into register A and change register B to b = b + 0x10

CMP A and B compare the values of registers A and B, which changes the CPSR

CBZ xd, addr Checks whether the xD register is 0. If yes, jump to addr

CBNZ xd, addr Determines whether the xD register is not 0. If it is not 0, jump to addr

B jump instruction, does not modify the LR register, so the subfunction call procedure does not appear on the stack

The BL jump instruction modifies the LR register so that the subfunction call procedure appears on the stack

STP A, B, [c] takes two 64-bit values from address C and stores them in registers A and B respectively

LDP A, B, [c] stored the values in registers A and B to address C

2.3. Features of Objective-C function calls

Objective-c function calls are a special kind of function call, but ultimately that’s how they translate into C function calls.

We all know that objective-C calls end up calling objc_msgSend(ID self, SEL selector,…) And then use the previous knowledge to analyze objc_msgSend

As you can see, x0 is the call receiver, x1 is the call selector, followed by the arguments. For details, see the relevant article in the appendix.

2.4. Find the symbol table

Figure 13

There are Binary Images in the Crash report:

1) Start and end addresses of modules: For example, the start address of MQQABC module in Figure 13 is 0x104C24000 and the end address is 0x1055AFFff, so we can judge which module the address of a register we are interested in belongs to by the start and end addresses of these modules

2) UUID module, as shown in figure 13 MQQABC UUID is f130b043a0c832d9958d89dab8339961, it allows you to determine your symbol file is correct, as shown in figure 14 dwarfdump

Figure 14

3) Find the symbol corresponding to the address with atos. -L needs to provide the starting address of the module mentioned in 1)

Figure 15

If the address is still an address, check the __objc_methname in the __TEXT or __RODATA section of the mach-o file. (Note: the first red box (0x) is left out when otool looks for a mach-o file.)

Figure 16

2.5. Crash Indicates the key information of the report

Figure 17

Figure 18. Find key information in combination with register values

Figure 19. Determine symbol table UUID and start and end addresses

Iii. Appendix Reference

1. ARM64 reference manual: infocenter.arm.com/help/topic/…

2. The technology blog: blog.cnbluebox.com/blog/2017/0…

3. Analyze objc_msgSend assembly code: www.cocoachina.com/ios/2017080…

4. ARM64 assembly agreed: infocenter.arm.com/help/topic/…