Click “like” to see, wechat search [iOS growth refers to north] to guide the book. Welcome to share, please feel free to ask any questions
In this chapter, we look at different diagnostic options for solving memory problems based on Xcode.
Memory allocation basis
The iOS platform allocates memory for our applications on the stack or heap.
Whenever we create local variables within the scope of a function, memory is allocated on the stack. Every time we call the malloc method (or a variant of it), memory is allocated from the heap.
The minimum memory size allocated on the heap is 16 bytes (we won’t go into implementation details). This means that when we accidentally overwrite the smallest allocated memory, a small number of memory collisions may go undetected.
Once memory is allocated, it is placed in the virtual memory area. Virtual memory areas exist for allocating roughly the same amount of memory. For example, we have MALLOC_LARGE, MALLOC_SMALL, and MALLOC_TINY memory regions. This strategy is mainly to reduce the amount of memory fragmentation. In addition, there is a region for storing image bytes, the CGvimage region. This will allow memory to optimize system performance.
The difficulty with detecting memory allocation errors is that these symptoms can be confusing because adjacent memory may be used for different purposes, so one logical region of the system may interfere with unrelated regions of the system. In addition, problems are discovered much later than they are introduced because of possible delays (or wait times).
Address Sanitizer
For details, see Google open Source tools github.com/google/sani…
A very powerful tool to assist in memory diagnostics is called the Address Sanitizer. (called @ asanchecker)
It requires us to reconfigure our Scheme Settings for Address Sanitizer to recompile the code:
The Address Sanitizer performs a memory count (known as shadow memory). He knows which memory locations are ‘poisoned.’ That is, memory that has not been allocated (or allocated and then freed).
Address Sanitizer leverages the compiler directly, so when compiling code, any access to memory requires shadow memory to be checked to see if memory locations are freed. If this happens, an error report is generated.
This is a very powerful tool because it addresses the two most important categories of memory errors:
- Heap Buffer Overflow
- Heap Use After Free
The problem of Heap Buffer Overflow is that we will access more memory areas than we have allocated. The problem with Heap Use After Free is that we will access an area of memory that has been freed.
Address Sanitizer can further other types of memory problems, but is rarely encountered: stack buffer overflows, global variable overflows, C ++ container overflows, and return error after use.
The trade-off for this convenience is that our programs will run two to five times slower. But it is worth eliminating some of the problems in our continuous integration system.
Memory conflict Example
Consider the following code in the sample ICdab_edge program. @icdabgithub
- (void)overshootAllocated { uint8_t *memory = malloc(16); for (int i = 0; i < 16 + 1; i++) { *(memory + i) = 0xff; }}Copy the codeThis code allocates the minimum amount of memory, which is 16 bytes. It then writes to 17 consecutive memory locations. We get a heap overflow error.
This problem by itself will not crash our application immediately. However, if the device is rotated, it triggers a potential failure and causes a crash. By enabling Address Sanitizer, the application crashes immediately. This has a huge benefit. Otherwise, we might have wasted a lot of time debugging screen-rotation related code.
The Address Sanitizer error report is exhaustive. For demonstration purposes, we show only selected portions of the report.
An error report begins with the following:
==21803==ERROR: AddressSanitizer:
heap-buffer-overflow on address
0x60200003a5e0 at
pc 0x00010394461b bp 0x7ffeec2b8f00 sp 0x7ffeec2b8ef8
WRITE of size 1 at 0x60200003a5e0 thread T0
#0 0x10394461a in -[Crash overshootAllocated] Crash.m:48
Copy the codeThis is enough to switch to the code and begin to understand the problem.
Further details are provided to show that our access exceeded the allocated 16 bytes of memory area.
0 x60200003a5e0 is located 16 - byte 0 bytes to the right of region [0 x60200003a5d0, 0 x60200003a5e0) allocated by thread T0 here: #0 0x103bcdaa3 in wrap_malloc (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x54aa3) #1 0x1039445ae in -[Crash overshootAllocated] Crash.m:46Copy the codePlease note that the use of “half-open” digital coverage digital representation, which [including low range of index, while) ruled out higher range of index. Therefore, we visit to 0 x60200003a5e0 is beyond the scope of distribution of [0 x60200003a5d0, 0 x60200003a5e0).
We also get a memory “map” around the problem, truncating 0x1C0400007460 to…. for demonstration purposes 7460:
SUMMARY: AddressSanitizer: heap-buffer-overflow Crash.m:48 in -[Crash overshootAllocated] Shadow bytes around the buggy address: .... 7460: fa fa 00 00 fa fa fd fd fa fa fd fa fa fa fd fa .... 7470: fa fa 00 00 fa fa fd fa fa fa 00 00 fa fa fd fd .... 7480: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fa .... 7490: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa .... 74a0: fa fa 00 fa fa fa 00 00 fa fa fd fd fa fa 00 00 =>.... 74b0: fa fa 00 00 fa fa 00 00 fa fa 00 00[fa]fa fa fa .... 74c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 74d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 74e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 74f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 7500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: faCopy the codeFrom [FA] we see that we hit the first byte of the ‘Redzone’ (Poisoned memory).
(heap memory) after free use example
Consider the following code in the sample ICdab_edge program. @icdabgithub
- (void)useAfterFree
{
uint8_t *memory = malloc(16); // line 54
for (int i = 0; i < 16; i++) {
*(memory + i) = 0xff;
}
free(memory); // line 58
for (int i = 0; i < 16; i++) {
*(memory + i) = 0xee; // line 60
}
}
Copy the codeThe code allocates the smallest memory space (16 bytes), then writes to that memory, frees it, and then tries again to write to the same memory.
Address Sanitizer reports where we access memory that has been freed:
35711==ERROR: AddressSanitizer:
heap-use-after-free on address
0x602000037270 at
pc 0x000106d34381 bp 0x7ffee8ec9ef0 sp 0x7ffee8ec9ee8
WRITE of size 1 at 0x602000037270 thread T0
#0 0x106d34380 in -[Crash useAfterFree] Crash.m:60
Copy the codeIt tells us where to release:
0 x602000037270 is located 0 bytes inside of 16 - byte region [0 x602000037270, 0 x602000037280) freed by thread T0 here: #0 0x106fbdc6d in wrap_free (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x54c6d) #1 0x106d34318 in -[Crash useAfterFree] Crash.m:58Copy the codeIt tells us where memory was originally allocated:
previously allocated by thread T0 here:
#0 0x106fbdaa3 in wrap_malloc
(libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x54aa3)
#1 0x106d3428e in -[Crash useAfterFree] Crash.m:54
SUMMARY: AddressSanitizer: heap-use-after-free Crash.m:60 in
-[Crash useAfterFree]
Copy the codeFinally, it shows us a picture of the memory distribution around an incorrect address, truncated “0x1C0400006DF0” to “…. “for demonstration purposes 6 df0 “:
Shadow bytes around the buggy address: .... 6df0: fa fa fd fd fa fa 00 00 fa fa fd fd fa fa fd fa .... 6e00: fa fa fd fa fa fa 00 00 fa fa fd fa fa fa 00 00 .... 6e10: fa fa fd fd fa fa fd fa fa fa fd fd fa fa fd fa .... 6e20: fa fa fd fa fa fa fd fd fa fa fd fd fa fa fd fa .... 6e30: fa fa fd fa fa fa 00 fa fa fa 00 00 fa fa fd fd =>.... 6e40: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa[fd]fd .... 6e50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 6e60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 6e70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 6e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa .... 6e90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fdCopy the codeWe see that the entry [fd] represents a free write to memory.
Memory management tool
There’s a range of tools that complement Address Sanitizer. Use them after you turn off Address Sanitizer. They will discover some of the failures that Address Sanitizer might have missed.
In contrast to Address Sanitizer, these memory management tools do not require a project to be recompiled.
Guard Malloc tools
A major disadvantage of this tool is that it is only suitable for emulators. Each piece of memory is allocated in its own page, protected front and back. The tool has largely been replaced by the Address Sanitizer.
Malloc Scribble
The purpose of Malloc Scribble is to make the symptoms of memory errors predictable by marking Malloc or free memory to fixed known values. Allocated memory is marked as 0xAA and freed memory is marked as 0x55. It does not affect the behavior of allocating memory on the stack. It is not compatible with the Address Sanitizer.
If we have an application that crashes in a different way every time it runs, Malloc Scribble is a good choice. It will help make crashes predictable and repeatable.
Consider the following code in the sample ICdab_edge program. @icdabgithub
- (void)uninitializedMemory { uint8_t *source = malloc(16); uint8_t target[16] = {0}; for (int i = 0; i < 16; i++) { target[i] = *(source + i); }}Copy the codeFirst, the source is provided with newly allocated memory. Because this memory is not initialized, Malloc Scribble (and Address Sanitizer reset) is set to 0xAA in Scheme Settings.
Then, set target. It is a buffer on the stack (not heap memory). Using code = {0}, we set all memory locations of the buffer that applies the Settings to 0. Otherwise, it will be a ram value.
And then we go into a cycle. By making a breakpoint in the debugger (for example, in the second iteration), we can print out the memory contents and see the following:
We see that the target buffer is zero away from the first two index positions (0xAA). We see that source memory is always 0xAA.
If we do not set Malloc Scribble, the target buffer will be filled with random values. In complex programs, such data can be entered into other subsystems that affect program behavior.
Zombie Objects
The purpose of zombie objects is to detect errors in the objective-C environment after NSObject objects are freed. Especially if we have a legacy code base that uses manual reference counting, it’s easy to overfree objects. This means that passing messages through Pointers can have unpredictable effects.
This setting can only be done at debug build time because the code will no longer free objects. Its performance profile is equivalent to leaking all objects that should be released.
Turning on the configuration makes the release object an NSZombie object. Any message sent to an NSZombie object will cause an immediate crash. Therefore, whenever a message is sent to an object that has been freed, we can ensure that a crash occurs.
Consider the following code in the sample ICdab_edge program. @icdabgithub
- (void)overReleasedObject { id vc = [[UIViewController alloc] init]; // Build Phases -> Compile Sources // -> Crash.m has Compiler Flags setting // -fno-objc-arc to allow the following line to be called [vc release]; NSLog(@"%@", [vc description]); }Copy the codeWhen the above code is called, a crash occurs and the following is recorded:
2018-09-12 12:09:10.236058+0100 ICdab_edge [92796:13650378] *** -[UIViewController description]: message sent to deallocated instance 0x7fba1ff071c0Copy the codeLooking at the debugger, we see:
Note that the object instance VC is of type _NSZombie_UIViewController *.
This type will be the original type of the freed object, but prefixed with _NSZombie_. This is most helpful, and should be kept in mind when studying program state in the debugger.
The Malloc stack
Sometimes we need to understand the past dynamic behavior of our application to solve the cause of an application crash. For example, we may have a memory leak and then be terminated by the system due to excessive memory usage. We might have a data structure and want to know which part of the code is responsible for allocating it.
The purpose of the Malloc Stack option is to provide the historical data we need. Apple has enhanced memory analysis by providing additional visualization tools. The Malloc Stack has a suboption of “History of all allocations and releases” or “Live allocations only.”
We recommend using the “All Assign” option unless it is too expensive to use. That could be due to an application that uses a lot of memory allocation. The live allocation only option is sufficient to catch memory leaks and low overhead, making it the default option in the user interface.
Here are the execution steps:
- in
DiagnosticsSettings in the Settings TABMalloc StackOptions. - Start the application.
- Press the
Debug Memgraphbutton - Analysis based on command line tools,File -> Export Memory Graph…
The Memgraph visualization tool in Xcode is comprehensive but intimidating. There is a useful WWDC video to cover the basics @wwDC2018_416
There are often too many low-level details to check. The best way to use graphical tools is when we have some assumptions about how the application uses memory incorrectly.
Example of Malloc stack storage: Detect circular references
A simple and effective result is to see if we have a memory leak. These memory locations are inaccessible and cannot be freed.
We use the tvOS sample application ICdab_cycle to show how Memgraph finds circular references. @icdabgithub
By setting the Malloc Stack in Scheme Settings, we launch the application and press the Memgraph button, as shown below:
By pressing the exclamation mark filter button, we can filter to show only leaks:
File-> Export Memory Graph… To export the memory graph to icdab_cycle.memgraph, we can view the equivalent information from the Mac Terminal application using the following command:
leaks icdab_cycle.memgraph Process: icdab_cycle [15295] Path: /Users/faisalm/Library/Developer/ CoreSimulator/Devices/ 1616CA04-D1D0-4DF6-BE8E-F63541EC1EED/ data/Containers/Bundle/Application/ E44B9EFD-258B-4D0E-8637-CF374638D5FF/ icdab_cycle.app/icdab_cycle Load Address: 0x106eb7000 Identifier: icdab_cycle Version: ??? Code Type: x86-64 Parent Process: DebugServer [15296] Date/Time: 2018-09-14 16:38:23.008 +0100 2018-09-14 16:38:12.398 +0100 OS Version: Apple TVOS 12.0 (16J364) Report Version: 7 Analysis Tool: /Users/faisalm/Downloads/ Xcode.app/Contents/Developer/Platforms/ AppleTVOS.platform/Developer/Library/CoreSimulator/ Profiles/Runtimes/ tvOS.simruntime/Contents/ Resources/RuntimeRoot/Developer/Library/ PrivateFrameworks/DVTInstrumentsFoundation.framework/ LeakAgent Analysis Tool Version: IOS Simulator 12.0 (16J364) Physical Footprint: 38.9m Physical Footprint (peak): 39.0m ---- Leaks Report Version: 3.0 Process 15295:30252 Nodes malloced for 5385 KB Process 15295:3 Leaks for 144 total leaked bytes.leak: 0x600000d506c0 size=64 zone: DefaultMallocZone_0x11da72000 Song Swift icdab_cycle Call stack: [thread 0x10a974380]: | 0x10a5f678d (libdyld.dylib) start | 0x106eba614 (icdab_cycle) main AppDelegate.swift:12 ....Copy the codeThe code that causes the memory leak is:
var mediaLibrary: Album?
func createRetainCycleLeak() {
let salsa = Album()
let carnaval = Song(album: salsa,
artist: "Salsa Latin 100%",
title: "La Vida Es un Carnaval")
salsa.songs.append(carnaval)
}
func buildMediaLibrary() {
let kylie = Album()
let secret = Song(album: kylie,
artist: "Kylie Minogue",
title: "It's No Secret")
kylie.songs.append(secret)
mediaLibrary = kylie
createRetainCycleLeak()
}
Copy the codeThe problem is that Song’s instance object carnaval in the createRetainCycleLeak() method strongly references Album’s instance object Salsa, which in turn strongly references Song’s instance object Carnaval, And when we return from this method, we don’t refer to any object from any other object. These two objects are disconnected from the rest of the object graph and cannot be released automatically due to their strong references to each other (called circular references). A very similar relationship between objects such as Album’s instance object Kylie does not cause a memory leak because it is referenced by the top-level object mediaLibrary.
Usage of the dynamic linker API
Sometimes programs are dynamically adaptive or extensible. For such programs, the dynamic linker AP I will programmatically load additional code modules. When an application is configured or deployed incorrectly, it can crash.
To debug these issues, you need to set the Dynamic Linker API Usage. But this can generate a lot of information, which can cause problems on slower platforms with limited startup time, such as the original Apple Watch
A sample application using the dynamic linker is available on GitHub. @dynamicloadingeg
The output is as follows:
_dyld_get_image_slide(0x105545000)
_dyld_register_func_for_add_image(0x10a21264c)
_dyld_get_image_slide(0x105545000)
_dyld_get_image_slide(0x105545000)
_dyld_register_func_for_add_image(0x10a5caf39)
dyld_image_path_containing_address(0x1055d2000)
.
.
.
dlopen(DynamicFramework2.framework/DynamicFramework2)
==> 0x60c0001460f0
.
.
.
Copy the codeThis generates a lot of logging. It is best to search for the dlopen command and then see what other functions in the Dlopen family are called
Dynamic library loading
Application crashes sometimes occur during the initialization phase, when the dynamic loader is loading the application binaries and their dependent frameworks. If we are sure that it is not custom code using the Dynamic linker API, but rather assembling the framework into a loaded binary we are interested in, then choosing Dynamic Library Loads is appropriate. Compared to enabling Dynamic Linker API Usage, we get much shorter logs.
Once started, we get a list of loaded binaries:
dyld: loaded: /Users/faisalm/Library/Developer/
CoreSimulator/Devices/
99DB717F-9161-461A-B11F-210C389ABA12/
data/Containers/Bundle/Application/
D916AC0F-6434-46A3-B18E-5EC65D194454/
icdab_edge.app/icdab_edge
dyld: loaded: /Applications/Xcode.app/Contents/Developer/
Platforms/iPhoneOS.platform/Contents/Resources/RuntimeRoot/
usr/lib/libBacktraceRecording.dylib
.
.
.
Copy the codecommunication
The article continues to update every week, you can search “iOS growth refers to north” on wechat to read and urge more. Learn together and grow