IOS Low-level exploration series

  • IOS low-level exploration – alloc & init
  • IOS underlayer exploration – Calloc and Isa
  • IOS Low-level exploration – classes
  • IOS Low-level exploration – cache_t
  • IOS Low-level exploration – Methods
  • IOS Low-level exploration – message lookup
  • IOS Low-level exploration – Message forwarding
  • IOS Low-level exploration – app loading
  • IOS low-level exploration – class loading
  • IOS Low-level exploration – classification loading
  • IOS low-level exploration – class extension and associated objects
  • IOS Low-level exploration – KVC
  • IOS Basics – KVO

IOS leak check and fill series

  • IOS leak fix – PerfromSelector
  • IOS bug fix – Threads
  • – RunLoop for iOS
  • IOS – LLVM & Clang

In the last article, we explored iOS object alloc and init, and how objects create and initialize memory. If you add attributes to an object, do they affect memory creation? One remaining problem is that through Calloc, our object has a memory address, but how does the ISA in the object structure relate to our object’s memory address?

A,callocExplore the underlying

Before we explore the calloc underlayer, let’s fill up on memory alignment.

1.1 Three Principles for Memory alignment

In iOS, the properties of objects need to be memory-aligned, and the objects themselves need to be memory-aligned. There are three principles for memory alignment

  • Data member alignment principles: Structure (struct)(or combined (union) data member, no

    A data member is placed at offset 0, and the starting location of each data member is to be stored

    From the size of the member or the size of the member’s children
  • Struct as members: If a structure has some struct members, the struct members are stored from an integer multiple of the size of the largest element in the structure
  • Finishing off: The total size of the structure, i.esizeofThe result,. Must be its internal maximum

    An integer multiple of the members. Any deficiencies must be made up.

    Translation:
  • The preceding address must be a multiple of the following address
  • The size of the nested structure inside the structure is an integer multiple of the size of the largest element in the nested structure
  • The * * * ***Struct** Must be an integer multiple of the largest byte

1.2 Object Memory allocation and system memory allocation

We do this by printing the following code:

NSLog(@"%lu - %lu",class_getInstanceSize([p class]),malloc_size((__bridge const void *)(p)));
Copy the code

You can see that the size of memory allocated by the object itself is different from the actual size allocated by the system. In this case, the object allocates 40 bytes of memory, while the system allocates 48 bytes.

40 bytes is not hard to understand because the current object has four attributes, three of which are 8 bytes, one of which is 4 bytes, plus 8 bytes of ISA, which is 32 + 4 = 36 bytes, and 36 is not divisible by 8 according to memory alignment rules, 36 moved back to 40 is a multiple of 8, so the memory size is 40.

48 bytes requires us to explore the underlying principles of Calloc.

One other thing to note here is that class_getInstanceSize and malloc_size return different results for the same object, because malloc_size is the size of the pointer after calloc that was returned directly. Recall from the last video, Here is a step before calloc is called:

size_t instanceSize(size_t extraBytes) {
    size_t size = alignedInstanceSize() + extraBytes;
    // CF requires all objects be at least 16 bytes.
    if (size < 16) size = 16;
    return size;
}
Copy the code

The internal implementation of class_getInstanceSize is:

size_t class_getInstanceSize(Class cls) { if (! cls) return 0; return cls->alignedInstanceSize(); }Copy the code

That is, class_getInstanceSize will output 8 bytes and malloc_size will output 16 bytes if the object has no attributes.

1.3 Explore calloc bottom layer

We started with the calloc function, but we can’t find the corresponding implementation directly in the libObjc source code. By looking at Xcode, we know that we should look for libMalloc source code:

There is a trick here. We are working on the underlying principles of Calloc, and libObjc and libMalloc are independent of each other, so there is no need to follow calloc in libMalloc source code. The second parameter is 40: (this is because the object currently sending the alloc message has 4 properties of 8 bytes each, plus 8 bytes of isa, so it is 40 bytes)

Next, open the source code for libMalloc and manually declare the following code in a new target:

void *p = calloc(1, 40);
NSLog(@"%lu",malloc_size(p));
Copy the code

But after Command + Run we get an error message:

At this time, we will use the search method, directly Command + Shift + F to search the corresponding symbols globally, but it will not be found. After careful observation, these symbols are located in the.o file, so we can remove the underline before the symbols and then search. At this point, you can comment out the corresponding code and run it again.

After we run it, we’re going to follow the source break point, and we’re going to come to this piece of code

ptr = zone->calloc(zone, num_items, size);
Copy the code

If we go directly to Calloc, it’s going to recurse, so we need to click on it, and then we’ll see something very complicated coming up:

Here we can print the line directly at the breakpoint using the LLDB command to see which file the implementation is in

P zone->calloc (void *(*)(_malloc_zone_t *, size_t, size_t)) $1 = 0x00000001003839c7 (.dylib`default_zone_calloc at malloc.c:249)Copy the code

That is, the actual implementation of zone->alloc is at line 249 of the malloc.c source file.

static void *
default_zone_calloc(malloc_zone_t *zone, size_t num_items, size_t size)
{
	zone = runtime_default_zone();
	
	return zone->calloc(zone, num_items, size);
}
Copy the code

Zone ->calloc again. We then print the memory address again using LLDB:

P zone->calloc (void *(*)(_malloc_zone_t *, size_t, size_t)) $0 = 0x0000000100384faa (.dylib`nano_calloc at nano_malloc.c:884)Copy the code

Again, we come to the nano_calloc method

static void *
nano_calloc(nanozone_t *nanozone, size_t num_items, size_t size)
{
	size_t total_bytes;

	if (calloc_get_size(num_items, size, 0, &total_bytes)) {
		return NULL;
	}

	if (total_bytes <= NANO_MAX_SIZE) {
		void *p = _nano_malloc_check_clear(nanozone, total_bytes, 1);
		if (p) {
			return p;
		} else {
			/* FALLTHROUGH to helper zone */
		}
	}
	malloc_zone_t *zone = (malloc_zone_t *)(nanozone->helper_zone);
	return zone->calloc(zone, 1, total_bytes);
}
Copy the code

_nano_malloc_check_clear = _nano_malloc_check_clear = _nano_malloc_check_clear = _nano_malloc_check_clear After analysis, we come to segregated_size_to_fit

static MALLOC_INLINE size_t
segregated_size_to_fit(nanozone_t *nanozone, size_t size, size_t *pKey)
{
	// size = 40
	size_t k, slot_bytes;

	if (0 == size) {
		size = NANO_REGIME_QUANTA_SIZE; // Historical behavior
	}
	// 40 + 16-1 >> 4 << 4
	// 40-16*3 = 48

	//
	/ / 16
	k = (size + NANO_REGIME_QUANTA_SIZE - 1) >> SHIFT_NANO_QUANTUM; // round up and shift for number of quanta
	slot_bytes = k << SHIFT_NANO_QUANTUM;							// multiply by power of two quanta size
	*pKey = k - 1;													// Zero-based!

	return slot_bytes;
}
Copy the code

The size we passed in is 40. After (40 + 16-1) >> 4 << 4, the result is 48, which is an integer multiple of 16.

Conclusion:

  • The attributes of the object are 8-byte aligned
  • The object itself is 16-byte aligned
    • Because memory is contiguous, 16-byte alignment avoids risk and fault tolerance and prevents access overflow
    • At the same time, it also improves the efficiency of addressing access, i.e., space swap time

Second,isaExplore the underlying

2.1 Union bit field

union isa_t {
    isa_t() { }
    isa_t(uintptr_t value) : bits(value) { }

    Class cls;
    uintptr_t bits;
#if defined(ISA_BITFIELD)
    struct {
        ISA_BITFIELD;  // defined in isa.h
    };
#endif
};
Copy the code

When we explore the isa, will find that isa is a consortium, which actually is from memory management level design, because all the members of the association is to share a memory, association memory depends on the size of the largest elements, members of the internal memory size for the isa pointer, need not declared a lot of additional properties, Save information directly in the internal ISA_BITFIELD. And because the union attributes are mutually exclusive, CLS and bits are assigned in both branches during isa initialization.

2.2 the isa structure

Isa, as a union, has a structure with the attribute ISA_BITFIELD, which is 8 bytes, or 64 bits. The following code is based on the ARM64 architecture:

#   define ISA_BITFIELD                                                      \
      uintptr_t nonpointer        : 1;                                       \
      uintptr_t has_assoc         : 1;                                       \
      uintptr_t has_cxx_dtor      : 1;                                       \
      uintptr_t shiftcls          : 33; /*MACH_VM_MAX_ADDRESS 0x1000000000*/ \
      uintptr_t magic             : 6;                                       \
      uintptr_t weakly_referenced : 1;                                       \
      uintptr_t deallocating      : 1;                                       \
      uintptr_t has_sidetable_rc  : 1;                                       \
      uintptr_t extra_rc          : 19
Copy the code
  • nonpointer: indicates whether it is correctisaPointer Pointer optimization is enabled
    • Zero: pureisaPointer to the
    • 1: Not only class object address,isaContains class information, object reference counts, and so on
  • Has_assoc: flag bit of the associated object. 0 does not exist and 1 exists
  • Has_cxx_dtor: does the object have a destructor for C++ or Objc? If it has a destructor, the destructor logic needs to be done. If not, the object can be freed faster
  • Shiftcls: Stores the value of the class pointer. With pointer optimization turned on, 33 bits are used to store class Pointers in the ARM64 architecture.
  • Magic: Used by the debugger to determine whether the current object is a real object or has no space to initialize
  • Weakly_referenced: Indicates whether an object is pointed to or has been pointed to an ARC weak variable. Objects without weak references can be released faster.
  • Deallocating: Indicates whether the object is freeing memory
  • Has_sidetable_rc: When the object reference count is greater than 20, it needs to borrow this variable to store carry
  • Extra_rc: When representing the reference count of this object, the reference count is actually subtracted by 1. For example, if the object’s reference count is 10, the extra_rc is 9. If the reference count is greater than 20, the following has_sideTABLE_rc is used.

2.3 Isa associated objects and classes

Isa is the first property in an object, because this step takes place at inheritance time, before the object’s member variables, property list, method list, and protocol list follow.

When we were exploring the underlying mechanism of alloc, there was a method called initIsa.

This method initializes the ISA consortium bit field. Here’s a line of code:

newisa.shiftcls = (uintptr_t)cls >> 3;
Copy the code

From this line of code, we know that the bit field Shiftcls actually stores information about the class. This class is the class to which the instantiated object points.

Debugging printing with LLDB tells us that an object’s ISA is associated with the class that the object belongs to.

Isa’s ISA_BITFIELD isa bit-field of isa:

// Note: this is the X64 architecture
# elif __x86_64__
#   define ISA_MASK        0x00007ffffffffff8ULL
#   define ISA_MAGIC_MASK  0x001f800000000001ULL
#   define ISA_MAGIC_VALUE 0x001d800000000001ULL
#   define ISA_BITFIELD                                                        \
      uintptr_t nonpointer        : 1;                                         \
      uintptr_t has_assoc         : 1;                                         \
      uintptr_t has_cxx_dtor      : 1;                                         \
      uintptr_t shiftcls          : 44; /*MACH_VM_MAX_ADDRESS 0x7fffffe00000*/ \
      uintptr_t magic             : 6;                                         \
      uintptr_t weakly_referenced : 1;                                         \
      uintptr_t deallocating      : 1;                                         \
      uintptr_t has_sidetable_rc  : 1;                                         \
      uintptr_t extra_rc          : 8
Copy the code

As you can see, the first 3 bits of ISA_BITFIELD are Nonpointer, has_ASsoc, has_cxx_dtor, the middle 44 bits are Shiftcls, and the last 17 bits are the rest of the content. Meanwhile, since iOS is a little endient mode, Then we need to remove the three bits on the right and the 17 bits on the left, so >>3<<3 and then <<17>>17 will be used.

From this test, we know that ISA implements the association between objects and classes.

We can also explore the object_getClass layer and find a line like this:

return (Class)(isa.bits & ISA_MASK);
Copy the code

This line of code matches the union field in ISA with the previous mask. What is the definition of this mask?

#   define ISA_MASK        0x00007ffffffffff8ULL
Copy the code

The value 0x00007ffffffffff8ULL is converted to binary:

0000 0000 0000 0000 0111 1111 1111 1111 
1111 1111 1111 1111 1111 1111 1111 1000
Copy the code

As it turns out, the mask is helping us filter out content other than Shiftcls.

After we directly match the isa address of the object with the mask, we get the same memory address as object.class.

2.4 Isa position analysis

2.4.1 Classes and metaclasses

We all know that objects can be created more than one, but what about classes? The answer is simple, one. So what if we test it?

Class class1 = [LGPerson Class]; //MARK: void lgTestClassNum(){Class class1 = [LGPerson Class]; Class class2 = [LGPerson alloc].class; Class class3 = object_getClass([LGPerson alloc]); Class class4 = [LGPerson alloc].class; NSLog(@"\n%p-\n%p-\n%p-\n%p",class1,class2,class3,class4); } // The following output is displayed: 0x100002108-0x100002108-0x100002108-0x100002108Copy the code

So we know that there will only be one class in memory.

(lldb) x/4gx LGTeacher.class
0x100001420: 0x001d8001000013f9 0x0000000100b38140
0x100001430: 0x00000001003db270 0x0000000000000000
(lldb) po 0x001d8001000013f9
17082823967917874

(lldb) p 0x001d8001000013f9
(long) $2 = 8303516107936761
(lldb) po 0x100001420
LGTeacher
Copy the code

The first structure in the memory structure of the class is still LGTeacher, so does it mean that the object -> class -> class loop? The second class here is actually a metaclass. It was created by the system for us. This metaclass cannot be instantiated by us either.

This is the following relationship:

(lldb) p/x 0x001d8001000013f9 & 0x00007ffffffffff8 (long) $4 = 0x00000001000013f8 (lldb) po 0x00000001000013f8 LGTeacher  (lldb) x/4gx 0x00000001000013f8 0x1000013f8: 0x001d800100b380f1 0x0000000100b380f0 0x100001408: 0x0000000101c30230 0x0000000100000007 (lldb) p/x 0x001d800100b380f1 & 0x00007ffffffffff8 (long) $6 = 0x0000000100b380f0 (lldb) po 0x0000000100b380f0 NSObjectCopy the code

Isa go 2.4.2

We tested the following results in Xcode:

This gives the official classic ISA bitmap

2.5 ISA Initialization Flowchart

Third, the nature of the object

In our knowledge, OC object is essentially a structure, which can be confirmed in the objc-private.h source file of libObjc source code.

struct objc_object {
private:
    isa_t isa;

public:

    // ISA() assumes this is NOT a tagged pointer object
    Class ISA(a);

    // getIsa() allows this to be a tagged pointer object
    Class getIsa(a); . Leave out the rest... }Copy the code

The class of the object can also be found in the objc-Runtime-new.h source file

struct objc_class : objc_object {
    // Class ISA;
    Class superclass;
    cache_t cache;             // formerly cache pointer and vtable
    class_data_bits_t bits;    // class_rw_t * plus custom rr/alloc flags. Leave out the rest... }Copy the code

That is, the first place in objC_class memory is ISA and the second place is superclass.

However, in the spirit of authenticity, we can rewrite our OC source file with Clang to see if this is the case.

clang -rewrite-objc main.m -o main.cpp
Copy the code

This will compile our main.m file into C++ format and output it as main.cpp.

We can see that LGPerson object underneath is a structure called objc_Object.

And our Class is also an objc_class structure underneath.

Four,

Now that the object of iOS low-level exploration section has been updated, let’s review what we’ve explored.

  • Alloc & init process anatomy
  • memory
  • Byte alignment algorithm
  • Isa initialization and positioning
  • Nature of object

In the next chapter we will explore chapter classes, so stay tuned