The Bionic library is one of the basic libraries for Android and a bridge between Android and Linux. The Bionic library includes interfaces to many basic system functions, most of which come from Linux, but with many nuances from standard Linux. At the same time, some new modules (like log and Linker) have been added to the Bionic library to serve the upper layer of Android code.

takeaway

Because the Bionic library is so boring to learn, I almost never actively use it during the actual development process, but that doesn’t mean it doesn’t exist.

Bionic library learning, at least:

• To understandAndroidIn some system library implementation withLinuxImplementation differences
• To understandAndroidHow is it executedThe system callsthe
• To understandMemory management functionAnd meet againDoug LeaPops (guess whyagain)
• To understandThe pipeThis relatively primitive means of interprocess communication
• To understandAndroidIn thread management andLinuxThe difference between
• To know a man calledFutex(YY: Feels like JVM synchronization is similar to this)
• Learn about the ones we useLog.d()The concrete implementation and Log system structure
• To understand.so,.oWhat kind of files are they (Executable file) and what the generic structure looks like
• To understandDynamic linkCore module oflinkerHow does it work
• To know a man calledptraceSystem call, for laterHook APITo prepare for
• Learn about some common open source protocols
• Learn about some of Android’s changes and new features
• Deepen your understanding of Linux

Ahem, did you find so manyTo understandThe word? Because from the way my brain’s behaving these days, it’s not usuallypostureThe brain has a habit of forgetting and can onlyTo understandTo comfort myself…

——- end here ——-

What exactly is the Bionic library for? Take a look at the introduction.

BionicIntroduction to the

Bionic contains the most basic lib libraries in the system, including libc, libm, libdl, libstdc++, libthread, and the Android linker.

There were already mature open source GNU Libc libraries, but the GNU Libc libraries were under the GPL open source license. One thing about the GPL is that it’s contagious: once a system has software that uses the GPL license, the code for that system must be open source.

You can find an article on Google and the Linux kernel’s GPL constraints online for a little history. As for open source protocols, the article finally lists some common open source protocols.

Let’s move on to Bionic.

Bionic is created by Google by adding features of Linux on the basis of C library of BSD open source protocol. The Bionic name is a cross between BSD and Linux. The BSD protocol is an almost unlimited open source approach favored by commercial companies. (Universal love encryption and obfuscation is not)

In addition to copyright issues, performance is also the reason Google is redeveloping the LIBC library. Bionic is tailored to the limited CPU cycles and available memory on mobile devices (removing many advanced features related to processes, threads, and synchronization), reducing library size while improving productivity.

BionicIntroduction to modules in

Let’s take a look at Bionic’s directory structure (simplified version) :

├─ Android. Flag school ─ libc flag school ─ libdl flag school ─ libthread_db flag school ─ libm flag school ─ libstdc++ flag school ─

Compared to 9.0 and 5.0, this part is quite a big change, but I won’t go into the differences (mainly not familiar with Linux…). So I’ll focus on Bionic for now and can’t wait to see Binder

The Libc library

Libc library is the most basic C language library file, it provides all the basic system functions, these functions mainly come from its system call encapsulation, Libc library is the application and Linux kernel interaction bridge.

The Libc library provides the following functions:

• Process management: creates processes, sets scheduling policies, and adjusts priorities
• Thread management: including thread creation and exit, thread synchronization and mutual exclusion, thread local storage, etc.
• Memory management: Includes allocating and releasing memory
• Time management: Obtaining and saving the system time and obtaining the current system running duration
• Time zone management: Setting and adjusting the time zone.
• Timer management: Provides the system timing service.
• File system management: Allows you to mount and remove file systems.
• File management: creates, deletes, and modifies files or directories.
• Socket: Creates and listens to sockets. Send and receive data.
• DNS resolution: Helps resolve network addresses.
• Signal: Used for interprocess communication.
• Environment variables: Sets and gets system environment variables.
• Android Log: Provides the function of interacting with the Android Log driver.
• Android Properties: Manages a shared area to set and read Android properties.
• Standard IO: provides formatted input and output functions
• String: Allows you to move, copy, and compare strings
• Wide characters: provides for wide charactersUNICODE characters) support

Libm library

Mathematical function library, provides common mathematical functions and floating point operations. However, floating point arithmetic in Android is implemented by software, which is slower than hardware supported floating point arithmetic and is best avoided.

Libdl library

Libdl library is originally used for dynamic library loading, but the implementation of dlopen, dlclose, dlsym and other functions in Android libdl library is just an empty shell. Functions such as Dlopen used by application processes are actually implemented in the Linker module.

Libstdc++ library

Libstdc++ is the standard c++ library, but the implementation on Android is very simple, just a few operators such as new, delete, etc.

Linker

First of all, linker is not the linker used for compiling programs. The linker used for compiling programs is ARM-ELf-LD. (This is compilation time)

Linker loads dynamic libraries and relocates them, equivalent to ld.so in Linux. (This is runtime)

Instead of using Linux’s Ld. so, Android developed the Linker program itself

BionicC system calls in the library

It’s a little underclass. bite the dust and keep learning

Introduction to System Calls

A Linux system call is a set of interfaces that the Linux kernel provides for user processes to use. Through these interfaces, user processes can obtain services provided by the Linux kernel, such as opening and closing files, starting and exiting a process, allocating and freeing memory, and so on.

However, Linux system calls are not the same as ordinary API calls. API is the definition of a function, which specifies the function function, has no direct relation to the kernel. The system call is to send requests to the inner core through interrupts to realize some services provided by the kernel.

Modern cpus typically implement privilege levels (x86 cpus) or operating modes (ARM cpus)

The x86 CPU has four privilege levels Ring0 to Ring3.

• Ring0Can use any CPU instruction, and Linux kernel code runs at this level.
• Ring3Many CPU instructions are restricted, and code for ordinary user processes runs inRing3Under.

The ARM CPU has seven operating modes:

• User mode (usr) : The mode in which common user processes work and have limited permissions
• Fast interrupt mode (FIQ) : Used for high-speed data transmission or channel processing
• External interrupt mode (IRQ) : Used for generic interrupt handling
• Management mode (SVC) : Protected mode (high permission) used by the operating system, reset, and software interrupt access
• Data Access Termination mode (ABT) : This mode is entered when data or instruction prefetch terminates and can be used for virtual memory and storage protection
• System mode (SYS) : Runs all privileged operating system tasks
• Define instruction termination mode (UND) : Software emulation for supporting hardware coprocessors (floating-point, microcomputing)

No matter the privilege level of x86 or the working mode of ARM, the purpose is to separate the system kernel from the user process and prevent the user process from damaging the system kernel. At the same time, the system kernel can also provide services for user processes.

How does an application use system calls?

First, it’s important to understand that operating systems typically switch from user to kernel mode through interrupts.

Let’s take a look at the posture associated with interruption

Interrupt knowledge

Interrupts generally fall into three categories:

• An interruption caused by a computer hardware anomaly or failure is calledInternal abnormal interrupt;
• The interruption caused by the execution of the instruction causing the interruption in the program is calledsoftirqs(This is also an interrupt related to the system call we will describe,softirqsUsually an instruction with which the user can manually trigger an interrupt. ;
• An interrupt caused by a request from an external device is calledExternal interrupt.

In simple terms, interrupts are understood as stopping the current run to do something special.

Interrupts generally have two properties, interrupt number and interrupt handler.

Different interrupts have different interrupt numbers, each of which corresponds to an interrupt handler.

The kernel has an array called interrupt vector tables to map this relationship. When the interrupt comes, the CPU will pause the executing code, find the corresponding interrupt handler according to the interrupt number to the table and call the execution. When the execution is complete, the previous code is resumed.

Interrupt numbers are finite, so one interrupt does not correspond to one system call (there are many system calls). Linux triggers all system calls with int 0x80.

So how do you differentiate between different calls? For each system call there is a system call number, which is put into a fixed register before the interrupt is triggered. The value of this register is read by the interrupt handler corresponding to 0x80, which then decides which system call code to execute.

System calls for different platforms

On x86, applications invoke system functions using a soft interrupt 0x80; Arm platforms use SWI soft interrupts. At the same time, Linux numbers each system call (0 to NR_syscall) and keeps a system call table in the kernel, which stores the system call number and its corresponding service program.

On x86, the system call number is passed through the EAX register. Such as:

movl    $__NR_brk, %eax
 int    $0x80,     120
Copy the code

In addition to passing system call numbers using eAX, many system calls also need to pass some parameters to the kernel. The x86 platform uses registers EBX, ECX, EDX, ESI, and EDI in order to pass parameters. Such as:

mov    16(%esp),    %ebx
mov    20(%esp),    %ecx
mov    24(%esp),    %edx
movl   $__NR_write,  %eax
int    $0x80
Copy the code

In THE ARM platform, the system call in Android is implemented by swI soft interrupt 0, for example:

mov    ip,  r7
ldr    r7,  =__NR_brk
swi    0
Copy the code

== OK, we have a brief understanding of the interrupt content, but the instruction example is a bit confusing. Let’s explain this part separately

Interrupt – related instruction description

Register classification (see this great god’s article for details)

• 4 data registers:EAX,EBX,ECX,EDX
• Two indexing registers:ESI,EDI
• 2 pointer registers:ESP,EBP
• Six segment registers:ES,CS,SS,DS,FS,GS
• 1 instruction pointer register:EIP
• 1 flag register:EFlags

Interrupt instruction:

• Swi: ARM Software Interrupt (SWI)

  The command format is as follows: swI immed_24

  • Among them:immed_24The 24-bit immediate number is an integer ranging from 0 to 16777215. The kernel program uses this soft interrupt number to distinguish between different operations by users and execute different kernel functions.

• Int: x86 soft interrupt instruction

  The command format is as follows: int op_num

  • Among them:op_numIndicates the corresponding interrupt number

Related ARM instructions used above:

• mov: Data movement instruction between registers
• ldr: instructions for moving data between memory and CPU

X86 architecture instructions are used above:

• None of the x86ldrInstruction as x86movInstructions move data from memory to registers.
• movX: where x can be the following character:
  • lUsed for 32-bit long word values
  • wUsed for 16-bit word values
  • bUsed for 8-bit byte values

The instructions above are easier to understand with this part. Anyway to here first suspended Baidu, the more information the more problems… = =

System call implementation method

Bionic /libc/ ARCH-x86 /syscalls stores the assembly code of the system call (the implementation code of ARM and MIPS is in the directory of ARCH-ARM and ARCH-MIps).

Each system call is placed in a file, and only a small piece of assembly code is implemented in each file, called the Syscall stub.

/* Generated by gensyscalls.py. Do not edit. */
#include <private/bionic_asm.h>
ENTRY(mount)
    pushl   %ebx
    .cfi_def_cfa_offset 8
    .cfi_rel_offset ebx, 0
    pushl   %ecx
    .cfi_adjust_cfa_offset 4
    .cfi_rel_offset ecx, 0
    pushl   %edx
    .cfi_adjust_cfa_offset 4
    .cfi_rel_offset edx, 0
    pushl   %esi
    .cfi_adjust_cfa_offset 4
    .cfi_rel_offset esi, 0
    pushl   %edi
    .cfi_adjust_cfa_offset 4
    .cfi_rel_offset edi, 0

    call    __kernel_syscall
    pushl   %eax
    .cfi_adjust_cfa_offset 4
    .cfi_rel_offset eax, 0

    mov     28(%esp), %ebx
    mov     32(%esp), %ecx
    mov     36(%esp), %edx
    mov     40(%esp), %esi
    mov     44(%esp), %edi
    movl    $__NR_mount, %eax
    call    *(%esp)
    addl    $4, %esp

    cmpl    $-MAX_ERRNO, %eax
    jb      1f
    negl    %eax
    pushl   %eax
    call    __set_errno_internal
    addl    $4, %esp
1:
    popl    %edi
    popl    %esi
    popl    %edx
    popl    %ecx
    popl    %ebx
    ret
END(mount)
Copy the code

All are some assembly instructions, the specific content of the instruction is not detailed explanation, can refer to interrupt related instructions

Another point is that the code is not hand-written, but generated from the bionic/libc/SYSCALLS.TXT file using the gensyscalls.

SYSCALLS.TXT file:

# signals
int     __sigaction:sigaction(int, const struct sigaction*, struct sigaction*)  arm,mips,x86
int     __rt_sigaction:rt_sigaction(int, const struct sigaction*, struct sigaction*, size_t)  all
Copy the code

Standard format (this section 9.0 and 5.0 are not quite the same, the 9.0 format is listed) :

return_type    func_name[:syscall_name[:call_id]]([parameter_list])    platform
Copy the code

Divided into three parts:

• The first part is the return type of the function.
• The second part is the function name and parameters.
  • func_nameRefers to the function name, which is the name of the C program when it is called.
  • syscall_nameRefers to the name of the system call, which is an internal name used primarily to generate the macro definition of the system number. The method of generating macro definitions is insyscall_nameI’ll put a string in front of it__NR_. In order to_exitLet’s take the delta function, which is inSYSCALLS.TXTDefined in the file as

    void    _exit:exit_group(int)    all
    Copy the code

    The macro corresponding to the generated system number is defined as__NR_exit_group. These macros are defined in the Linux Kernel code.

  • call_idThe system call number is not usually listed. throughsyscall_nameThe generated macro definition gets the system call number.
  • parameter_listThis is the parameter list
• The third part is used to specify the target platform, including:all,arm,mips,x86

BionicMemory management functions in

Not to talk about it in detail, and then talk about it to deviate from my learning route in this book.

A few notes on memory

For traditional 32-bit processors, the addressing space is up to 4GB. The address space of 0 to 3 GB is allocated to user processes, and 3 to 4 GB is used by the kernel.

Rather than gaining access to all 0 to 3 GB address Spaces at startup, user processes need to apply to the kernel for read and write access to a specific address. What’s more, only address space is requested, and no actual physical memory is allocated.

Only when a process accesses a memory address, if the physical page corresponding to that address does not exist, the kernel generates a page-missing interrupt, during which physical memory is allocated and the page table is created. If user processes no longer need a block of address space, they can be freed by the kernel, and the corresponding physical memory is also freed.

Due to the slow running of page-missing interrupts, the performance of the entire system can be reduced if memory is frequently allocated and freed by the kernel. Therefore, the common operating system provides an address space allocation and reclamation mechanism in the user process, namely the memory manager:

• Memory managerA large address space, called the address space, is pre-requested from the kernelThe heap.
• When a user process needs to allocate memory, theMemory managerfromThe heapIs looking for a free chunk of memory to allocate to the user process.
• When a user process frees a block of memory,Memory managerIt is not immediately released by the kernel, but placed in the free list for the next allocation.

The memory manager also adjusts the heap size dynamically:

• When the heap space runs out, it continues to request the kernel
• When there is too much free memory in the heap, some of it is returned to the kernel

Linux memory management methods

Linux has two ways to allocate and free memory space. One is to use the system call BRK. The other is to use the system calls mmap and munmap.

The Linux kernel typically divides the user address space into large areas such as code area, data area, heap, stack, and so on.

Here’s a schematic:

brk

The system call BRK is used to adjust the high address bounds of the heap. Boundaries are pushed up when memory is allocated and pulled down when memory is freed.

The advantage of BRK is that it allocates memory quickly, but the disadvantage is that it may not allocate large chunks of memory. Because the area between heap and stack is not completely empty, some memory may have been allocated.

The high address boundary of the heap area will cause an allocation failure if it encounters an allocated area. Therefore, BRK is usually used to allocate relatively small memory space, such as memory blocks less than 256KB.

mmap

The system call Mmap is used to allocate large chunks of memory. Mmap looks for an appropriate space between the heap and stack to allocate to user processes.

• If the memory size is not appropriate, there is also a system callmremapTo change the size.
• After using, you can passmunmapFree up memory.

Bionic memory manager

Before Android 7.0, you could specify dlmalloc or Jemalloc as the memory manager. Later, 7.0 added a Project Svelte; Later, I couldn’t find the standalone source for DLmalloc in the 9.0 project.

Jemalloc’s Zhihu portal

And about DLmalloc, this part comes from Baidu Baike ha.

Dlmalloc is a very popular memory allocator. It was written by Doug Lea in 1987. Up to now, the latest version is 2.8.3, and it is widely used and studied for its high efficiency and other characteristics.

Dlmalloc implementation is only one source file (and a header file), about 5000 lines, which notes accounted for a large amount of space, because there are so many comments, on the surface, easy to understand, indeed, without the pursuit of detail, the general idea is easy to understand (that’s right, you just know), However, dlmalloc, as a high-quality masterpiece, uses a lot of skills in its implementation. Without spending a certain amount of energy on the implementation details, there is no way to deeply understand why it does so and what the benefits of doing so are. Only when you really understand it, you will find that it is so wonderful after aftertaste.

This is theJava multi-threadThen I heard againDoug LeaThe multithreaded core was almost a one-handed invention by Pops (especiallyAQS), what a genius

Even today, the technology in DLmalloc is behind The Times in some places, and there are many excellent Allocators, such as Google’s TCmalloc, freeBSD’s Jemalloc, whose performance can reach tens or even hundreds of times that of DLmalloc in some cases. But many of dlmalloc’s ideas and basic algorithms have had a profound impact on later generations.

dlmallocA brief understanding of

dlmallocSource download link

Dlmalloc groups the free space of the heap according to size in a linked list. These lists can be used to quickly find the appropriate size of memory when allocating memory. If the required free space cannot be found, DLmalloc uses system calls to expand the heap space

Take a look at the block diagram first:

Dlmalloc’s memory blocks are called trunks, and the size of each block must be aligned by address (8 bytes by default), so the size of a Trunk block must be a multiple of 8.

Dlmalloc uses three different linked list structures to organize free memory blocks of different sizes:

• Blocks smaller than 256 bytes use structuresmalloc_chunkOrganized by size. Since the space is less than 256 bytes, a maximum of 32 bytes can be usedmalloc_chunkStructure of the circular linked list to organize blocks less than 256 bytes
• Blocks larger than 256 bytes are structuredmalloc_tree_chunkThe blocks are organized into a binary tree according to size
• When larger than a certain size (DEFAULT_MMAP_THRESHOLDIf the default threshold is 256 KB, the system passesmmapThe way to allocate a piece of memory space separately and through the structuremalloc_segmentComposed of linked lists for management

dlmallocProcess of Allocating Memory

Search these lists to quickly find a free chunk of memory that best matches the required size (thus avoiding fragmentation as much as possible). If there is no memory block of suitable size, divide a large memory block into two parts, allocate one part, and add the other part to the corresponding free list according to the size.

dlmallocFreeing Memory process

Adjacent free blocks are merged into one large block to reduce memory fragmentation. If the number of free blocks exceeds a threshold set internally by DLmalloc, DLmalloc starts returning memory to the system (presumably to the kernel).

dlmallocA simple description of the function of

The book also introduces some specific functions do not extend ha

In addition to managing the heap space of processes, DLmalloc can also provide private heap management. A private heap is a separate block of address space allocated outside the heap, managed by DLmalloc in the same way. Private heap functions can be distinguished by the following characteristics:

• dlmallocIs used to manage processesThe heapAll of the functions of spacedlThe prefix, such asdlmalloc,dlfreeEtc.
• The management function for the private heap doesmspace_The prefix, such asmspace_malloc,mspace_freeEtc.

The pipe

Pipes are a means of interprocess communication emerging from Unix systems. They are divided into anonymous pipes (PIPE) and named pipes (FIFO).

Historically, pipes were half-duplex, meaning that data could only flow in one direction at a time. Some systems now offer full-duplex piping. But for portability, we assume that the system does not have this capability.

If you needIPC, the best practice is to useBinder; This can be used if only interthread communication is requiredAnonymous pipe

Anonymous pipeSome knowledge

Anonymous pipes are mainly used for communication between parent and child processes. Android support

Creating an anonymous pipe creates a memory file in the kernel (which is not visible in the file system) and two file descriptors. Pipe users use these two file descriptors to read and write data in memory files for information exchange.

In general, file descriptors cannot be passed between processes; only parent or sibling processes can share file descriptors by inheritance. Therefore, anonymous pipes are mainly used for communication between parent and child processes.

Anonymous pipeBelong toHalf duplexThe data can only go from one end to the other. The schematic diagram is as follows:The processes on both sides of the pipe treat the pipe as the same file, with one process writing to the pipe and the other reading from the pipe. If two-way communication is required between processes, there is no need to set up two pipes.

Let’s take a look at the characteristics of anonymous pipes:

• Only one-way communication is provided, that is, both processes can access the file, and if process 1 writes to the file, process 2 can only read the contents of the file.
• Can only be used for interprocess communication that has a kinship relationship, usually for parent-child communication
• Pipes communicate based on byte streams
• Depends on the file system, whose life cycle ends with the end of the process (along with the process)
• It has its own synchronous mutex effect

The biggest advantage of anonymous pipes is that they are simple and flexible, but being able to communicate between parent and child processes limits their use. Hence, named pipes later appeared.

A named pipeThe characteristics of

A named pipe, also known as a first-in, first-out queue (FIFO), is a special pipe that exists in a file system by establishing an inode node. Mkfifo and FAT32 file system formats are not supported by Android.

Named pipes are very similar to anonymous pipes, but have distinct characteristics of their own:

• A named pipeCan be used for communication between any two processes, not just the same process.
• A named pipeStored as a special file in the file system, not likeAnonymous pipeThat’s stored in the kernel. When the process ofA named pipeAfter the end of the use ofA named pipeThe named pipe still exists in the file system and will not disappear unless it is deleted.

Like anonymous pipes, named pipes can only be used for one-way data transfer. If you want to use named pipes to implement two-way data transfer between two processes, you are advised to use two one-way named pipes.

BionicThread management function in

The thread management functions in Bionic are quite different from the implementation of the unified Linux version, and Android does a lot of tailoring to suit its needs.

BionicFeatures of thread functions

Android pthread threads management related source implementation in bionic/libc/bionic/pthread *

Pthreads in Android are implemented based on Futex, and use shorter code to implement common operations. Briefly note some features:

• Pthread_mutex_t,pthread_cond_t defines a type of only 4 bytes

• Support normal, recursive, and error-check mutex attributes.

  • PTHREAD_MUTEX_NORMAL: This type of mutex does not automatically detect deadlocks. If a thread attempts to lock a mutex repeatedly, the thread will be deadlocked. Trying to unlock a mutex locked by another thread can cause unexpected results. A thread attempting to unlock a mutex that has already been unlocked can also cause unexpected results.
  • PTHREAD_MUTEX_ERRORCHECK: This type of mutex automatically detects deadlocks. If a thread attempts to lock a mutex repeatedly, an error code is returned. Attempts to unlock a mutex locked by another thread will return an error code. An error code is also returned if a thread attempts to unlock a mutex that has already been unlocked.
  • PTHREAD_MUTEX_RECURSIVE: If a thread locks this type of mutex repeatedly, it does not cause a deadlock. Multiple times a thread repeats a mutex must be unlocked by the same thread in order to unlock the mutex and for other threads to acquire the mutex. Attempts to unlock a mutex locked by another thread will return an error code. An error code is also returned if a thread attempts to unlock a mutex that has already been unlocked

• The pthread_cancel function is not supported. In its place are the pthread_cleanup_push, pthread_cleanup_POP, and pthread_exit functions.

In fact, this part also includes the introduction of pthread operation functions, TLS thread local storage, Mutex, Condition, feeling a bit in-depth. So let’s take a quick look at this and come back when you need to. (PS: Lazy)

FutexSynchronization mechanism

Futex is short for Fast UserSpace Mutex. Futex is a fundamental component of Linux that can be used to build various higher-level synchronization mechanisms, such as locks or semaphores. Futex is not only used in Android thread functions, but also in some modules as a means of interprocess synchronization.

Linux has supported Futex since 2.5.7. On Unix-like systems, traditional interprocess synchronization is accomplished by operating on a kernel object that is visible to all processes that need to be synchronized. This method is inefficient because it involves switching between kernel mode and user mode.

Futex’s solution is to operate entirely in user space without a race, requiring no system calls and only going into the kernel to wait or wake up when a race occurs. Therefore, Futex is a synchronization mechanism combining user mode and kernel mode, which requires the cooperation of the two modes. Futex variables must be located in user space instead of kernel objects. Futex code is also divided into user mode and kernel mode. In the case of no competition in user mode, when the competition occurs, the sys_FUtex system call will enter kernel mode for processing.

FutexSystem call

In Linux, Futex’s system call is as follows:

#define __NR_futex      240
Copy the code

The prototype of the corresponding Futex system call is:

#include <linux/futex.h>
#include <sys/time.h>
int futex (int *uaddr, int op, int val, const struct timespec *timeout,int *uaddr2, int val3);
Copy the code

Among them:

• *uaddr: is the address of shared memory in user mode, where an aligned integer counter is stored
• op: indicates the operation type. There are five predefined valuesBionicOnly the following two are used in:
  • FUTEX_WAIT: Atomic checkuaddrIs the value of the counter invalIf so, let the process sleep untilFUTEX_WAKEOr time out (timeout). That is to attach the process touaddrCorresponding waiting queue up.
  • FUTEX_WAKE: wake up at mostvalA waitinguaddrIn the process.

In the Bionic, provides two functions to packaging Futex system call (located in the Bionic/libc/private/bionic_futex. H) :

static inline int __futex_wait(volatile void* ftx, int value, const timespec* timeout) {
  return __futex(ftx, FUTEX_WAIT, value, timeout, 0);
}
static inline int __futex_wake(volatile void* ftx, int count) {
  return __futex(ftx, FUTEX_WAKE, count, NULL, 0);
}
Copy the code

There are two similar functions:

static inline int __futex_wake_ex(volatile void* ftx, bool shared, int count) {
  return __futex(ftx, shared ? FUTEX_WAKE : FUTEX_WAKE_PRIVATE, count, NULL, 0);
}
static inline int __futex_wait_ex(volatile void* ftx, bool shared, int value) {
  return __futex(ftx, (shared ? FUTEX_WAIT_BITSET : FUTEX_WAIT_BITSET_PRIVATE), value, nullptr,
                 FUTEX_BITSET_MATCH_ANY);
}
Copy the code

The _ex function takes one more shared argument than the other two:

• whensharedThe value oftrueSaid,waitandwakeOperations are used forBetween processesTo suspend and wake up
• whensharedThe value offalseSaid,waitandwakeOperations are used forBetween threads in a processTo suspend and wake up

FutexSynchronization logic of

First, clarify the state of Futex variable value:

• 0 indicates no lock.
• 1 indicates that there is a lock and no contention.
• 2 indicates that the state is in contention.

The process is as follows:

• Create a global integer variable asFutexVariable (an integer counter) with an initial value of 0. If used for inter-process synchronization, this variable must be in shared memory.
• Check when a process or thread holds a lockFutexWhether the variable is 0.
  • If it is 0, willFutexSet the variable to 1 and continue.
  • If it is not 0, willFutexAfter the variable is set to 2, runFUTEX_WAITThe system call enters the pending wait state.
• When a process or thread releases a lock
  • ifFutexIf the value of the variable is 1, no other threads are waiting for the lockFutexSet the variable to 0 and you’re done.
  • ifFutexThe value of the variable is 2, indicating that there are threads waiting for the lockFutexThe variable is set to 0 and executed simultaneouslyFUTEX_WAKESystem call to wake up the waiting process.

For Futex variable operations, the comparison and assignment operations need to be atomic.

Mutexclass

The Glibc library has user-mode lock interfaces such as pthread_mutex_lock() and pthread_mutex_unlock() to provide a fast FUTEX mechanism.

Bionic’s pthread implementation also provides a standard pthread_mutex_lock()/pthread_mutex_unlock() interface.

The Mutex class encapsulates the pthread_mutex_lock() and pthread_mutex_unlock() interfaces.

The source path in the book, however, has not make the path is on 9.0: the system/core/libutils/include/utils/Mutex. H, can under reference.

In the AndroidLogThe module

Since Android is developed in host-target mode, the main way to solve the problem is to analyze logs,

Let’s look at the architecture of the Log system:

The output of the Log system is divided into five levels:

• ERROR: Displays error information
• WARN: Displays warning information
• INFO: Displays general prompt information
• DEBUG: Displays debugging information
• VERBOSE: Used to output low-value information

Log sizing is not mandatory, but properly used can make debugging easier

Android has a large number of logs, especially for communication systems, so Android outputs logs to different buffers. Buffers currently defined include:

public static final int LOG_ID_MAIN = 0; Public static final int LOG_ID_RADIO = 1; Public static final int LOG_ID_EVENTS = 2; Public static final int LOG_ID_SYSTEM = 3; Public static final int LOG_ID_CRASH = 4; / / crash informationCopy the code

Buffers are defined primarily for use by system components. Java layer log. * prints to LOG_ID_MAIN.

Log System interfaces and usage

Java layer interface calls

Log.*(String tag, String MSG) in android.util.Log. Let’s look at a few special ones:

• Log.*(String tag, String msg, Throwable tr): the increasing number ofThrowable trIs to make it easier to print stack information in case of an exception.
• The Log. WTF (series): cooperate withsetWtfHandler(TerribleFailureHandler handler)Use, passsetWtfHandlerSet with a callback functionhandler, so in useLog.wtf(), such serious cases can be handled uniformly.

Java class Log.*(String tag, String MSG)

public static int d(String tag, String msg) {
    return println_native(LOG_ID_MAIN, DEBUG, tag, msg);
}
Copy the code

Println_native is a JNI calls, is located in the frameworks/base/core/JNI/android_util_Log CPP, we look at the part of the content:

static const JNINativeMethod gMethods[] = { /* name, signature, funcPtr */ { "isLoggable", "(Ljava/lang/String; I)Z", (void*) android_util_Log_isLoggable }, { "println_native", "(IILjava/lang/String; Ljava/lang/String;) I", (void*) android_util_Log_println_native }, { "logger_entry_max_payload_native", "()I", (void*) android_util_Log_logger_entry_max_payload_native }, }; static jint android_util_Log_println_native(JNIEnv* env, jobject clazz, jint bufID, jint priority, jstring tagObj, Jstring msgObj) {// omit int res = __android_log_buf_write(bufID, (android_LogPriority)priority, tag, MSG); // partially omitted}Copy the code

Println_native finally calls the __android_log_buf_write function defined in the System /core/liblog module.

Native layer calls

Native layer uses macro definition. Common forms include:

• ALOGV: the equivalent ofLog.v()
• ALOGD: the equivalent ofLog.d()
• ALOGW: the equivalent ofLog.w()
• ALOGI: the equivalent ofLog.i()
• ALOGE: the equivalent ofLog.e()

All of these instructions are printed to the LOG_ID_MAIN buffer. ALOGD, for example, we look at the definition file (system/core/liblog/include/log/log_main h) part of the content:

#ifndef ALOGD
#define ALOGD(...) ((void)ALOG(LOG_DEBUG, LOG_TAG, __VA_ARGS__))
#endif

#ifndef ALOG
#define ALOG(priority, tag, ...) LOG_PRI(ANDROID_##priority, tag, __VA_ARGS__)
#endif

#ifndef LOG_PRI
#define LOG_PRI(priority, tag, ...) android_printLog(priority, tag, __VA_ARGS__)
#endif

#define android_printLog(prio, tag, ...) \
  __android_log_print(prio, tag, __VA_ARGS__)
Copy the code

Finally is actually called __android_log_print function, we look at the realization of the function under the system/core/liblog logger_write. C file:

LIBLOG_ABI_PUBLIC int __android_log_print(int prio, const char* tag, const char* fmt, ...) Return __android_log_write(prio, tag, buf); } LIBLOG_ABI_PUBLIC int __android_log_write(int prio, const char* tag, const char* msg) { return __android_log_buf_write(LOG_ID_MAIN, prio, tag, msg); }Copy the code

Finally it also points to the __android_log_buf_write method, which we’ll take a closer look at in a moment

In addition, there are two sets of instructions:

• SLOG*: the output to theLOG_ID_SYSTEMThe buffer that defines the file issystem/core/liblog/include/log_system.h
• RLOG*: the output to theLOG_ID_RADIOThe buffer that defines the file issystem/core/liblog/include/log_radio.h

Java 和 nativeCall to follow up

__android_log_buf_write is called from the native layer and Java layer.

LIBLOG_ABI_PUBLIC int __android_log_buf_write(int bufID, int prio,
                                              const char* tag, const char* msg) {
  // omit some content
  return write_to_log(bufID, vec, 3);
}
static int __write_to_log_init(log_id_t, struct iovec* vec, size_t nr);
static int (*write_to_log)(log_id_t, struct iovec* vec,
                           size_t nr) = __write_to_log_init;
Copy the code

__android_log_buf_write calls write_to_log(bufID, vec, 3), and write_to_log is a pointer, Point to function __write_to_log_init (source code to make full use of write_to_log function pointer, transform pointer to the function to complete the log output).

According to the information in the book, it will finally point to the system call write() to print the output through the kernel layer log driver, this part is not very clear in 9.0, trace source only found the following part of the information:

static int __write_to_log_daemon(log_id_t log_id, struct iovec* vec, size_t nr) {
 // omit some content
  write_transport_for_each(node, &__android_log_transport_write) {
    if (node->logMask & i) {
      ssize_t retval;
      retval = (*node->write)(log_id, &ts, vec, nr);
      if (ret >= 0) { ret = retval; }}}// omit some content
  write_transport_for_each(node, &__android_log_persist_write) {
    if (node->logMask & i) {
      (void)(*node->write)(log_id, &ts, vec, nr); }}}Copy the code

The feeling is that write() here will end up in the system call write().

I’m going to stop there. I’m going to move onBionicIt’s taking a little long

Executable file format analysis

Before looking at Bionic’s Linker, let’s take a look at Android’s executable file format. Linker itself is not very complex, but it is not easy to understand the logic of a program without an understanding of the executable.

The Android executables and dynamic libraries are the Linux ELF file format, but since Android uses its own Linker, they are not fully compatible with normal Linux systems.

ELFIntroduction to File Formats

ELF is short for Executable and Linkable Format and was originally published by Unix LABS as part of the ABI. The PURPOSE of the ELF standard is to provide software developers with a set of binary interface definitions that work in multiple operating system environments, thereby reducing the amount of secondary development

ELF files are organized in the form of sections, which describe information about files such as code, data, symbol tables, relocation tables, global compilation tables, etc.

Executable file is loaded into memory, not the complete mapping into memory, but according to the definition of the ELF file format, section by section loaded into, as a result, the image of the executable file formats and memory is not exactly the same, the file is loaded into memory segment (segment) way of organization, such as: code section, data section, dynamic segment, etc.

ELFFormat file structure and memory structure comparison diagram:

There are three types of files in ELF format:

• Executable file
• Dynamic library file (.soFile)
• Relocation file (.oFile)

All three have an ELF file header that describes basic information about the entire executable, such as the format of the object code, the architecture, the offsets and sizes of various segments or sections, and so on.

Executable files and dynamic libraries have Program Header tables, but relocation files do not.

The ELF file also has a Section Header Table that describes the contents of each Section in the file. The contents of this table and the program header table are somewhat duplicated because the two tables are used for different purposes:

• In the compile and link phase, which isELFFile generation phase, requiredSection Header Table
• Program Header TableIn theELFThe load phase of the file

The purpose of analyzing ELF file format is to understand the loading process of executable files, so we will focus on learning program header table hash

ELFThe file header

The ELF file format in the book and Android 9 location defined in the definition of some different, 9.0 source file is located in the bionic/libc/kernel/uapi/Linux/ELF. H

ELF headers are defined as follows:

typedef struct elf32_hdr {
  unsigned char e_ident[EI_NIDENT];     // Target file identity
  Elf32_Half e_type;                    // Target file type
  Elf32_Half e_machine;                 // The architecture of the target platform
  Elf32_Word e_version;                 // Target file version
  Elf32_Addr e_entry;                   // The entry address of the program
  Elf32_Off e_phoff;                    // The offset of the program header table
  Elf32_Off e_shoff;                    // The offset of the section header table
  Elf32_Word e_flags;                   // File specific, processor-specific flags
  Elf32_Half e_ehsize;                  // The size of ELF header bytes
  Elf32_Half e_phentsize;               // The size of the entry in the program header table in bytes
  Elf32_Half e_phnum;                   // The number of entries in the program header table
  Elf32_Half e_shentsize;               // The size of the entry in the section header table in bytes
  Elf32_Half e_shnum;                   // The number of entries in the section header table
  Elf32_Half e_shstrndx;                // Index the string in the section header table
} Elf32_Ehdr;
Copy the code

In the program header table, the most important is to record the position of the program header table and section header table, indicating the number of entries and the size of the entry field. This can be viewed with the readelf and objdump directives.

Taking readelf-h Linker as an example, let’s look at the header message:

ELF header: Magic: 7F 45 4C 46 01 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 1 (Current) OS/ABI: UNIX-system V ABI version: 0 Type: DYN (Shared target file) System architecture: ARM version: 0x1 Entry point address: 0x1A640 Program header Start: 52 (bytes into file) Start of section headers: 1162348 (bytes into file) flag: 0x5000200, Version5 EABI, soft-float ABI the size of this header: 52 (bytes) 32 (bytes) Number of Program headers: 9 Size of headers: 40 (bytes) Number of headers: 25 String table index header: 22Copy the code

Program Header Table

Program Header Table (Program Header Table) is used to record the address, size and other information of various segments in the file. It is needed when the Program is loaded and linked.

The program header table is an array of structures Elf32_Phdr. Each structure records information about each segment loaded into memory, including type, address, size, and so on.

The structure Elf32_Phdr is defined as follows:

typedef struct elf32_phdr {
  Elf32_Word p_type;        // Segment type
  Elf32_Off p_offset;       // Segment offset in the file
  Elf32_Addr p_vaddr;       // The virtual address of the segment loaded into memory
  Elf32_Addr p_paddr;       // The physical address of the segment loaded into memory
  Elf32_Word p_filesz;      // The size of the segment in the file
  Elf32_Word p_memsz;       // The size of the segment after loading into memory
  Elf32_Word p_flags;       // The name of the section
  Elf32_Word p_align;       // Memory alignment
} Elf32_Phdr;
Copy the code

P_type specifies the segment type.

The name	The numerical	instructions
NULL	0	Indicates that this array entry is not in use
LOAD	1	Indicates that this array entry describes a loadablePeriod of.Period ofBy the size of thep_memszandp_fileszSpecified. aExecutable fileThere can be more than oneLOADPeriod of
DYNAMIC	2	Indicates that this array entry describes dynamic linking information. All areas about dynamic linking are described in this section. The length of the array is not specified. Instead, the last item of the array is NULL to indicate the end of the array.
INTERP	3	Represents information about the dynamic loader that this array item describes, as in AndroidLinker. This type is valid onlyExecutable fileThere can only be one in a file. If the field must exist, it must be locatedLOADBefore the field
NOTE	4	Indicates that this array entry describes the location and size of the additional information
SHLIB	5	Semantics are not specified. Programs that contain this type of segment do not conform to the ABI
PHDR	6	Represents the array entry that describesProgram header tableIts size and location in the file and memory. This type ofPeriod ofThere can only be one in the file. If this type existsPeriod of, must precede all loadable segment items, includingINTERP
TLS	7	Indicates that this array entry describes thread-local storage template information

We use readelf-l Linker to view the header table:

Elf file type DYN (Shared Object File) entry point 0x1a640 Total 9 program headers, starting at offset 52 program headers:  Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align PHDR 0x000034 0x00000034 0x00000034 0x00120 0x00120 R 0x4 LOAD 0x000000 0x00000000 0x00000000 0xbeaf8 0xbeaf8 R E 0x1000 LOAD 0x0bf730 0x000c0730 0x000c0730 0x05e48 0x0f494 RW 0x1000 DYNAMIC 0x0c4b28 0x000c5b28 0x000c5b28 0x000b0 0x000b0 RW 0x4 NOTE 0x000154 0x00000154 0x00000154 0x00020 0x00020 R 0x4 GNU_EH_FRAME 0x0be814 0x000be814 0x000be814 0x002e4 0x002e4 R 0x4 GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x10 EXIDX 0x0a20e8 0x000a20e8 0x000a20e8 0x03b78 0x03b78 R 0x4 GNU_RELRO 0x0bf730 0x000c0730 0x000c0730 0x058D0 0x058D0 RW 0x10 Section to Segment Mapping: Segment mapping... 00 01 .note.gnu.build-id .dynsym .dynstr .gnu.hash .rel.dyn .text .ARM.exidx .rodata .ARM.extab .eh_frame .eh_frame_hdr 02 .data.rel.ro .init_array .dynamic .got .data .bss 03 .dynamic 04 .note.gnu.build-id 05 .eh_frame_hdr 06 07 .ARM.exidx  08 .data.rel.ro .init_array .dynamic .gotCopy the code

Although the program header table contains many segments, only segments of type LOAD are mapped from the file to memory. The rest of the segments will also appear in the LOAD segment if they have actual segments.

Analysis from the printed information above:

• linkertheProgram headerThere are ninePeriod of, which contains twoLOADType, when the file is loaded, actuallymmapThese are the only two that go into memoryPeriod ofThey are what are calledCode segmentandData segment. It is also possible to tell by attribute that one is read-onlyR–Code segmentOne can read and writeRW–Data segment
• Program headerHere are nine of themPeriod ofSeparately containedsection.
  • Code segmentandData segmentRespectively corresponding to the node region01and02Items.
  • The DYNAMIC (03)It contains only one.dynamicSegment, this one.dynamicSection and02In the item.dynamicThe segments are the same. only.dynamicData such as the start position and size of the section is saved inThe DYNAMIC (03)In the. So you have to go throughDYNAMICSection of the find.dynamicSection area.

So, even though the LOAD address space overrides the.dynamic section, you can’t find the.dynamic section through it, you have to go through the Dynamic section. The purpose of this design is that when the system loads the executable file, all it needs to do is map the entire segment into memory, and access each segment is still through the address recorded by the corresponding segment

The.dynamic section generally defines the start address and size of the following sections.

• .pltSection: Contains process linked tables
• .gotSection: contains the global offset table
• rel.pltSection: Relocation table containing function symbols
• rel.dynSection: Relocation table containing non-functional symbols
• .dynsymSection: contains symbol tables
• .dynstrSection: Contains a string table
• .hashSection: Hash table containing symbols

We can see the structure of the.dynamic section using readelf -d libjni_projector. So.

Dynamic section at offset 0x5d60 contains 33 entries: Tag Type Name/Value 0x00000003 (PLTGOT) 0x6eb4 0x00000002 (PLTRELSZ) 640 (bytes) 0x00000017 (JMPREL) 0x1660 0x00000014 (PLTREL) REL 0x00000011 (REL) 0x1330 0x00000012 (RELSZ) 816 (bytes) 0x00000013 (RELENT) 8 (bytes) 0x6ffffffa (RELCOUNT) 68 0x00000006 (SYMTAB) 0x16c 0x0000000b (SYMENT) 16 (bytes) 0x00000005 (STRTAB) 0x74c 0x0000000a (STRSZ) 2489 (bytes) 0x6ffFFEF5 (GNU_HASH) 0x1108 0x00000001 (NEEDED) Shared library: [liblogso] 0x00000001 (NEEDED) Shared library: [libandroid_runtime.so] 0x00000001 (NEEDED) [libnativehelper.so] 0x00000001 (NEEDED) shared library: [libutils.so] 0x00000001 (NEEDED) shared library: [libc++ so] 0x00000001 (NEEDED) shared library: [libnativehelper.so] 0x00000001 (NEEDED) shared library: [libd.so] 0x00000001 (NEEDED) shared Library: [libd.so] 0x0000000e (SONAME) Library SONAME: [libjni_projector.so] 0x0000001a (FINI_ARRAY) 0x6c30 0x0000001c (FINI_ARRAYSZ) 4 (bytes) 0x0000001e (FLAGS) BIND_NOW 0x6FFFFFFB (FLAGS_1) :  NOW 0x6ffffff0 (VERSYM) 0x1208 0x6ffffffc (VERDEF) 0x12c4 0x6ffffffd (VERDEFNUM) 1 0x6ffffffe (VERNEED) 0x12e0 0x6fffffff (VERNEEDNUM) 2 0x00000000 (NULL) 0x0Copy the code

The NEEDED item identifies libraries that need to be loaded dynamically.

We look at thelinkerIn thePeriod ofandsectionThe corresponding relation of:

Function reposition some positions

In Linux, the offset table of global variables is used to solve the problem of external references. The offset table contains the addresses of external references. In this way, only the address of the global table entry needs to be referenced indirectly in the program code.

When Linux needs to relocate a reference symbol, it first loads the library, then looks up the table in the library to find the relative address of the function, and finally adds the relative address of the function to the base address of the library load to get the virtual address of the function.

BionicIn theLinkerThe module

Creation of executable files

Let’s take a quick look at the process of creating an executable file:

• First, the C code (.c) afterThe compilerPreprocessing, compiling into assembly code (.asm)
• Then, processed by the assembler, object code is generated (.o)
• Then, through the linker, the corresponding library is linked to generate an executable (.out)
• The lastOSLoad the executable file into memory for execution.

As shown in figure:

Loading of executable files

There are two different types of executables on Linux:

• One is a statically linked executable: it contains all the functions needed to run, and can be run independently of any external libraries.
• One is dynamically linked executables that do not contain dependent library files and are much smaller in size.

Statically executable programs are used in special situations, such as system initialization, when the entire system is not ready and dynamically linked programs are not available. The system’s startup program Init is a static executable.

In Android, the way to generate a static executable is to add the following configuration to the compiled script:

LOCAL_FORCE_STATIC_EXECUTABLE := true
Copy the code

Linux executes an executable file by:

After the parent forks, execve is executed in the child. This function loads the executable into memory and jumps to the executable entry when the environment is ready. The usual entry point for an executable is the _start_main() function.

Static link

For static links, Android automatically adds two to the application. O crtbegin_static.o crtend_android.o crtbegin_so.o crtend_so.o crtbegin_so. The corresponding source files are located in the bionic/libc/arch-common/bionic directory: crtbegin.c and crtend. The _start_main() function is in crtbegin.c:

__used static void _start_main(void* raw_args) {
  structors_array_t array;
  array.preinit_array = &__PREINIT_ARRAY__;
  array.init_array = &__INIT_ARRAY__;
  array.fini_array = &__FINI_ARRAY__;

  __libc_init(raw_args, NULL, &main, &array);
}
Copy the code

Finally, the __libc_init function is called, where the first argument is raw data passed by the Linux kernel loader, the third argument is a function pointer to main, and the fourth argument is the addresses of several segments. After __libc_init completes initialization of the liBC library, main is called.

I found a good book called Programmer self-cultivation: links, loading and libraries. It is interesting to read this part. Ahem, if you want to review me electronically

Dynamic link

When linking dynamically, the execve system call analyzes the executable’s file header to find the linker. The Linux file is ld.so, while Android is Linker.

Execve loads Linker into the executable’s space, and then executes linker’s _start function. Linker finishes loading the dynamic library and relocating the symbols before running the actual executable code.

Would use _start function in bionic/would/the arch/arm/begin S each architecture implementation directory (has).

#include <private/bionic_asm.h> ENTRY(_start) // Force unwinds to end in this function. .cfi_undefined r14 mov r0, Sp bl __linker_init // Execute the __linker_init function /* linker init returns the _entry address in the main image */ bx r0 // the __linker_init function returns the entry address of the executable program END(_start)Copy the code

This is a bit different from the book, but the flow is the same: _start jumps to __linker_init, __linker_init returns to the executable entry register r0 after initialization, and bx r0 jumps to the application entry function.

Initialization of an executable program

Initialization of an executable program is done through the __linker_init function, implemented in 9.0 in bionic/linker/linker_main.cpp.

__linker_init has a soinfo structure. Soinfo is a very important data structure in Android. This section is defined in bionic/linker/linker_soinfo.h. Android constructs a soinfo structure for both executable files and dynamic libraries, which holds all the section information of an application.

Let’s take a look at the __linker_init source code snippet, heavily censored and annotated:

extern "C" ElfW(Addr) __linker_init(void* raw_args) { soinfo linker_so(nullptr, nullptr, nullptr, 0, 0); Linker_so.base = linker_addr; linker_so.base = linker_addr; linker_so.base = linker_addr; linker_so.size = phdr_table_get_load_size(phdr, elf_hdr->e_phnum); linker_so.load_bias = get_elf_exec_load_bias(elf_hdr); linker_so.dynamic = nullptr; linker_so.phdr = phdr; linker_so.phnum = elf_hdr->e_phnum; linker_so.set_linker_flag(); //2 prelink, this method is executed, soinfo is basically filled in. Exit if (! linker_so.prelink_image()) __linker_cannot_link(args.argv[0]); //3 Load all linker dependent libraries and reposition them, if lost, exit if (! linker_so.link_image(g_empty_list, g_empty_list, nullptr)) __linker_cannot_link(args.argv[0]); //4 Initialize the main thread(including the TLS table).__libc_init_main_thread (args); //5 Initialize linker's static libc library's global variable __libc_init_globals(args); //6 initializes linker's own global variable linker_so.call_constructors(); Solist = get_libdl_info(kLinkerPath, linker_so, linker_link_map); //7 Obtain the soinfo corresponding to libdl and add it to the list. g_default_namespace.add_soinfo(solist); ElfW(Addr) start_address = __linker_init_post_relocation(args); ElfW(Addr) start_address = __linker_init_post_relocation(args) return start_address; }Copy the code

The 9.0 implementation is a bit different from the book, but the overall process is still the same, just more detailed.

Bionic has learned to resist… But hold on a little longer

linkerHow to replacelibdl.so

Linux uses the dlopen function when loading a dynamic library. Libdl. So,dlsys, dlpen, dlopen,dlclose,dlsys, etc. Libdl. So is not implemented directly by Google.

During the loading of executable files, the soinfo structure of all loaded dynamic libraries will be put into a linked list. When a new dynamic library is loaded, it will first check whether it already exists in the linked list. If it does not, the load will continue.

Linker forgeries the soinfo structure of libdel. so and places it at the first element of the linked list, so the linked libdel. so is not actually loaded.

Note that step 7 in the __linker_init source code snippet above does just that. However, the source code in 9.0 and the book has changed a lot, adding namespace logic. For details, see the Introduction to Android Linker

The debugger –ptraceandHook API

Ptrace system calls are typically used in debugger software, where the debugger uses the ptrace function to control the execution of the target process. Some Android security managers use the ptrace function to insert their own dynamic libraries into the system or other processes to monitor the system.

ptraceIntroduction to System Calls

Look at the definition of ptrace in Bionic:

long ptrace(int __request, ...);
// The implementation calls __ptrace
long __ptrace(int req, pid_t pid, void* addr, void* data);
Copy the code

Arguments to __ptrace:

• req: Type of operation requested to be performed
• pid: indicates the ID of the target process
• addr: Address of the target process
• data: Operation-related data varies according to the requested operation. If I write,dataStores the data that needs to be written; If it’s a read operation,dataThe returned data will be stored

Let’s look at the types of req operations:

• PTRACE_TRACEME: indicates that the parent process traces the execution of a child process. Any signals sent to the child will cause it to stop execution, and the parent will be notified when it calls wait(). Later, when the child calls exec(), the core sends it SIGTRAP signals, giving the parent control before the new program starts executing. The PID, ADDR, and data parameters are ignored.

  This is the only request that will be used by the child process, and the rest will be used by the parent process.

• PTRACE_PEEKTEXT: Reads a long integer from the target process’s code snippet. The memory address is determined by addr

• PTRACE_PEEKDATA: Reads a long integer from the data segment of the target process. The memory address is determined by addr.

• PTRACE_PEEKUSR: Reads a long int from the child’s user area addr and returns it as the result of the call.

• PTRACE_POKETEXT,PTRACE_POKEDATA: Copies the long int pointed to by data to the location pointed by addr in the child’s memory space.

• PTRACE_POKEUSR: copies the long int pointed to by data to the location pointed by addr in the child user area.

• PTRACE_GETREGS, PTRACE_GETFPREGS: Copies the values of the child’s general purpose and floating point registers to the location pointed to by data in the parent process. The addr parameter is ignored.

• PTRACE_SETREGS, PTRACE_SETFPREGS: Copies the data pointed to by data from the parent process to the child process’s general purpose and floating point registers. The addr parameter is ignored.

• PTRACE_SETSIGINFO: Copies the data pointed by data in the parent process to the child process as a siginfo_t structure. The addr parameter is ignored.

• PTRACE_SETOPTIONS: Sets the values in the parent process pointed to by data to the ptrace option. Data is interpreted as a bitmask, specified by the flag below

  • PTRACE_O_TRACESYSGOOD: when the forwardsyscallFor traps, bit 7, the highest bit of the first byte, is set in the signal code. Such as:SIGTRAP | 0x80. This helps trackers identify common traps and those made bysyscallCause the trap.
  • PTRACE_O_TRACEFORK: (SIGTRAP | PTRACE_EVENT_FORK << 8) causes the child process to call next timefork(), and automatically tracks new processes that have SIGSTOP signals set at the start of execution. The PID of the new process can passPTRACE_GETEVENTMSGTo obtain.
  • PTRACE_O_TRACEVFORK: (SIGTRAP | PTRACE_EVENT_VFORK << 8) causes the child process to stop its execution the next time it calls vfork() and automatically tracks the new process that has SIGSTOP signal set at the start of execution. The PID of the new process can be obtained from PTRACE_GETEVENTMSG.
  • PTRACE_O_TRACECLONE: (SIGTRAP | PTRACE_EVENT_CLONE << 8) causes the child process to stop its execution the next time it calls Clone (), and automatically tracks the start of executionSIGSTOPSignal new process. The PID of the new process can passPTRACE_GETEVENTMSGTo obtain.
  • PTRACE_O_TRACEEXEC: (IGTRAP | PTRACE_EVENT_EXEC << 8) causes the child process to call next timeexec()To stop its execution.
  • PTRACE_O_TRACEVFORKDONE: (SIGTRAP | PTRACE_EVENT_VFORK_DONE << 8) causes the child process to call next timeexec()And stop its execution when it is finished.
  • PTRACE_O_TRACEEXIT: (SIGTRAP | PTRACE_EVENT_EXIT << 8) causes the child process to stop execution when it exits. The exit status of the child process can passPTRACE_GETEVENTMSG.

• PTRACE_GETEVENTMSG: Retrieves the pTrace event message that has just occurred and stores it in the parent process at the location pointed by data. The addr parameter is ignored.

• PTRACE_CONT: restarts a stopped process. If data points to data other than 0 and is not SIGSTOP, it will be interpreted as a signal passed to the child process. That way, the parent process can control whether a signal is sent to the child process. The addr parameter is ignored.

• PTRACE_SYSCALL, PTRACE_SINGLESTEP: Restart the execution of the child process as PTRACE_CONT, but specify that the child process stops execution at the next entry or exit from the system call, or after executing a single instruction. This can be used for single-step debugging. The addr parameter is ignored.

• PTRACE_SYSEMU, PTRACE_SYSEMU_SINGLESTEP: A program for user mode emulates all system calls of a child process.

• PTRACE_KILL: Sends the SIGKILL signal to the child process to terminate its execution. Data, addr parameters are ignored.

• PTRACE_ATTACH: Attaches to the process specified by the PID, making it a trace target for the current process.

• PTRACE_DETACH: Reverse operation of PTRACE_ATTACH.

Hook APISome of the content

Hook API technology has a long history. In the case that the operating system fails to provide the required functions, using Hook API means to achieve some useful functions is also a last resort.

The earliest Hook API mentioned in the book is to realize the cursor retrieval function of the electronic dictionary on Windows. The string output function of the system is replaced by the function in the electronic dictionary, so that the string at any position on the screen can be obtained. Much ~ ~ ~ ~

Linux usually uses the ptrace function to achieve the purpose of Hook API due to its high security. However, you need root permission to call ptrace.

The principle of Hook API is to use the ptrace function to inject a small piece of code into the target program. The task of this small piece of code is to load the dynamic library developed by ourselves into the target process, and then find the position of the specific function in the target process in the global offset table, and replace it with the function address of the dynamic library.

As Android security has improved, this part of the implementation has become more and more difficult. But sabotage is human nature and worth studying. Article: Android Native Hook for everyone to taste

Open source licenses

The open Source license defines your rights and responsibilities when using open source software. It defines what you can and cannot do with open source software.

Although the open source agreement does not necessarily have the legal effect, but when it comes to software copyright disputes, the open source agreement is also one of the very important evidence.

Let’s briefly introduce some of the more common ones

GNU GPL (General Public License)

The amount of open source software that follows the GPL is enormous, and most open source software, including the Linux system, is based on it.

As long as the software contains gPL-compliant products or code, the software must also comply with the GPL license, which means that it must be free and open source, and cannot be charged from closed source. Therefore, this license is not suitable for commercial software.

Berkeley Software Distribution (BSD)

The GPL’s intent is to make code open source/free to use and reference/modify/derive code open source/free to use and does not permit the distribution and sale of modified and derived code as closed source commercial software.

That’s why Linux is available for free, including Linux for commercial companies and a wide variety of free software on Linux developed by individuals, organizations, and commercial software companies.

The main content of the GPL is that whenever a product under the GPL is used in a piece of software (” use “means a library reference, modified code, or derived code), that product must also be under the GPL, and must be open source and free. This is called contagiousness.

Gpl-licensed products have no problems being used as a separate product and enjoy the advantage of being free.

Because the GPL strictly requires software products using the GPL class library to use the GPL, commercial software or departments that have confidentiality requirements for code using the GPL open source code are not suitable for integration/adoption as a basis for class libraries and secondary development.

Apache License Version

Apache Licence is a protocol adopted by Apache, a well-known non-profit open source organization. This protocol is similar to BSD in that it encourages code sharing and respect for the copyright of the original author, and also allows code to be modified and redistributed (as open source or commercial software).

The conditions to be met are similar to those for BSD:

• You need to give the user of your code an Apache Licence
• If you change the code, you need to explain it in the modified file.
• In extended code (modified and derived from source code) it is necessary to carry the agreements, trademarks, patent claims, and other instructions specified by the original author in the original code.
• If the product to be released contains a Notice file, the Apache Licence must be included in the Notice file. You may add your own license to the Notice, but this should not be construed as a change to the Apache Licence.

Apache Licence is also a business-friendly license. Users can also modify the code as needed and distribute/sell it as an open source or commercial product.

Open Source Protocol Overview diagram

From Baidu Encyclopedia

conclusion

That’s the end of the Bionic chapter. It’s really hard to learn. Recently I was on a business trip and I was really unfamiliar with the knowledge of this chapter, so THE completion time was delayed, but the profit was great, ha ha ha!

Binder for Android is the next chapter I’ve been dreaming of.