Memory management is the core task of operating system. It is also critical for programmers and system administrators. In the next few articles, I’ll look at memory management from a practical standpoint and dive into its internals. While these concepts are general, the examples are mostly from Linux and Windows with the 32-bit x86 architecture. This first article described how programs are distributed in memory.
Each process in a multitasking operating system runs in its own memory “sandbox.” The sandbox is a virtual address space that has a total of 4GB of memory address blocks in 32-bit mode. These virtual addresses are mapped to physical addresses through the kernel page Table, and these virtual addresses are maintained by the operating system kernel and thus consumed by processes. Each process has its own set of page tables, but this is a bit tricky. Once virtual addresses are enabled, they are applied to all software on that computer, including the kernel itself. Therefore, part of the virtual address space must be reserved for use by the kernel:
However, this is not to say that the kernel uses a lot of physical memory; on the contrary, it uses very little of the available address space to map to the physical memory it needs. Kernel space is marked in the kernel page table as being used exclusively by privileged code (Ring 2 or lower), so if a user-mode program tries to access it, a page failure error will be triggered. In Linux, kernel space is always present and is mapped to the same physical memory in all processes. Kernel code and data are always addressable, ready to handle interrupts or system calls. In user mode, by contrast, the address space changes with each process switch:
The blue areas represent the virtual address space mapped to physical addresses, and the white areas are the unmapped parts. In the example above, Firefox, known as “gluttonous,” uses a lot of virtual memory space. Different strips in the address space correspond to different memory segments, such as heap, stack, and so on. Note that these segments are simply a simplified representation of a series of memory addresses and have nothing to do with Intel-type segments. However, this is a standard segment layout in Linux processes:
When computers were happy and safe, the starting virtual addresses of those segments were exactly the same on almost every process in the machine. This will make it easy to dig security holes remotely. Exploits often require references to absolute memory locations: an address in the stack, the address of a library function, and so on. Remote attacks can select this address with their eyes closed because the address space is the same. When attackers do this, people get hurt. As a result, address space randomization became popular. Linux randomizes the stack, memory-mapped segment, and heap by adding an offset to its starting address. Unfortunately, the 32-bit address space is very crowded, leaving not much room for address space randomization and thus hampering the effect of address space randomization.
The highest segment in the process address space is the stack, which stores local variables and function parameters in most programming languages. Calling a method or function pushes a new stack frame onto the stack. The stack frame is removed when the function returns. This simple design is probably due to the fact that the data follows a strict LIFO order, which means that no complex data structure is required to track the contents of the stack — a simple pointer to the top of the stack can do it. Push-in and eject are therefore very fast and accurate. It could also be that continuous stack area reuse tends to keep active stack memory in the CPU cache, which speeds up access. Each thread in the process has its own stack.
Pushing more data to the stack than is just right can exhaust the map area of the stack. This triggers a page fault, which in Linux is handled by expand_stack(), which calls acct_stack_growth() to check that the stack is growing properly. If the stack size is lower than the RLIMIT_STACK value (typically 8MB size), then this is a normal stack growth and fair use of the program, otherwise unknown problems may occur. This is a common mechanism for adjusting the stack size on demand. However, if the stack size reaches the above limit, a stack overflow will occur and the program will receive a segment failure Segmentation Fault error. When the stack area of the map is expanded to meet the need, the map area does not shrink when the stack shrinks. Like the FEDERAL budget in the United States, it will only expand.
Dynamic stack growth is the only exception when it is allowed to access an unmapped area of memory, as shown in white. Any other access to an unmapped memory region will trigger a page failure, causing a segment failure. Some mapping areas are read-only, so attempting to write to these areas will also trigger a segment failure.
Underneath the stack, there are memory-mapped segments. Here, the kernel maps the file contents directly into memory. Any application can request a mapping via the Linux mmap() system call (code implementation) or Windows CreateFileMapping().aspx /MapViewOfFile().aspx). Memory mapping is a convenient and efficient way to implement file I/O. Therefore, it is often used to load dynamic libraries. Sometimes, it is also used to create an anonymous memory map that does not match any files. This map is often used as a substitute for program data. In Linux, if you request a large block of memory via malloc(), the C library will create such an anonymous map instead of using heap memory. By “large” we mean the number of bytes exceeding the MMAP_THRESHOLD setting, which defaults to 128 kB and can be adjusted by mallopt().
Next up is the “heap”, and in our next address space, the heap provides runtime memory allocation, like a stack, but unlike a stack, it allocates data with a longer lifetime than the function that allocates it. Most programming languages provide heap management support for programs. Therefore, meeting memory requirements is something that both the programming language runtime and the kernel do. In C, the heap allocation interface is the Malloc () family, whereas in programming languages that support garbage collection, such as C#, this interface uses the new keyword.
If there is enough space in the heap to satisfy a memory request, it can be handled by the programming language runtime without kernel involvement. Otherwise, the heap is enlarged to meet the size required by the memory request through the BRK () system call (code implementation). Heap management is complex, and in the face of our program’s chaotic allocation pattern, it strives to strike a balance between speed and memory efficiency through complex algorithms. The time required to service a heap request can be significant. Real-time systems have a special-purpose allocator to deal with this problem. The heap is also fragmented, as shown in the following figure:
Finally, we reach the lower parts of memory: BSS, data, and program text. In C, the contents of static (global) variables are stored in BSS and data. The difference between them is that BSS holds the contents of an uninitialized static variable whose value is not set by the programmer in the source code. The BSS memory area is _ anonymous _ : it is not mapped to any files. If you use static int cntActiveUsers, the CONTENT of cntActiveUsers is stored in the BSS.
Data segments, in turn, are used to store the contents of static variables initialized in source code. This memory region is non-anonymous. It maps a portion of the program’s binary value mirror and contains the static variable content given the initialized value in the source code. So, if you say static int cntWorkerBees = 10, the contents of cntWorkerBees are stored in the data segment with an initial value of 10. Although it is possible to map to a file through data segments, this is a private memory mapping, meaning that if memory changes, it does not reflect the change to the underlying file. This has to be the case, otherwise the assigned global variable will change the binary image on your disk, which would be incredible!
It is difficult to graph a data segment because it uses a pointer. In that case, the gonzo’s _ content _(a 4-byte memory address) is stored on the data segment. However, it does not point to a real string. And that string exists in a text segment, which is read-only, and it’s used to hold things like string constants in your code. Text segments also map your binaries in memory, but if your program writes to this segment, it will trigger a segment failure error. Although it is not as effective in C as avoiding such pointer errors in the first place, this mechanism can also help avoid pointer errors. Here is a diagram showing these segments and sample variables:
You can check memory areas in Linux processes by reading the /proc/pid_of_process/maps file. Remember that a segment can contain many regions. For example, each memory-mapped file is typically in its own region in the MMAP segment, while dynamic libraries have additional regions like BSS and data. In the next article, we will elaborate on what “area” really means. In addition, sometimes people say “data segment” means “data + BSS + heap”.
You can use the nm and objdump commands to examine binary images and display their symbols, addresses, segments, etc. Finally, in Linux, the virtual address layout described above is an “elastic” layout, which has been the default for years. It assumes that RLIMIT_STACK has a value. If there is no value, Linux reverts to the “classic” layout shown below:
This is the virtual address space layout. Subsequent articles discuss how the kernel keeps track of these memory areas, memory mapping, how files are read and written, and the implications of memory usage data.