Author: Doug, 10+ year veteran of embedded development.
Public number: [IOT town], focusing on: C/C++, Linux operating system, application design, Internet of Things, SCM and embedded development and other fields. Public account reply [books], get Linux, embedded field classic books.
Transfer: welcome to reprint the article, reprint need to indicate the source.
[TOC]
Food is eaten mouthful by mouthful, and computers evolve step by step, such as this Intel CPU model history:
In order to take advantage of increasingly powerful computers, operating systems are becoming increasingly bloated and complex.
In order to learn the basic principles of an operating system from the bottom up, we have to throw away the cloak of an operating system and start with the most primitive hardware and programming methods to understand some of the fundamental knowledge.
In this article, we will continue to explore how the 8086, the groundbreaking processor, uses the segment mechanism to address memory.
What is a code snippet?
Previous article: Linux From Scratch 01: How does a CPU execute an instruction? It has been mentioned that in the processor, when executing each instruction code, the CPU mechanically and simply calculates the converted physical address from the two registers CS:IP, and reads instructions of a certain length from the memory address pointed to by this physical address. The Arithmetic Logic Unit (ALU) is then given to execute.
The physical address can be calculated using CS x 16 + IP.
When a CPU reads an instruction, it automatically knows how many bytes it needs to read from the opcode.
After an instruction is read, the contents of the IP register increment to the address of the next instruction in memory.
For example, at the beginning of memory 20000H, there are two instructions:
mov ax, 1122H
mov bx, 3344H
Copy the codeWhen executing the first instruction, CS = 2000H, IP = 0000H, after the address translation of the physical address is: 2000H * 16 + 0000 = 20000H(multiplied by 16 also means the hexadecimal number moved 1 bit to the left) :
When the first instruction code B8, 22, 11 is read, the contents of the IP register are automatically increased by 3 ‘to point to the next instruction:
After the second instruction BB 44 33 is read, the contents of the IP register are increased by 3 to 0006H.
As I wrote in the previous article, the CPU simply reads, executes, reads, and executes instructions repeatedly from the CS:IP address in memory.
It can be seen that, in order to complete a meaningful work, all the instruction codes must be concentrated together, unified in memory in a certain address space, can be read and executed by CPU.
The address space in memory is called a segment, and because the segment contains the instructions compiled by the code, it is also called a code segment.
Thus, the meaning of the two registers CS and IP used to address the code segment is clear:
CS: segment register, in which the value is shifted one bit to the left and the resulting value represents the beginning address, or base address, of the code segment in memory;
IP: Instruction pointer register, which represents the address of an instruction and the offset from the base address. That is, the IP register is used to help the CPU remember which instruction has been processed and which instruction is to be processed next.
What is a data segment?
To be a meaningful program, it is not enough to just have instructions; it must also manipulate data.
The data should also be grouped together in an address space in memory, which is also a segment called a data segment.
In other words, a code segment and a data segment are two address Spaces in memory that store instructions and data, respectively.
Can imagine: if the instructions and data are not stored separately, but mixed together, then the CPU in reading an instruction, will certainly put the data as instructions to read, execute, just like the following, no error is strange!
The CPU accesses a data segment in memory in a similar way to accessing a code segment, which is a base address plus an offset to get a physical address in the data segment.
In 8086 processing, the segment register of the data segment is DS, that is, when the CPU executes an instruction that needs to access the data segment, it will shift the value of the data segment register one bit to the left and get the address as the base address of the data segment.
Unfortunately, there is no other REGISTER in the CPU, like the IP register, to represent the offset address register of the data segment.
This is not a bad thing, because the developer of the program knows best what it needs to do with the data as it processes it, so we can have more flexibility in telling the CPU how to calculate the offset address of the data.
Just like a monkey breaking off corn, you don’t have to do it in any order, you can break off whatever you want. Similarly, when the program manipulates data, it simply gives the value of the offset address of the data, no matter which data it manipulates.
Type and length of data
However, there are important concepts to keep in mind when manipulating each piece of data in a data segment: what type of data is it, and how many bytes it occupies in memory.
When we define a variable in a high-level language (eg: C), we must specify the type of the variable. Once the type is determined, the amount of space it takes up after being loaded into memory is determined.
Like this one:
Assuming 30000H is the base address of the data segment (which means 3000H in the DS register), what is the size of the data at 30000H: 11H? 2211 h? Or 44332211 h?
All of these are possible because there is no definite type of data!
We know that in C, if we have a pointer PTR that ends up pointing to the 30000H physical address here (the PTR in C code is the virtual address, and the 30000H physical address is executed after address translation).
If PTR is defined as:
char *ptr;
Copy the codeSo we can say that the PTR pointer points to a value of 11H.
If PTR is defined as:
int *ptrt;
Copy the codeWe can say that the PTR pointer points to a value of 44332211H(assuming the little endian format).
That is, the data that the pointer PTR points to depends on the type at which the pointer variable is defined.
This is the case in high-level languages, but what about assembly languages?
PS: As I said before, the main purpose of this article is to learn about the Linux operating system, but in order to learn something relatively low-level, you have to throw off the operating system at the beginning and go to the closest place to the hardware.
But how to look at it? Still have to rely on some primitive means and tools, then assembly code is undoubtedly the best, but also the only means;
However, the assembly code involved is the simplest, just to illustrate the principle;
In assembly language, the CPU uses the relevant registers in the instruction code to determine the length of operation data.
As mentioned in the previous article, the CPU is slow to manipulate memory relative to registers.
Therefore, when THE CPU processes the data in the data segment, it usually reads the original data into the general register (such as AX, Bx, CX dx) first, and then calculates it.
Once you have the result, write it back to the data segment in memory (if necessary).
So when the CPU reads and writes data, it decides the length of read and write data according to the register used in the instruction code. Such as:
mov ax, [0]
Copy the codeWhere [0] represents the location in the memory data segment where the offset address is 0.
When the CPU executes this instruction, it will go to the physical address of 30000H(assuming that the value of the data segment register DS is 3000H at this time) and take out 2 bytes of data and put them into the general register AX. At this time, the value in ax register is 2211H.
Why take out 2 bytes? Because the length of the AX register is 16 bits, which is 2 bytes.
So what if I just want to take 1 byte?
The 16-bit general purpose register AX can be split into two 8-bit registers: AH and AL.
mov al, [0]
Copy the codeSince the AL register in the instruction code is 8 bits, the CPU reads only one byte 11 at 3000h and places it in the AL register. (The high 8 bits of the AX register, or ah, remain the same.)
What if I want to take 3 bytes or 4 bytes?
As a fairly old processor, the 8086 CPU is 16-bit and can only manipulate 8-bit or 16-bit data.
Addressing range
It can be concluded from the above contents:
Both code segment and data segment are addressed by [base address + offset address].
The base address is placed in the respective segment register, and the CPU will automatically move the value of the segment register 1 bit to the left as the base address of the segment.
The offset address determines the specific address of each segment. The maximum offset address is 16 bits, or 64KB of space.
Note: the segment register is shifted 1 bit to the left in hexadecimal, which is equivalent to multiplying by 16, so the base address of the segment is all multiples of 16.
Let’s see what 64 KB of space has to do with 20 address lines.
The 8086 processor has 20 address lines, representing a total of 1MB of memory. Given more space, it is not blessed because it cannot address more than 1MB of address space.
This 1MB of memory space can be divided into many segments.
For example, the range of addresses in the first paragraph is:
Let’s figure out the space of the last segment.
FFFF0 + FFFF = 10FFEF =1M + 64K-16bytes
The size of the address space exceeds 1 MB, but after all, there are only 20 address lines, it is impossible to address more than 1 MB of address space, so the system will take a winding way to locate an address space, similar to the mathematics of the module operation.
In addition, when representing a memory address, the value of the physical address is not given directly (e.g. 3000A), but is represented by the segment address: offset address (e.g. 3000:000A).
The stack
A stack is a kind of data space, but it operates in a special way.
The stack operates in four words: last in, first out.
When introducing the data segment above, we manually set the offset address of the data in the instruction code, because the location of the data, what it means and how to use it are the most clear in the developer’s mind.
But the stack is different. Although it is also used to store data, it is operated by special instructions provided by the processor: push and POP.
Push: to put a piece of data into the stack space; Pop: to pop a piece of data from the stack space;
Note: the data here is fixed at 2 bytes, i.e., a word.
Anyone who has written a C/C++ program knows that there is a push operation when a function is called; When the function returns, there is an off-stack operation.
Since a stack is also a block of memory, it is represented as a segment in memory.
Since it is a segment, there must be a segment register that represents its base address, and the segment register of this stack is SS.
In addition, because the stack is loaded and unloaded in sequential address order, the processor also provides an offset address register for the stack: SP(called: top pointer), which points to the position of the uppermost element in the stack space.
Here’s an example:
The base address of the stack space is 1000:0000, and the address space of SS:SP is the top of the stack, where the element at the top of the stack is 44.
When the following two instructions are executed:
mov ax, 1234H
push as
Copy the codeThe value in SP at the top of the stack first decreases by 2 to 000A:
Then, the value 1234H in register AX is placed at the memory location pointed to by SS:SP:
The order of operations on the stack is reversed:
pop bx
Copy the codeFirst, put the data 1234H in the memory unit pointed by SS:SP into register bX, and then add the value in the top pointer register SP by 2 to become 000C:
This describes the execution of stack operations in an 8086 processor.
If you read other stack descriptions, you can see that the “full decrement” stack operation is used here, as well as: full increment, empty decrement, and empty increment.
Full: refers to the space to which the pointer points at the top of the stack, is a valid data. When a new data is pushed onto the stack, the top pointer points to the next empty location and then puts the data into that location.
Null: indicates that the space to which the pointer points at the top of the stack is an invalid data. When a new data is pushed onto the stack, the data is placed in this position first, and the top pointer points to the next empty position.
Incremental: indicates that when data is pushed onto the stack, the pointer at the top of the stack increases to the higher address.
Decrement: indicates that when data is pushed onto the stack, the pointer on the top of the stack decrement to the lower address.
Real mode and protected mode
From the above way of addressing memory, we can write programs that can manipulate data anywhere in memory as long as it is addressable.
Such addressing is called real mode. Real, is real, practical meaning, concise, direct, without what winding.
Since people write code, you’re bound to make some stupid little mistakes. Or someone malicious, deliberately manipulating code or data in memory space that should not or should not be manipulated.
In order to protect the memory effectively, from 80386, introduced the protection mode to address the memory.
Some books mention the concept of IA-32A, which is short for Intel Architecture 32-bit and was first adopted in the 386.
Although the protected mode is introduced, there is also a real mode, namely forward compatibility. The computer starts up in real mode, and the BIOS goes into protected mode after loading the master boot record and setting some registers.
Under the protection mode introduced after 386, the address line becomes 32, and the maximum addressing space can reach 4GB.
Of course, the registers in the processor became 32 bits.
We still use the segment base address + offset to calculate a physical address. Assuming that the contents of the segment register are 0 and the maximum length of the offset address is 32 bits, the maximum space that a segment can represent is 4GB.
This is also why the maximum addressable space per process in modern processors today is 4GB(generally a virtual address).
In a word: The fundamental difference between real mode and protected mode is whether or not memory is protected.
Segmentation policies in Linux
The segmentation mechanism described above is just one of the memory addressing mechanisms provided in x86 processors.
On top of x86 processors, Windows and Linux are running to get other operating systems.
We developers program in the face of the operating system and write programs that are taken over by the operating system, not directly by the x86 processor.
The operating system creates a layer of isolation between the application and the x86 processor:
Therefore, how to take advantage of the segmentation mechanism provided by x86 is a problem that the operating system needs to worry about.
What policies the operating system provides for applications to use are another matter.
So how does the Linux operating system wrap and use the segment addressing provided by x86?
Remember this picture from the last post:
These are the four main segment descriptors in Linux2.6. Regardless of what segment descriptors are, they are ultimately used to describe a space in memory.
In modern operating systems, segmentation and paging are both ways of dividing and managing memory and are somewhat redundant in function.
Linux uses segmentation in a very limited way, preferring paging.
The figure above defines four segments, each with a base address of 0x00000000 and a Limit of 0xFFFFF.
From the value of Limit: the maximum is 2 to the 20th, and only 1 MB of space.
However, the G field indicates the granularity of the segment, and 1 means that the granularity is 4K, so 1 MB * 4K = 4 GB, that is, the maximum space of the segment is 4 GB.
All four segments have the same base address and address range! The main difference is the Type and DPL fields.
DPL indicates the priority. The priority value of the two user segments (code segment and data segment) is 3, and the priority value is the lowest (the higher the value is, the lower the priority is). The two kernel segments (code segment and data segment) have priority values of 0, with the highest priority.
This leads to an important conclusion in Linux systems: logical addresses and linear addresses are numerically equal because the base address is 0x00000000.
We’ll talk about memory segmentation and paging addressing in Linux in more detail later.
Recommended reading
[1] C language pointer – from the underlying principle to the tricks, with graphics and code to help you explain thoroughly [2] step by step analysis – how to use C to achieve object-oriented programming [3] The original GDB underlying debugging principle is so simple [4] inline assembly is terrible? Finish this article and end it!
Other series albums: selected articles, C language, Linux operating system, application design, Internet of Things
