Go language memory alignment details

You have to work really hard to look effortless!

Wechat search public number [long Coding road], together From Zero To Hero!

preface

Even though unsafe, what’s the difference between a man who doesn’t delve into the broadening package and a man who plays with women’s emotions? Although I can’t, I can learn 🐶, so in this article, let’s take a look at what memory alignment is!

Note: the test examples in this article are based on Go1.17 64-bit machines

Basic knowledge of

In the Go language, unsafe.sizeof (x) is used to determine the number of bytes of memory a variable takes up (not including the Sizeof what x refers to).

For example, for string arrays, on 64-bit machines, safe.sizeof () returns an arbitrary string array of 24 bytes, independent of its underlying data:

func main(a) {
	s := []string{"1"."2"."3"}
	s2 := []string{"1"}
	fmt.Println(unsafe.Sizeof(s))  / / 24
	fmt.Println(unsafe.Sizeof(s2)) / / 24
}
Copy the code

For built-in types of the Go language, the memory footprint is as follows:

func main(a) {
	fmt.Println(unsafe.Sizeof(int(1)))                  / / 8
	fmt.Println(unsafe.Sizeof(uintptr(1)))		    / / 8
	fmt.Println(unsafe.Sizeof(map[string]string{}))     / / 8
	fmt.Println(unsafe.Sizeof(string("")))		    / / 16
	fmt.Println(unsafe.Sizeof([]string{}))		    / / 24

	var a interface{}
	fmt.Println(unsafe.Sizeof(a))                       / / 16
}
Copy the code

See a problem

For a structure, the memory footprint should be equal to the sum of the memory footprint of the underlying types. Let’s look at some examples:

type Example struct {
	a bool // 1 byte
	b int	 // 8 bytes
	c string // 16 bytes
}

func main(a) {
	fmt.Println(unsafe.Sizeof(Example{})) / / 32
}
Copy the code

The three basic types of the Example structure add up to 25 bytes, but the final output is 32 bytes.

Let’s look at two structs, even though they contain fields of the same type, but not in the same order, and the final output sizes are different:

type A struct {
	a int32
	b int64
	c int32
}

type B struct {
	a int32
	b int32
	c int64
}

func main(a) {
	fmt.Println(unsafe.Sizeof(A{})) / / 24
	fmt.Println(unsafe.Sizeof(B{})) / / 16
}
Copy the code

What’s causing this problem, which brings us to memory alignment.

What is memory alignment

We know that accessing a variable in a computer requires accessing its memory address. Theoretically, it seems that access to any type of variable can start at any address, but the reality is this: When accessing a variable of a particular type, it is usually accessed at a specific memory address, which requires that there is a limit on the location of the data in memory. Various types of data are arranged in space according to certain rules, instead of being sequenced one after another, which is called alignment.

Memory alignment is the domain of the compiler. The compiler places each “data unit” in its proper place in the program.

Why memory alignment

1. Some cpus can access any data at any address, while others can only access data at a specific address. Therefore, the code is not portable depending on the hardware platform. If the allocated memory is aligned at compile time, it is platform portable.

2. The CPU does not access the memory byte by byte, but by word size. For example, the 32-bit CPU word length is 4 bytes, and the 64-bit CPU word length is 8 bytes. If the address of a variable is not aligned, it may take multiple accesses to read the contents of the variable, but after alignment, it may require only one memory access. Therefore, memory alignment can reduce the number of CPU accesses to memory and increase the throughput of CPU accesses to memory.

Assuming that the step size of each access is 4 bytes, if the data of B is not aligned with memory, two memory accesses are required to obtain the data of B, and the complete data of B is obtained after data collation:

If it is aligned, one memory access will get the full data of B, reducing one memory access:

unsafe.AlignOf()

Unsafe.alignof (x) returns m. When variables are memory-aligned, you need to ensure that the memory address allocated to X is divisible by m. So we can use this method to determine the address of the variable x in memory alignment:

• For any type of variable x,unsafe.Alignof(x)At least one.
• For struct struct type variable x, evaluate the value of f for each field of xunsafe.Alignof(x.f).unsafe.Alignof(x)Is equal to the maximum of that.
• For array array variable x,unsafe.Alignof(x)Equal to the alignment multiple of the element types that make up the array.

For the system’s native base type variable x, unsafe.alignof (x) returns min(word length /8, unsafe.sizeof (x)), which is the smaller value for the computer word length and type:

func main(a) {
  fmt.Println(unsafe.Alignof(int(1))) / / 1 - min (8, 1)
  fmt.Println(unsafe.Alignof(int32(1))) // 4 -- min (8,4)
	fmt.Println(unsafe.Alignof(int64(1))) // 8 -- min (8,8)
  fmt.Println(unsafe.Alignof(complex128(1))) / / 8 - min (8 dec)
}  
Copy the code

Memory alignment rule

When we talk about memory alignment, we put variables at a particular address, so how do we calculate a particular address? This involves memory alignment rules:

• Member alignment rule

For an underlying type variable, if unsafe.alignof () returns a value of m, the address of the variable needs to be divisible by m (if the current address is not divisible, fill in the blank bytes until it is).

• Global alignment rule

For a structure, if unsafe.alignof () returns m, ensure that the structure’s overall memory footprint is an integer multiple of m. If the structure is not currently an integer multiple, fill in blank bytes after it.

With memory alignment, it is possible to ensure that when accessing a variable address:

1. If the variable occupies less memory than the word length, the data is guaranteed in one access;
2. If the variable occupies more memory than the word length: The first memory access address is guaranteed to be the first address of the variable.

For example

Case 1:

type A struct {
	a int32
	b int64
	c int32
}

func main(a) {
	fmt.Println(unsafe.Sizeof(A{1.1.1}))  / / 24
}
Copy the code

1. Unsafe.sizeof (int32(1))=4, the memory usage is 4 bytes, and unsafe.alignof (int32(1))=4. 0 is divisible by 4:

2. Sizeof(int64(1)) = 8, the memory size is 8 bytes, and unsafe.alignof (int64(1)) = 8, the first address is divisible by 8, and the current address is 4. The address closest to 4 that is divisible by 8 is 8, so four blank bytes need to be added, starting with 8:

3. Sizeof(int32(1))=4, the memory usage is 4 bytes, and unsafe.alignof (int32(1))=4, the memory alignment must ensure that the first address of the variable is divisible by 4. The current address is 16, and 16 is divisible by 4:

4. Unsafe.alignof (A{}) = 8, the maximum value of the three variable members. Unsafe.alignof (A{}) = 8, the memory alignment needs to be an integer multiple of 8.

5. The resulting memory footprint of this structure is 24 bytes.

Example 2:

type B struct {
	a int32
	b int32
	c int64
}

func main(a) {
	fmt.Println(unsafe.Sizeof(B{1.1.1}))  / / 16
}
Copy the code

1. Unsafe.sizeof (int32(1))=4, the memory usage is 4 bytes, and unsafe.alignof (int32(1))=4. 0 is divisible by 4:

2. Sizeof(int32(1))=4, the memory usage is 4 bytes, also unsafe.alignof (int32(1))=4, the memory alignment must ensure that the first address of the variable is divisible by 4, and the current address is 4, and 4 is divisible by 4:

3. Unsafe.sizeof (int64(1))=8, 8 bytes, and unsafe.alignof (int64(1))=8. The alignment must ensure that the first address of the variable is divisible by 8, and the current address is 8, 8 is divisible by 8:

4. Now that all member alignment is complete, we need to look at the overall alignment rule: Unsafe.alignof (B{}) = 8, the maximum value of the three variable members, and memory alignment is required to ensure that the memory usage of the structure is an integer multiple of 8. The current memory usage is 16 bytes, which is in line with the rules. The final memory usage of the structure is 16 bytes.

Alignment rules for empty structures

If the empty structure is used as the built-in field of the structure: when the variable is in the front and middle of the structure, no memory is occupied; When the variable is at the end of the structure, memory alignment is required so that the size of the previous variable is the same.

type C struct {
	a struct{}
	b int64
	c int64
}

type D struct {
	a int64
	b struct{}
	c int64
}

type E struct {
	a int64
	b int64
	c struct{}}type F struct {
	a int32
	b int32
	c struct{}}func main(a) {
	fmt.Println(unsafe.Sizeof(C{})) / / 16
	fmt.Println(unsafe.Sizeof(D{})) / / 16
	fmt.Println(unsafe.Sizeof(E{})) / / 24
  fmt.Println(unsafe.Sizeof(F{})) / / 12
}
Copy the code

conclusion

In this article we learned about memory alignment in Go. The main contents are as follows:

• Unsafe.sizeof (x) returns the Sizeof x’s memory footprint
• Two structures, even though they contain the same number of variable types, occupy different amounts of memory in different locations, which leads to memory alignment
• Memory alignment includes both member alignment and global alignment, related to unsafe.alignof (x)
• When an empty structure is a member variable, it takes up memory depending on its location
• In real development, we can optimize the memory footprint by adjusting the location of variables (usually in order of the size of the variables, the overall memory footprint is smaller).

More and more

Personal blog: lifelmy.github. IO /

Wechat official account: Long Coding road