Hello, everybody, today I want to talk to you in this article about the Go language pointer this topic, compared with C, Go language in the design for the use of security Pointers in the type and operation of the restrictions, which allows Go programmers to enjoy the convenience of Pointers, and avoid the danger of Pointers. In addition to the normal Pointer, the Go language also provides a generic Pointer in the unsafe package via unsafe.Pointer. This generic Pointer, along with several other features of the unsafe package, allows users to bypass the Go language’s type system and manipulate memory directly. For example: Pointer type conversion, read and write structure private members do so. The Go language’s designers decided to put these features in the Unsafe package because of its powerful functions and the dangerous consequences of reading or writing the wrong memory address. Unsafe packages are often used even in low-level source code, where unsafe packages are often sought after. This is an essential skill for anyone who wants to become an advanced Gopher. Looking beyond the unsafe package, we’ll take a look at the basics of pointer usage and constraints.

Basic knowledge of

A pointer holds the memory address of a value. Type *T represents a pointer to a value of type T. Its zero value is nil.

The & operator generates a pointer to its operand.

i := 42
p = &i
Copy the code

The * operator retrieves the value of the pointer to the address, also known as dereference.

fmt.Println(*p) // Read the stored value through the pointer p
*p = 21         // Set the value of the memory address stored by the pointer p
Copy the code

Why do we need pointer types? Consider this example from go101:

package main

import "fmt"

func double(x int) { 
   x += x
}

func main() { 
   var a = 3 
   double(a)
    fmt.Println(a) // 3
}
Copy the code

Double a in double, but the function in this example can’t do that. This is because function arguments in Go are passed by value. X in double is just a copy of argument A. Operations on x inside the function cannot be fed back to argument A.

Replacing the argument with a pointer solves this problem.

package main
import "fmt"

func double(x *int) { 
   *x += *x
    x = nil
}

func main() {
    var a = 3
    double(&a)
    fmt.Println(a) // 6
    p := &a
    double(p)
    fmt.Println(a, p == nil) // 12 false
}
Copy the code

At first glance you might be confused by the following line of code

x = nil
Copy the code

If you think a little bit about the Go language above where arguments are passed by value, you’ll see that this line of code doesn’t affect the outside variable A at all. Since arguments are passed by value, x in the function is just a copy of &a.

*x += *x
Copy the code

This sentence doubles the value x points to (that is, &a points to the variable a). But the operation on x itself (a pointer) does not affect the outer a, so x=nil inside double does not affect the outer a.

Pointer constraint

Compared with the flexibility of Pointers in C, Pointers in Go have many restrictions, but this allows us to enjoy the convenience brought by Pointers while avoiding the danger of Pointers. Let’s briefly discuss some of Go’s limitations on pointer operations

Limitation 1: Pointers cannot participate in operations

Here’s a simple example:

package main

import "fmt"

func main(a) {
	a := 5
	p := a
	fmt.Println(p)
	p = &a + 3
}
Copy the code

The above code will not compile and will report a compilation error:

invalid operation: &a + 3 (mismatched types *int and int)
Copy the code

That is, Go does not allow mathematical operations on Pointers.

Limitation 2: Pointers of different types cannot be converted to each other.

The following program also failed to compile:

package main

func main(a) {
	var a int = 100
	var f *float64
	f = *float64(&a)
}
Copy the code

Limitation 3: Pointers of different types cannot be compared and assigned to each other

This restriction is the same as restriction 2 above, because Pointers cannot be converted between types, so we cannot use == or! Similarly, pointer variables of different types cannot assign to each other. For example, the following error is generated.

package main

func main(a) {
	var a int = 100
	var f *float64
	f = &a
}
Copy the code

Pointers to the Go language are type-safe, but they have many limitations, so Go also provides generic Pointers for type conversions, which is the unsafe.pointer provided by the unsafe package. In some cases, it makes code more efficient and, of course, more dangerous.

The unsafe packages

The Unsafe package is used for compilation to bypass the Go language’s type system and manipulate memory directly. For example, the unsafe package is used to manipulate unexported members of a structure. The Unsafe package gave me the ability to read and write directly to memory.

The Unsafe package has only two types and three functions, but it is powerful.

type ArbitraryType int
type Pointer *ArbitraryType

func Sizeof(x ArbitraryType) uintptr
func Offsetof(x ArbitraryType) uintptr
func Alignof(x ArbitraryType) uintptr
Copy the code

ArbitraryType is an alias for int, and in Go ArbitraryType has a special meaning. Represents an arbitrary Go expression type. Pointer is an alias for the int Pointer type. In Go, you can cast any Pointer type to unsafe.Pointer.

All three functions take arguments of type ArbitraryType, which accepts variables of any type.

  • SizeofIt takes a value of any type (an expression) and returns the number of bytes it takes, unlike C, where sizeof takes a type and is a value, such as a variable.
  • Offsetof: Returns the number of bytes in memory from the start of the structure. The argument must be a member of the structure (the address to which the structure pointer points is the start of the structure, the memory address of the first member).
  • Alignof returns the number of bytes aligned with a variable. This function accepts any type of variable, but only if the variable is onestructType, and you can’t put this directlystructType as an argument, only thisstructThe e value of a type variable is used as an argument, but the details will be covered in a future article on memory alignment.

Note that all three of these functions return the uintptr type, which is interchangeable with unsafe.Pointer. All three functions are executed at compile time

unsafe.Pointer

Unsafe.Pointer is called a generic Pointer, and the official documentation has four important descriptions for this type:

  1. A Pointer of any type can be converted to Pointer
  2. Pointer can be converted to a Pointer of any type
  3. Uintptr can be converted to Pointer
  4. Pointer can be converted to uintptr

Unsafe.Pointer is a specially-defined Pointer type that, in Go, is used as a bridge between Pointers and can hold the addresses of variables of any type.

What does “address that can hold variables of any type” mean? Unbroadening.Pointer a variable that is converted using unbroadening. The variable must be of a Pointer type, otherwise the compiler will error.

a := 1
b := unsafe.Pointer(a) / / an error
b := unsafe.Pointer(&a) / / right
Copy the code

Like normal Pointers, unbroadening.Pointer is also comparable, and supports null comparisons to nil.

Unsafe.Pointer cannot be mathematically evaluated directly, but it can be converted to uintptr, which is mathematically evaluated to the Uintptr type, and then to the uint. Pointer type.

// uintptr <==> unda.Pointer <==> *TCopy the code

uintptr

Uintptr is a built-in type of Go language. It is an integer that can store Pointers. On 64-bit platforms, the underlying data type is uint64.

// uintptr is an integer type that is large enough to hold the bit pattern of
// any pointer.
type uintptr uintptr

typedef unsigned long long int  uint64;
typedef uint64          uintptr;
Copy the code

An unsafe.Pointer can also be converted to a uintptr and stored in a variable of the uintptr type (note: this variable only has the same numeric value as the current Pointer, not a Pointer), which can then be used to do the necessary Pointer numeric operations. (Uintptr is an unsigned integer that is large enough to hold an address.) This conversion is also reversible, but arbitrarily converting one of the uintptr Pointers to unsafe.Pointer might break the type system because not all numbers are valid memory addresses.

One other thing to note is that the Uintptr doesn’t have pointer semantics, meaning that the memory address that stores the uintptr values is reclaimed when GC occurs in Go. Unsafe.Pointer has Pointer semantics, which protects it from garbage collection.

Having talked about concepts, we’ll take a look at how to use unsafe.Pointer to transform and how to read and write private members of the Uintptr structure.

The sample application

Use broadening.Pointer for Pointer type conversions

import (
    "fmt"
    "reflect"
    "unsafe"
)
 
func main(a) {
 
    v1 := uint(12)
    v2 := int(13)
 
    fmt.Println(reflect.TypeOf(v1)) //uint
    fmt.Println(reflect.TypeOf(v2)) //int
 
    fmt.Println(reflect.TypeOf(&v1)) //*uint
    fmt.Println(reflect.TypeOf(&v2)) //*int
 
    p := &v1
    p = (*uint)(unsafe.Pointer(&v2)) // Use unbroadening.Pointer for type conversion
 
    fmt.Println(reflect.TypeOf(p)) // *unit
    fmt.Println(*p) / / 13
}
Copy the code

Use broadening.Pointer to read and write a private member of a structure

The Offsetof method can obtain the Offsetof the structure member, and then obtain the address of the member, read and write the address of the memory, you can achieve the purpose of changing the value of the member.

Here’s a memory allocation related fact: a structure is allocated a contiguous chunk of memory, and the address of the structure also represents the address of the first member.

package main
 
import (
    "fmt"
    "unsafe"
)
 
func main(a) {
 
    var x struct {
        a int
        b int
        c []int
    }
 
    // unsafe.Offsetof must take a field, such as x.b. The method returns the Offsetof field b to the start address of x, including possible empty Spaces.

    // uintptr(unsafe.pointer (&x)) + unsafe.offsetof (x.b)
    
    // equivalent to pb := &x.b
    pb := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))

    *pb = 42
    fmt.Println(x.b) 42 "/ /"
}
Copy the code

This is not a bad thing, though cumbersome, because these features should be used with caution. Don’t try to introduce a temporary variable of type Uintptr as it can break the security of your code

It is risky to use the following instead:

tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)
pb := (*int16)(unsafe.Pointer(tmp))
*pb = 42
Copy the code

As the program progresses, Goroutine often expands or shrinks the stack, copying data from the old stack into the new stack and changing the pointer orientation. An unsafe.Pointer is a Pointer, so the Pointer will be updated when the data it points to is moved to a new stack. But the uintptr temporary variable is just an ordinary number, so its value shouldn’t change. The error code introduces a non-pointer temporary variable TMP, causing the system to fail to recognize this as a pointer to variable x. By the time the second statement is executed, the data of variable x may have been transferred, and the temporary variable TMP is no longer the current address of &x.b. A third assignment to a previously invalid address space will crash the entire program.

String and []byte zero-copy conversion

This is a very classic example. Implement zero-copy conversion between string and bytes slices.

String and []byte are represented at runtime as reflect.stringHeader and reflect.sliceHeader

type SliceHeader struct {
	Data uintptr
	Len  int
	Cap  int
}

type StringHeader struct {
	Data uintptr
	Len  int
}
Copy the code

Zero-copy conversion can be achieved by simply sharing the underlying []byte array.

The code is relatively simple and will not be explained in detail. Constructor reflect.StringHeader and reflect.SliceHeader to convert string to []byte.

import (
	"fmt"
	"reflect"
	"unsafe"
)

func main() {
	s := "Hello World"
	b := string2bytes(s)
	fmt.Println(b)
	s = bytes2string(b)
	fmt.Println(s)

}

func string2bytes(s string) []byte {
	stringHeader := (*reflect.StringHeader)(unsafe.Pointer(&s))

	bh := reflect.SliceHeader{
		Data: stringHeader.Data,
		Len: stringHeader.Len,
		Cap: stringHeader.Len,
	}

	return *(*[]byte)(unsafe.Pointer(&bh))
}

func bytes2string(b []byte) string {
	sliceHeader := (*reflect.SliceHeader)(unsafe.Pointer(&b))

	sh := reflect.StringHeader{
		Data: sliceHeader.Data,
		Len:  sliceHeader.Len,
	}

	return *(*string)(unsafe.Pointer(&sh))
}
Copy the code

conclusion

The Unsafe package is also widely used in the Go source code. Using the unsafe package to bypass the Go pointer and directly manipulate memory is risky, but in some cases can improve code efficiency.

The resources

  • Unscrambling the Go language in depth

  • The unsafe. Pointer and uintptr: www.cnblogs.com/echojson/p/…

If you like my article, please give me a thumbs up. I will share what I have learned and seen and first-hand experience in technical articles every week. Thank you for your support. Wechat search concerns the public number “network management talking BI talking” every week to teach you an advanced knowledge, there are special for the development of engineers Kubernetes tutorial.