After watching Su Bingtian enter the finals, I was so excited that I dared not delay more than 5 minutes to go to the toilet.
This historic moment made me, who had decided to rest, sit up in my dying illness and begin to write.
primers
Today’s article was introduced from a bug I worked on on Saturday. The context is that there is an Labels slice in a struct, and the Labels in configuration variables need to be added to it when assembling data.
Let’s see what goes wrong.
for i := range m{
m[i].Labels = append(r.Config.Relabel, m[i].Labels...) . }Copy the codeIf I =0, it is normal. If I =0, it is normal. If I =0, it is normal. .labels contents.
Append adds data directly to the first parameter slice when it has enough capacity.
Change to the following code, switch the position, everything is fine.
m[i].Labels = append(m[i].Labels,r.Config.Relabel...)
Copy the codeThis is an implicit pitfall, as assignment copies in GO tend to be shallow copies, and it’s easy for developers to overlook this, leading to unexpected problems that should be addressed in the future.
Today’s article starts with this problem and the deep copy problem mentioned in the assignment for the previous article.
What is light? What is a deep?
I did c++ many years ago, and its object copy is shallow copy, the principle is to call the default copy constructor, need to be manually overwritten, the copying process, especially Pointers need to be carefully generated release, to avoid memory leaks.
Later, I worked with Python and found that the problem of shallow and deep copying is common in backend languages, and Go is no exception.
A shallow copy is a full copy for a value type and a copy of its address for a reference type. That is, copying an object that changes a variable of the reference type also affects the source object.
This is why, when a channel is passing parameters, it writes something internally and the receiver receives it successfully.
In Go, Pointers, slice, channels, interfaces, maps, and functions are shallow copies. The three most problematic types are Pointers, slices, and maps.
The convenience is that you pass it as an argument without taking the address and can modify its contents, and you don’t need to return a value as long as there is no overwriting inside the function.
However, as a member variable in the structure, the problem is exposed after copying the structure. Changes in one place result in changes in the other.
Four ways to deep copy
When I was talking with my girlfriend about deep copy, she told me that the most convenient way to serialize deep copy is to serialize it to JSON and then deserialize it.
When I heard about this solution, I was amazed. It was really easy, but since serialization uses reflection, it wasn’t very efficient.
There are four ways to make a deep copy
- 1. Handwritten copy function
- 2,
jsonSerialization deserialization - 3,
gobSerialization deserialization - 4. Use reflection
Github’s open source libraries are mostly optimized in two ways: 1 and 4. The reflection method here will be discussed later.
My lot github.com/minibear233… A component will be written later to provide various ready-made ways of deep copying.
Handwritten copy function
Defines a structure that contains slices, dictionaries, and Pointers.
type Foo struct {
List []int
FooMap map[string]string
intPtr *int
}
Copy the codeCopy the function manually and name it Duplicate
func (f *Foo) Duplicate(a) Foo {
var tmp = Foo{
List: make([]int.0.len(f.List)),
FooMap: make(map[string]string),
intPtr: new(int),}copy(tmp.List, f.List)
for i := range f.FooMap {
tmp.FooMap[i] = f.FooMap[i]
}
iff.intPtr ! =nil {
*tmp.intPtr = *f.intPtr
} else {
tmp.intPtr = nil
}
return tmp
}
Copy the code- Initializes the structure inside the function
copyIs a copy function that comes with the librarymaponlyrangeLet’s copy it. HeremapfornilNot an error- A pointer must be nulled before it is used, assigning a value to the pointer rather than overwriting the pointer address
test
func main(a) {
var a = 1
var t1 = Foo{intPtr: &a}
t2 := t1.Duplicate()
a = 2
fmt.Println(*t1.intPtr)
fmt.Println(*t2.intPtr)
}
Copy the codeThe deep copy succeeded
2
1
Copy the codeJson serialization deserialization
This is a simple way to do a deep copy, but the structure must be annotated and no private fields are allowed
type Foo struct {
List []int `json:"list"`
FooMap map[string]string `json:"foo_map"`
IntPtr *int `json:"int_ptr"`
}
Copy the codeOffer a direct solution
func DeepCopyByJson(dst, src interface{}) error {
b, err := json.Marshal(src)
iferr ! =nil {
return err
}
err = json.Unmarshal(b, dst)
return err
}
Copy the code- Among them
srcanddstIt’s the same type of structure dstThe address must be used because a new value is changed to the data the address points to
Usage, I omitted error handling
a = 3
t1 = Foo{IntPtr: &a}
t2 = Foo{}
_ = DeepCopyByJson(&t2, t1)
fmt.Println(*t1.IntPtr)
fmt.Println(*t2.IntPtr)
Copy the codeThe output
3
3
Copy the codeGob serialization deserialization
This is an encoding method provided by the library, similar to Protobuf, Gob (short for Go Binary). Similar to pickle in Python and Serialization in Java.
Encoded at the sender and decoded at the receiver.
func DeepCopyByGob(dst, src interface{}) error {
var buffer bytes.Buffer
iferr := gob.NewEncoder(&buffer).Encode(src); err ! =nil {
return err
}
return gob.NewDecoder(&buffer).Decode(dst)
}
Copy the codeusage
a = 4
t1 = Foo{IntPtr: &a}
t2 = Foo{}
_ = DeepCopyByGob(&t2, t1)
fmt.Println(*t1.IntPtr)
fmt.Println(*t2.IntPtr)
Copy the codeThe output
4
4
Copy the codeBenchmarking (Performance testing)
I wrote benchmark test cases for each of these three methods, and GO would automatically be called repeatedly until a reasonable time range was calculated.
The benchmark code, here is just one, the other two functions are tested in a similar way:
func BenchmarkDeepCopyByJson(b *testing.B) {
b.StopTimer()
var a = 1
var t1 = Foo{IntPtr: &a}
t2 := Foo{}
b.StartTimer()
for i := 0; i < b.N; i++ {
_ = DeepCopyByJson(&t2, t1)
}
}
Copy the codeRun the test
$ go test -test.bench=. -cpu=1.16 -benchtime=2s
goos: darwin
goarch: amd64
pkg: my_copy
cpu: Intel(R) Core(TM) i5- 8257.U CPU @ 1.40GHz
BenchmarkFoo_Duplicate 35887767 62.64 ns/op
BenchmarkFoo_Duplicate- 16 37554250 62.56 ns/op
BenchmarkDeepCopyByGob 104292 22941 ns/op
BenchmarkDeepCopyByGob- 16 103060 23049 ns/op
BenchmarkDeepCopyByJson 2052482 1171 ns/op
BenchmarkDeepCopyByJson- 16 2057090 1175 ns/op
PASS
ok my_copy 17.166s
Copy the code- in
macThere is no significant difference between mononuclear and multicore environments - Running speed,
Manual copy mode > JSON > GOB - Copy methods are two orders of magnitude different
summary
Occasional applications can be copied using json serialization deserialization, but there is a drawback, besides being slow, that private member variables cannot be copied.
For frequently copied programs, manual copy is recommended and the copy process can be customized. You can even customize the subtle differences in fields between different constructs.
PS: Built-in copy and reflect.copy support only slice or array copy, built-in copy is more than twice as fast as reflection.
Expand the data
- The language to transmit data using Gob c.biancheng.net/view/4597.h…).
- Built-in copy function and reflect the copy function difference studygolang.com/topics/1352…
- Benchmark segmentfault.com/a/119000001…
Highlights from the past
- Go Standard library: JSON parsing traps and lazy tricks for version changes
- Explain the concurrent receive control structure SELECT in Go
- Go Exception handling details
- Thinking about Go structure
- Go concurrent wait
Did not expect to finish the article or to the next day, the original is not easy, point a praise to support, love you yao yao!
This time we briefly touched on benchmarking, but next time we’ll go into more detail about unit tests and benchmarking, and we’ll see you next time!
I hope you can help me click a like, double click, encourage me, this is very important to me, thank you sister!