This paper is participating in theNetwork protocols must be known and must be known”Essay campaign

preface

For efficiency, most RPC frameworks serialize the contents of the transmission into a format that is convenient for transmission. Today, we will talk about the protobuf encoding in the GRPC framework described in the previous article. Google official documentation :(Protocol Buffers)

protobuf

Protobuf is a kind of binary data after serialization, which is very small in size, so the transmission efficiency is very high. Recently, when studying the live live barrage of Douyin, it was found that Protobuf is also used. It has three characteristics: 1. Good compatibility, independent platform across independent language, can cross any platform any language, binary coding 2. Strong performance, very small message volume after serialization 3. Good capacity expansion, updating the field does not affect the original data

Golang encodes protbuf

Start with a simple protobuf file called test.proto:

syntax = "proto3";
option go_package = "./test";
package test;

message Data {
   int32 code = 1;
   string msg = 2;
   string data = 3;
}
Copy the code

This format is very much like a goalng struct, but note that this file only defines the format and is not used for transport. Syntax indicates the protobuf version option go_package specifies the location of the output file after compilation. The package name generated after compilation defines a Data message structure. The element is preceded by the Data type, which will be compiled into binary code for transmission strictly according to the Data type during serialization. The 1, 2, and 3 in the last column represent the sequence of messages, ensuring order.

Then download the ProtocolBuffers /protobuf compiler for Windows

Copy protoc. Exe from bin to GOPATH/binThen execute it in the GOPATH directory

go get -u github.com/golang/protobuf/protoc-gen-go
Copy the code

Build compiler plugin protoc-gen-go

Then execute command compilation in the test.proto project root directory to generate the codec file that can be used by Goalng from the protobuf file:

protoc  --go_out=. *.proto
Copy the code

Generate test.pb.go file:

Now write a main.go file to experiment with protobuf serialization encoding:

package main import ( "io/ioutil" "main/test" "os" "github.com/golang/protobuf/proto" ) func main() { msg := &test.Data{ Code: 200, Msg: "success ", Data: } // Data encoding, Marshal(MSG) ioutil.WriteFile("./test.txt", data, os.modeperm) // Write the serialized file to test.txt}Copy the code

Remember to initialize the project with Go Mod Init Main and Go Mod Tidy before running and automatically download the required packages. Next run main.go:

Take a look at the test.txt file size:

On a whim, write a similar JSON save to look at the size:

{"code": 200," MSG ":" success ","data":" test "}Copy the code

After protbuf serialization, serialization is less than half of the JSON data