This is the 14th day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

Protobuf, short for Protocol Buffers, is a language-independent, platform-independent, extensible data description language for serializing structured data. As a description language for interface specifications, Protobuf can be used as a basic tool for designing secure cross-language PRC interfaces.

The basic grammar

Hello. Proto file

syntax = "proto3";

package main;

message String {
    string value = 1;
}
Copy the code
  1. The first line declares the use of the Proto3 syntax. Otherwise, the proto2 syntax is used by default. Currently, V3 is recommended. This declaration must be the first non-empty, non-commented line of the file.
  2. The package directive indicates that the current package is main, and the user can customize the package path and name for different languages.
  3. The message keyword defines a String message body, which corresponds to a String structure in the resulting Go language code. Each message body field contains three attributes: type, field name, and field number. The definition of the message body is not repeatable except for the type. Here there is only one String value member encoded with a 1 number instead of a name.
  4. The most basic unit of data in a Protobuf is a message, similar to a structure in the Go language. Members of message or other underlying data types can be nested within message.

About Identification Number

The fields in the message body define unique numeric values. These numbers are used to identify individual fields in the binary format of the message and cannot be changed once they are used. Note: identifiers within [1,15] occupy one byte during encoding. The id within [16,2047] occupies 2 bytes. Therefore, you should reserve identifiers within [1,15] for message elements that occur frequently.

The minimum id can start at 1 and go up to 2^ 29-1, or 536,870,911. Do not use the [19000-19999] identifier, which is reserved in the Protobuf protocol implementation. If you must use these reserved identifiers in a.proto file, an alarm will be raised at compile time. Similarly, you can’t use any identifiers you’ve reserved previously.

Add comments

.proto files to add comments, you can use C/C++ style // and /*… */ Syntax format

Keep field

If the and field is removed from the previously defined message, its field number should be reserved, using the keyword reserved:

syntax "proto3";

message Stock {
    reserved 3.4;
    // ...
}
Copy the code

You can also use the reserved keyword as a placeholder for fields that might be added in the future. Successive field numbers can be occupied using the to keyword.

syntax "proto3";

message Info {
    reserved 2.9 to 11.15;
    // ...
}
Copy the code

Generate the corresponding Go code

The Protobuf core toolset is developed in C++, and the official protoc compiler does not support Go. To generate the corresponding Go code based on the hello.proto file above, you need to install the corresponding plug-in.

  • Install the official Protoc tool from github.com/google/prot… Download.
  • Installed on the Go code generation plug-in, Go through the get github.com/golang/protobuf/protoc-gen-go command to install.

Generate the corresponding Go code with the following command:

$ protoc --go_out=. hello.proto

  • The go_out parameter tells the Protoc compiler to load the corresponding protoc-gen-go tool and place the generated code in the current directory. Finally, a list of Protobuf files to process.

  • The plugins = plugin1 + plugin2: The proto file we defined refers to RPC services, and RPC code is not generated by default, so we need to give plugins in go_out to protoc-gen-go to tell the compiler, Please support RPC (the built-in GRPC plug-in is specified here).

Basic data types

With a protobuf, the generated data type is not exactly the same as the original type. Here are some common type mappings:

The generated hello.pb.go file

The pb.go file is the corresponding GO code generated from the proto file, which will be referenced in the actual application.

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: hello.proto

package main

import (
	fmt "fmt"
	proto "github.com/golang/protobuf/proto"
	math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type String struct {
	Value                *String  `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

func (m *String) Reset(a)         { *m = String{} }
func (m *String) String(a) string { return proto.CompactTextString(m) }
func (*String) ProtoMessage(a)    {}
func (*String) Descriptor(a) ([]byteAnd []int) {
	return fileDescriptor_61ef911816e0a8ce, []int{0}}func (m *String) XXX_Unmarshal(b []byte) error {
	return xxx_messageInfo_String.Unmarshal(m, b)
}
func (m *String) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
	return xxx_messageInfo_String.Marshal(b, m, deterministic)
}
func (m *String) XXX_Merge(src proto.Message) {
	xxx_messageInfo_String.Merge(m, src)
}
func (m *String) XXX_Size(a) int {
	return xxx_messageInfo_String.Size(m)
}
func (m *String) XXX_DiscardUnknown(a) {
	xxx_messageInfo_String.DiscardUnknown(m)
}

var xxx_messageInfo_String proto.InternalMessageInfo

func (m *String) GetValue(a) *String {
	ifm ! =nil {
		return m.Value
	}
	return nil
}

func init(a) {
	proto.RegisterType((*String)(nil), "main.String")}func init(a) { proto.RegisterFile("hello.proto", fileDescriptor_61ef911816e0a8ce) }

var fileDescriptor_61ef911816e0a8ce = []byte{
	// 84 bytes of a gzipped FileDescriptorProto
	0x1f.0x8b.0x08.0x00.0x00.0x00.0x00.0x00.0x02.0xff.0xe2.0xe2.0xce.0x48.0xcd.0xc9.0xc9.0xd7.0x2b.0x28.0xca.0x2f.0xc9.0x17.0x62.0xc9.0x4d.0xcc.0xcc.0x53.0xd2.0xe1.0x62.0x0b.0x2e.0x29.0xca.0xcc.0x4b.0x17.0x52.0xe2.0x62.0x2d.0x4b.0xcc.0x29.0x4d.0x95.0x60.0x54.0x60.0xd4.0xe0.0x36.0xe2.0xd1.0x03.0xc9.0xeb.0x41.0x24.0x83.0x20.0x52.0x49.0x6c.0x60.0xad.0xc6.0x80.0x00.0x00.0x00.0xff.0xff.0x76.0x0c.0x0e.0x54.0x49.0x00.0x00.0x00,}Copy the code

  • The String type automatically generates a set of methods in which the ProtoMessage method indicates that this is a method that implements the proto.Message interface. Protobuf also generates a Get method for each member, which provides a convenient way of taking values, handles some cases of null pointer values, and can be Reset by using the Reset method.

  • .pb.go file initialization method. Note statements related to fileDescriptor. FileDescriptor_61ef911816e0a8ce the fileDescriptor_61ef911816e0a8ce representation is a compiled proto file and is the overall description of the proto file. It contains proto file names, import content, package names, option Settings, all defined message bodies, all defined enums, all defined services, all defined RPC methods, and so on.

  • Each Message Type contains a Descriptor method, which describes a Message body definition. This method will look for its own Message Field in the fileDescriptor and return it.

Combination of Protobuf and RPC

Re-implement HelloService based on String type

package main

import (
	"log"
	"net"
	"net/rpc"
	"rpc/protoc"
)

// HelloService is rpc server obj
type HelloService struct{}

// Input and output parameters of the Hello method are represented by the Protobuf defined String type.
// Since the new input parameter is the structure type, we use the pointer type as the input parameter. The internal code of the function has also been adjusted accordingly.
func (p *HelloService) Hello(request *protoc.String, reply *protoc.String) error {
	reply.Value = "hello:" + request.GetValue()
	return nil
}

func main(a) {
	rpc.RegisterName("HelloService".new(HelloService))

	listener, err := net.Listen("tcp".": 1234")
	iferr ! =nil {
		log.Fatal("ListenTCP error:", err)
	}

	conn, err := listener.Accept()
	iferr ! =nil {
		log.Fatal("Accept error", err)
	}

	rpc.ServeConn(conn)
}
Copy the code

Here is the code for the HelloService request client.go:

package main

import (
	"fmt"
	"log"
	"net/rpc"
	"rpc/protoc"
)

func main(a) {
	client, err := rpc.Dial("tcp"."localhost:1234")
	iferr ! =nil {
		log.Fatal("dialing err:", err)
	}

	var reply = &protoc.String{}
	var param = &protoc.String{
		Value: "hello wekenw",
	}

	err = client.Call("HelloService.Hello", &param, &reply)
	iferr ! =nil {
		log.Fatal(err)
	}
	fmt.Println(reply)
}
Copy the code

Start the server. Start the client. The client execution result is as follows:

$ go run client.go
value:"hello:hello wekenw"
Copy the code