preface

As mentioned earlier, gRPC uses Protobuf for serialization and deserialization, and also uses it as an interface definition language, and one of the core of Protobuf is the use of protoc tools, today, let me take you to explore the pit of Protoc!

Note: The protoc version used in this article is 3.7.1

The installation

#The installation
$ brew install protoc

#Check the version
$ protoc --versionLibprotoc 3.7.1Copy the code

Based on article

Protoc: Protoc: protoc: protoc: protoc: protoc

├ ─ ─ proto1 │ ├ ─ ─ greeter │ │ ├ ─ ─ greeter. Proto │ │ └ ─ ─ greeter_v2. Proto │ └ ─ ─ pb_goCopy the code

Greeter. proto file contents:

syntax = "proto3";

package greeter;

option go_package="proto1/greeter";

service Greeter {
    rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
    string name = 1;
}
message HelloReply {
    string message = 1;
}
Copy the code

The compile command is as follows:

protoc –proto_path=. –go_out=. proto1/greeter/greeter.proto

The instructions above can be broken down into three parts, each corresponding to the three important parameters of the Protoc. Let’s first look at the parameters provided by the Protoc:

$ protoc --helpUsage: protoc [OPTION] proto_files-ipath, --proto_path=PATH specifies search PATH --plugin=EXECUTABLE:.... --cpp_out=OUT_DIR Generate C++ header and source. --csharp_out=OUT_DIR Generate C# source file. --java_out=OUT_DIR Generate Java source file. --js_out=OUT_DIR Generate JavaScript source. --objc_out=OUT_DIR Generate Objective C header and source. --php_out=OUT_DIR Generate PHP source file. --python_out=OUT_DIR Generate Python source file. --ruby_out=OUT_DIR Generate Ruby source file @<filename> protoCopy the code

1. Search for path parameters

The first important parameter is the search PATH parameter, shown above -ipath, –proto_path=PATH. Proto file, which can be specified with either -i or –proto_path=.

If this parameter is not specified, the search is performed in the current path by default. In addition, this parameter can be specified multiple times, which means we can specify multiple paths to search.

These three instructions, for example, all mean the same thing

# Makefile
GOPATH:=$(shell go env GOPATH)

#Proto1 Related instructions.PHONY: proto1 path1: protoc --go_out=. proto1/greeter/greeter.proto path2: Protoc - i. -- go_out =. Proto1 / greeter/greeter. Proto # dot represents the current path, pay attention to the -i parameter is not equal to path3: protoc --proto_path=. --go_out=. proto1/greeter/greeter.protoCopy the code

Note: I would prefer to write protoc build commands in makefiles because they are easy to find and easy to execute.

2. Language plug-in parameters

Language parameters are –cpp_out=, –python_out=, etc. Protoc supports up to 13 languages, all of which are fairly common. The language parameter shown above indicates that the protoc already has a built-in compiler for that language, so we don’t need to install it.

–go_out= –go_out= –go_out= –go_out= –go_out= –go_out= –go_out= –go_out= –go_out= –go_out=

#The latest version
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

#Specify the version
$Go install google.golang.org/protobuf/cmd/[email protected]
Copy the code

If you want to learn more about the compiled plug-in for the language, you can click on the official documentation of the Protobuf.

3. Proto File location parameters

Proto file location parameters namely the @ < filename >, specify the location of our proto file, such as proto1 / greeter/greeter. Proto.

Advanced article

Two confusable parameters

Proto files (non-protoc) have two confusable arguments, package and xx_package. Xx refers to the language in which you are compiled, such as go_package if you are programming in Go.

package

The package argument is for the protobuf, the namespace of the proto file, and is used to avoid collisions with our defined interface or message.

For an example, suppose I have two files a. proto and B.proto, as follows

# A.proto
message UserInfo {
    uint32 uid = 1;
    string name = 2;
}

# B.proto
message UserInfo {
    uint32 uid = 1;
    string name = 2;
    uint32 age = 3;
    string work = 4;
}
Copy the code

As shown above, both files have A UserInfo message, so if I need to reference file B in file A, if I do not specify the package, I cannot tell whether I want to call the UserInfo of A or B.

xx_package

Here is an example using go_package. This parameter mainly declares the location of the Go code, or it can solve the package name problem (because the proto file will generate a.pb. Go file after compilation, since it is a Go file, there is a package name problem).

The normal path to.pb.go is to place all.pb.go files in a proto file of the same name. However, some people do not want to do this. For example, they want to store all.pb.go files in a specific folder, such as pb_go.

The first:

#Change --go_out and leave go_package unchanged
$ protoc --proto_path=. --go_out=./proto1/pb_go proto1/greeter/greeter.protoThe generated PB file is in the pb_go/proto1/greeter directory, which is a bit redundant, but the package name of the pb file is still greeterCopy the code

The second:

#Modify go_package and leave go_OUT unchanged
option go_package="proto1/pb_go";

$ protoc --proto_path=. --go_out=. proto1/greeter/greeter_v2.protoThe generated PB file is stored in the pb_go directory. The pb file package name is pb_goCopy the code

In addition, xx_package actually has two ways to declare, the first is to declare in the file, you have seen; The second option is to declare it on the command line in the following format: M${PROTO_FILE}=${GO_IMPORT_PATH}.

Currently, official documentation is recommended to state that the purpose is to shorten the length of the Protoc directive, and I personally recommend that.

Ok, I believe that through the above examples, you can roughly understand the difference between package and XX_package. Again, it has been stated in the official documents that package and XX_package are not related and belong to different categories.

–go_out Detailed interpretation

Paths =import:., –go_out=paths=source_relative:., or –go_out=plugins= GRPC :.

What does that mean?

The –go_out parameter is used to specify how the protoc-gen-Go plugin will work and where the GO code will be generated, and this is how the plugin will work.

The two main parameters of go_OUT are plugins and paths, which represent the plug-in used to generate the Go code and the location of the generated Go code, respectively. –go_out is written with commas separating arguments and colons at the end to specify where the code is generated, such as –go_out=plugins= GRPC,paths=import:.

The paths parameter has two options, import and source_relative. By default, import is used to create the directory level according to the full package path of the generated Go code. Source_relative indicates that the directory level of the Go code is created according to the directory level of the proto source file, except if the directory already exists.

For example:

option go_package="proto1/pb_go";

#Directive 1: Paths is import and the PB file ends up in the pb_go directory
$ protoc --proto_path=. --go_out=. proto1/greeter/greeter_v2.proto
$ protoc --proto_path=. --go_out=paths=import:. proto1/greeter/greeter_v2.proto

#Directive 2: Paths are source_relative and PB files end up in the Proto1 / Greeter directory
$ protoc --proto_path=. --go_out=paths=source_relative:. proto1/greeter/greeter_v2.proto
Copy the code

Plugins include GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code, GRPC code

Import other proto files

Sometimes we may have multiple proTO files, and each proTO file may not be completely independent, but they may refer to each other. What can we do in this case?

└── anti├ ─ common. Proto ├─ greeterCopy the code

Greeter. proto contains the following contents:

syntax = "proto3";

package greeter;

import "proto2/common.proto";

option go_package="proto2/greeter";

service Greeter {
    rpc SayHello (common.Request) returns (common.Response) {}
}
Copy the code

For example, if we have a common proto file called common.proto, greeter.proto can use the import keyword if we want to reference the message in it.

The compilation instructions are as follows:

protoc –proto_path=. –go_out=. proto2/greeter/greeter.proto proto2/common.proto

On protoc-gen-go multi-packet problem

Suppose the current proto file is stored as follows:

Proto ├─ plain, plain, plain, plain, plain, plainCopy the code

If you want to compile all the proto files (assuming the Go language is generated), the normal command would look like this:

protoc –proto_path=. –go_out=. proto/*.proto proto/user/*proto proto/greeter/*proto

But some friends may want to be lazy, want to directly like this:

protoc –proto_path=. –go_out=. proto/*.proto

The answer is no, because protoc-gen-go does not support this form and will only compile common.proto in the end

Write in the last

This article mainly describes the basic usage of protoc, and several protoc often encountered problems, these problems are mainly around the generation of PB file location, and how to define the package name, suggestions or more hands-on, more practice. The source code for this article is available on Github

This article back and forth to change several times, write hand pain, if feel useful, trouble for me to praise!

I am Yan Gan, welcome to follow my public account Yan Gan Coding, your likes and attention are the biggest motivation for my creation! !

Reference documentation

Implementation of Protobuf import functionality in Go projects