GRPC: Use Buf to quickly compile GRPC proto files

introduce

As anyone who has used GRPC should know, the protocol buffer requires compiling the *. Proto file into the *. Go file using the associated command line.

Depending on your needs, different command line files are used. Take the Go language as an example, we need the following command line files.

In addition to installing the above command line, we need to run at least four different commands to compile the *. Proto file as needed, which is quite obscure.

Please visit the following address for the full tutorial:

• rkdev.info/cn
• Rkdocs.net lilify. app/cn (standby)

useBufcompile

We can quickly configure the compilation process through Buf. Although some configuration is required up front, it is much simpler and safer than writing complex scripts.

Let’s take a look at an example.

example

Let’s take a simple Hello World example and generate a grPC-based microservice step by step.

• example
• The document

Step 1: Install the command line

You are advised to use the RK command line interface (CLI) to quickly install required tools.

# Install RK CLI
$ go get -u github.com/rookie-ninja/rk/cmd/rk

# List available installation
$ rk installCOMMANDS: buf install buf on local machine cfssl install cfssl on local machine cfssljson install cfssljson on local machine gocov  install gocov on local machine golangci-lint install golangci-lint on local machine mockgen install mockgen on local machine pkger install pkger on local machine protobuf install protobuf on local machine protoc-gen-doc install protoc-gen-doc on local machine protoc-gen-go install protoc-gen-go on local machine protoc-gen-go-grpc install protoc-gen-go-grpc on local machne protoc-gen-grpc-gateway install protoc-gen-grpc-gateway on local machine protoc-gen-openapiv2 install protoc-gen-openapiv2 on local machine swag install swag on local machine rk-std install rk standard environment on local machine help, h Shows a list of commands or help for one command
# Install protobuf, buf, protoc-gen-go, protoc-gen-go-grpc, protoc-gen-grpc-gateway, protoc-gen-openapiv2
$ rk install protobuf
$ rk install protoc-gen-go
$ rk install protoc-gen-go-grpc
$ rk install protoc-gen-go-grpc-gateway
$ rk install protoc-gen-openapiv2
$ rk install buf
Copy the code

You can also install it on the official website

Step 2: Create API /v1/greeter.proto

syntax = "proto3";

package api.v1;

option go_package = "api/v1/greeter";

service Greeter {
  rpc Greeter (GreeterRequest) returns (GreeterResponse) {}}message GreeterRequest {
  string name = 1;
}

message GreeterResponse {
  string message = 1;
}
Copy the code

Step 3: Create API /v1/gw_mapping.yaml

We use the gw_mapping.yaml file to map GRPC -> Restful apis. This way we can avoid writing a bunch of option code in the *.proto file.

For the syntax, see github.com/googleapis/…

type: google.api.Service
config_version: 3

# Please refer google.api.Http in https://github.com/googleapis/googleapis/blob/master/google/api/http.proto file for details.
http:
  rules:
    - selector: api.v1.Greeter.Greeter
      get: /api/v1/greeter
Copy the code

Step 4: Create buf.yaml

Yaml tells BUf where to look for the Proto file. We specify API/folder.

version: v1beta1
name: github.com/rk-dev/rk-demo
build:
  roots:
    - api
Copy the code

Step 5: Create buf.gen. Yaml

The following parameters tell BUF what to do when compiling. In our documents, we mainly do the following things.

• Specify the compiled files to be placed in the API /gen folder
• Compile the proto file
• Compile grPC-related proto files
• Compile the proto file associated with grPC-gateway
• Openapi-v2-related file (Swagger)

version: v1beta1
plugins:
  # protoc-gen-go needs to be installed, generate go files based on proto files
  - name: go
    out: api/gen
    opt:
     - paths=source_relative
  # protoc-gen-go-grpc needs to be installed, generate grpc go files based on proto files
  - name: go-grpc
    out: api/gen
    opt:
      - paths=source_relative
      - require_unimplemented_servers=false
  # protoc-gen-grpc-gateway needs to be installed, generate grpc-gateway go files based on proto files
  - name: grpc-gateway
    out: api/gen
    opt:
      - paths=source_relative
      - grpc_api_configuration=api/v1/gw_mapping.yaml
  # protoc-gen-openapiv2 needs to be installed, generate swagger config files based on proto files
  - name: openapiv2
    out: api/gen
    opt:
      - grpc_api_configuration=api/v1/gw_mapping.yaml
Copy the code

Step 6: Compile Proto

Once the configuration is complete, no matter how the *. Proto file is modified, we can compile the *. Proto file by running buf generate, which is very convenient.

$ buf generate
Copy the code

The following files will be created.

$ tree api/genAPI/gen └ ─ ─ v1 ├ ─ ─ greeter. Pb. Go ├ ─ ─ greeter. Pb. Gw go ├ ─ ─ greeter. Swagger. Json └ ─ ─ greeter_grpc. Pb. Go 1 directory, 4 filesCopy the code

Step 7: Reference in main.go

Now that we’ve compiled the *.proto file, all that’s left is to reference the proto API we just generated in the main.go file.

Here, we introduce rK-boot library, through which users can quickly build grPC-based micro-services.

• The document
• The source code
• example

Create the boot. Yaml

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 8080                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    sw:
      enabled: true               # Enable Swagger UI
      jsonPath: "api/gen/v1"      # Boot will look for swagger config files from this 
Copy the code

Create the main. Go

package main

import (
   "context"
   "fmt"
   "github.com/rookie-ninja/rk-boot"
   "github.com/rookie-ninja/rk-demo/api/gen/v1"
   "google.golang.org/grpc"
)

// Application entrance.
func main(a) {
   // Create a new boot instance.
   boot := rkboot.NewBoot()

    / / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
    // ******* Register GRPC & Gateway *******
    / / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

   // Get grpc entry with name
   grpcEntry := boot.GetGrpcEntry("greeter")
    // Register grpc registration function
   grpcEntry.AddRegFuncGrpc(registerGreeter)
    // Register grpc-gateway registration function
   grpcEntry.AddRegFuncGw(greeter.RegisterGreeterHandlerFromEndpoint)

   // Bootstrap
   boot.Bootstrap(context.Background())

   // Wait for shutdown sig
   boot.WaitForShutdownSig(context.Background())
}

// Implementation of [type GrpcRegFunc func(server *grpc.Server)]
func registerGreeter(server *grpc.Server) {
   greeter.RegisterGreeterServer(server, &GreeterServer{})
}

// Implementation of grpc service defined in proto file
type GreeterServer struct{}

func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) {
   return &greeter.GreeterResponse{
      Message: fmt.Sprintf("Hello %s!", request.Name),
   }, nil
}
Copy the code

Folder structure

$ tree. ├ ─ ─ API │ ├ ─ ─ gen │ │ └ ─ ─ v1 │ │ ├ ─ ─ greeter. Pb. Go │ │ ├ ─ ─ greeter. Pb. Gw go │ │ ├ ─ ─ greeter. Swagger. Json │ │ └ ─ ─ Greeter_grpc. Pb. Go │ └ ─ ─ v1 │ ├ ─ ─ greeter. Proto │ └ ─ ─ gw_mapping. Yaml ├ ─ ─ the boot. The yaml ├ ─ ─ buf. Gen. Yaml ├ ─ ─ buf. Yaml ├ ─ ─ ├── main. Go directories, 12 filesCopy the code

validation

$ go run main.go
Copy the code

$ curl "localhost:8080/api/v1/greeter? name=rk-dev"{"message":"Hello rk-dev!" }Copy the code

Access Swagger: localhost:8080/sw