Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article also participated in the “Digitalstar Project” to win a creative gift package and creative incentive money

introduce

This article will show you how to implement API log tracking in a gRPC distributed scenario.

What is API log tracing?

An API request spans multiple microservices, and we want to retrieve logs for the entire link with a unique ID.

We will use RK-boot to start the gRPC service.

Please visit the following address for the full tutorial:

rkdev.info/cn

Rkdocs.net lilify. app/cn (standby)

The installation

go get github.com/rookie-ninja/rk-boot
Copy the code

Quick start

Rk-boot integrates grPC-gateway by default and starts by default.

We create/API /v1/ Greeter apis for validation and enable logging, Meta, and tracing interceptors.

1. Create API/v1 / greeter. Proto

syntax = "proto3";

package api.v1;

option go_package = "api/v1/greeter";

service Greeter {
  rpc Greeter (GreeterRequest) returns (GreeterResponse) {}
}

message GreeterRequest {
  string name = 1;
}

message GreeterResponse {
  string message = 1;
}
Copy the code

2. Create the API/v1 / gw_mapping. Yaml

type: google.api.Service
config_version: 3

# Please refer google.api.Http in https://github.com/googleapis/googleapis/blob/master/google/api/http.proto file for details.
http:
  rules:
    - selector: api.v1.Greeter.Greeter
      get: /api/v1/greeter
Copy the code

3. Create buf. Yaml

version: v1beta1
name: github.com/rk-dev/rk-demo
build:
  roots:
    - api
Copy the code

4. Create buf. Gen. Yaml

version: v1beta1
plugins:
  # protoc-gen-go needs to be installed, generate go files based on proto files
  - name: go
    out: api/gen
    opt:
     - paths=source_relative
  # protoc-gen-go-grpc needs to be installed, generate grpc go files based on proto files
  - name: go-grpc
    out: api/gen
    opt:
      - paths=source_relative
      - require_unimplemented_servers=false
  # protoc-gen-grpc-gateway needs to be installed, generate grpc-gateway go files based on proto files
  - name: grpc-gateway
    out: api/gen
    opt:
      - paths=source_relative
      - grpc_api_configuration=api/v1/gw_mapping.yaml
  # protoc-gen-openapiv2 needs to be installed, generate swagger config files based on proto files
  - name: openapiv2
    out: api/gen
    opt:
      - grpc_api_configuration=api/v1/gw_mapping.yaml
Copy the code

5. Compile proto file

$ buf generate
Copy the code

The following files will be created.

$tree API/API/gen gen └ ─ ─ v1 ├ ─ ─ greeter. Pb. Go ├ ─ ─ greeter. Pb. Gw go ├ ─ ─ greeter. Swagger. Json └ ─ ─ greeter_grpc. Pb. Go 1 directory, 4 filesCopy the code

6. Create boota.yaml & Servera.go

Server-a listens on port 1949 and sends A request to server-B.

We through RKGRPCCTX InjectSpanToNewContext () method into Context, the Tracing information sent to the Server – B.

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 1949                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    interceptors:
      loggingZap:
        enabled: true
      meta:
        enabled: true
      tracingTelemetry:
        enabled: true
Copy the code

package main import ( "context" "demo/api/gen/v1" "fmt" "github.com/rookie-ninja/rk-boot" "github.com/rookie-ninja/rk-grpc/interceptor/context" "google.golang.org/grpc" ) // Application entrance. func main() { // Create a new boot instance. boot := rkboot.NewBoot(rkboot.WithBootConfigPath("bootA.yaml")) // Get grpc entry with name grpcEntry := boot.GetGrpcEntry("greeter") grpcEntry.AddRegFuncGrpc(registerGreeter) grpcEntry.AddRegFuncGw(greeter.RegisterGreeterHandlerFromEndpoint) // Bootstrap boot.Bootstrap(context.Background()) // Wait for shutdown sig boot.WaitForShutdownSig(context.Background()) } func registerGreeter(server *grpc.Server) { greeter.RegisterGreeterServer(server, &GreeterServer{}) } type GreeterServer struct{} func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) { // Call serverB at 2008 with grpc client opts := []grpc.DialOption{ grpc.WithBlock(), grpc.WithInsecure(), } conn, _ := grpc.Dial("localhost:2008", opts...) defer conn.Close() client := greeter.NewGreeterClient(conn) // Inject current trace information into context newCtx := rkgrpcctx.InjectSpanToNewContext(ctx) client.Greeter(newCtx, &greeter.GreeterRequest{Name: "A"}) return &greeter.GreeterResponse{ Message: fmt.Sprintf("Hello %s!" , request.Name), }, nil }Copy the code

7. Create bootb.yaml & serverB. Go

Server-b listens on port 2008.

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 2008                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    interceptors:
      loggingZap:
        enabled: true
      meta:
        enabled: true
      tracingTelemetry:
        enabled: true
Copy the code

package main import ( "context" "demo/api/gen/v1" "fmt" "github.com/rookie-ninja/rk-boot" "google.golang.org/grpc" ) // Application entrance. func main() { // Create a new boot instance. boot := rkboot.NewBoot(rkboot.WithBootConfigPath("bootB.yaml")) // Get grpc entry with name grpcEntry := boot.GetGrpcEntry("greeter") grpcEntry.AddRegFuncGrpc(registerGreeterB) grpcEntry.AddRegFuncGw(greeter.RegisterGreeterHandlerFromEndpoint) // Bootstrap boot.Bootstrap(context.Background()) // Wait for shutdown sig boot.WaitForShutdownSig(context.Background()) } func registerGreeterB(server *grpc.Server) { greeter.RegisterGreeterServer(server, &GreeterServerB{}) } type GreeterServerB struct{} func (server *GreeterServerB) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) { return &greeter.GreeterResponse{ Message: fmt.Sprintf("Hello %s!" , request.Name), }, nil }Copy the code

8. Folder structure

├ ─ ─ API │ ├ ─ ─ gen │ │ └ ─ ─ v1 │ │ ├ ─ ─ greeter. Pb. Go │ │ ├ ─ ─ greeter. Pb. Gw go │ │ ├ ─ ─ greeter. Swagger. Json │ │ └ ─ ─ Greeter_grpc. Pb. Go │ └ ─ ─ v1 │ ├ ─ ─ greeter. Proto │ └ ─ ─ gw_mapping. Yaml ├ ─ ─ bootA. Yaml ├ ─ ─ bootB. Yaml ├ ─ ─ buf. Gen. Yaml ├ ─ ─ ├── bass exercises ── bass Exercises ── bass ExercisesCopy the code

9. Start ServerA & ServerB

$ go run serverA.go
$ go run serverB.go
Copy the code

10. Send a request to ServerA

Selections curl "localhost: 1949 / API/v1 / greeter? name=rk-dev"Copy the code

11. Verify logs

The logs of the two services have the same traceId but different requestId.

We can trace RPC by grep traceId.

ServerA

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- endTime = 2021-10-20 T00:02:21. 739688 + 08:00... ids={"eventId":"0d145356-998a-4999-ab62-6f1b805274a0","requestId":"0d145356-998a-4999-ab62-6f1b805274a0","traceId":"c36a 45eb076066df39fa407174012369"} ... operation=/api.v1.Greeter/Greeter resCode=OK eventStatus=Ended EOECopy the code

ServerB

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- endTime = 2021-10-20 T00:02:21. 739125 + 08:00... ids={"eventId":"8858a6eb-e953-42ad-bdc3-c466bbbd798e","requestId":"8858a6eb-e953-42ad-bdc3-c466bbbd798e","traceId":"c36a 45eb076066df39fa407174012369"} ... operation=/api.v1.Greeter/Greeter resCode=OK eventStatus=Ended EOECopy the code

concept

When we are not calling chain services such as Jaeger, we want to keep track of RPC requests in a distributed system through logging.

The RK-Boot interceptor writes traceId to the log via the openTelemetry library to trace the RPC.

When a log interceptor, raw data interceptor, or chain interceptor is started, the interceptor writes the following three ids to the log.

EventId

EventId is generated automatically when the log interceptor is started.

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 1949                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    interceptors:
      loggingZap:
        enabled: true
Copy the code

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --... ids={"eventId":"cd617f0c-2d93-45e1-bef0-95c89972530d"} ...Copy the code

RequestId

When the log interceptor and raw data interceptor are started, the RequestId and EventId are automatically generated and the two ids are consistent.

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 1949                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    interceptors:
      loggingZap:
        enabled: true
      meta:
        enabled: true
Copy the code

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --... ids={"eventId":"8226ba9b-424e-4e19-ba63-d37ca69028b3","requestId":"8226ba9b-424e-4e19-ba63-d37ca69028b3"} ...Copy the code

EventId remains consistent even if the user overrides RequestId.

rkgrpcctx.AddHeaderToClient(ctx, rkgrpcctx.RequestIdKey, "overridden-request-id")
Copy the code

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --... ids={"eventId":"overridden-request-id","requestId":"overridden-request-id"} ...Copy the code

TraceId

TraceId is generated automatically when the call chain interceptor is started.

---
grpc:
  - name: greeter                   # Name of grpc entry
    port: 1949                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    interceptors:
      loggingZap:
        enabled: true
      meta:
        enabled: true
      tracingTelemetry:
        enabled: true
Copy the code

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --... ids={"eventId":"dd19cf9a-c7be-486c-b29d-7af777a78ebe","requestId":"dd19cf9a-c7be-486c-b29d-7af777a78ebe","traceId":"316a 7b475ff500a76bfcd6147036951c"} ...Copy the code

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

GRPC: How to implement distributed log tracking?

introduce

The installation

Quick start

1. Create API/v1 / greeter. Proto

2. Create the API/v1 / gw_mapping. Yaml

3. Create buf. Yaml

4. Create buf. Gen. Yaml

5. Compile proto file

6. Create boota.yaml & Servera.go

7. Create bootb.yaml & serverB. Go

8. Folder structure

9. Start ServerA & ServerB

10. Send a request to ServerA

11. Verify logs

concept

EventId

RequestId

TraceId

GRPC: How to implement distributed log tracking?

introduce

The installation

Quick start

1. Create API/v1 / greeter. Proto

2. Create the API/v1 / gw_mapping. Yaml

3. Create buf. Yaml

4. Create buf. Gen. Yaml

5. Compile proto file

6. Create boota.yaml & Servera.go

7. Create bootb.yaml & serverB. Go

8. Folder structure

9. Start ServerA & ServerB

10. Send a request to ServerA

11. Verify logs

concept

EventId

RequestId

TraceId

Related Posts

One AI programmer: GitHub Copilot

Daily data volume tens of millions of levels, MySQL, TiDB two storage solutions landing comparison

DCR: A more democratic successor to Bitcoin