Go-kit microservices: Service link tracking

Link to track

Modern Internet services are usually implemented using complex, large-scale distributed systems. These applications tend to be built from a large number of software modules that may be developed by different teams, may use different programming languages, and can span thousands of machines across multiple physical facilities. In this environment, tools to help understand system behavior and reasoning about performance problems are invaluable.

Microservice architecture is a distributed architecture. In actual development, we divide service units according to business requirements. A system usually consists of multiple business units. In this scenario, a request may be processed by multiple business units to complete the response, making it difficult to locate errors or exceptions if they occur.

To solve this problem, Google opened source Drapper, a Distributed link Tracing component, and published the paper “Dapper, a Large-scale Distributed Systems Tracing Infrastructure” to introduce the design idea of Drapper. Under the influence of this paper, Twitter designed, developed and opened source Zipkin, a distributed link tracking system.

Zipkin

Zipkin is a distributed tracking system that helps collect time data to address latency issues in microservice architectures. It also provides the function of collecting and querying the time data of distributed system. The architecture of Zipkin is shown below:

According to the architecture diagram, Zipkin consists of four components: Collector, Storage, API and UI. Reporter is provided and collected by the application system. Its working principle is as follows:

Embed a Tracer in the application, which uses a Span to record the timing and metadata information of application actions;
Reporter sends Span to Zipkin’s data Collector;
Collector Stores data to the database through the Storage component.
UI component queries and displays tracking data through API interface;

Zipkin traces a request through a Trace structure. A request is processed by several services. Each service generates a Span. The main data model for Span is as follows:

field	type	instructions
traceId	string	Randomly generated to uniquely identify a trace that all spans contain.
name	string	The span name, which can be named using method, is displayed in the UI component
parentId	string	The number of the parent span, if empty, represents the root span
id	string	The serial number of span
timestamp	integer	Span creation time
duration	integer	Span duration
annotations	Annotation	Associate an event with a timestamp to interpret delay information
tags	Tags	Span tag for search, display, and analysis

Zipkin already offers support for common languages such as C#, Go, Java, JavaScript, Ruby, Scala, PHP, as well as Python, C/C++, Lua, and more.

We practice

Step-0: Preparation

This paper will continue the “Go-Kit Microservice series” and use Go-Kit to integrate Zipkin to realize link tracking of arithmetic operation service, which will include two parts:

In the gatewaygatewayTo add link tracing collection logic, and add tracing Settings in reverse proxy.
Add link tracing and collection logic in the transport and Endpoint layers of the arithmetic operation service.

Go Kit adds Zipkin support by default in the Tracing package, so integration should be easier. To start, you need to download the following dependencies:

# Zipkin official library
go get github.com/openzipkin/zipkin-go

The following three packages are dependent, downloadable on demand
git clone https://github.com/googleapis/googleapis.git [your GOPATH]/src/google.golang.org/genproto

git clone https://github.com/grpc/grpc-go.git [your GOPATH]/src/google.golang.org/grpc

git clone https://github.com/golang/text.git [your GOPATH]/src/golang.org/text
Copy the code

This walkthrough uses the gateway and register services in arithmetic_consul_demo. Copy the directory and rename it arithmetic_trace_demo. Delete Discover.

Step-1: Docker starts Zipkin

Open the filedocker/docker-compose.yml, add Zipkin configuration information on consul (using official recommendations)openzipkin/zipkin), the final content is as follows:

version: '2'

services:
  consul:
    image: progrium/consul:latest
    ports:
      - 8400:8400
      - 8500:8500
      - 8600:53/udp
    hostname: consulserver
    command: -server -bootstrap -ui-dir /ui

  zipkin:
    image: openzipkin/zipkin
    ports:
      - 9411:9411
Copy the code

Open the terminal and switch todockerDirectory, run the following command to start Consul and Zipkin.

sudo docker-compose up
Copy the code

After the startup is successful, open the browser and enterhttp://localhost:9411Check whether the startup is successful.

Step-2: Modifies the gateway

The gateway will be the first and last stop of link tracing. We need to intercept all requests to the gateway and record the tracing information. The Gateway acts as both a server for external requests and a client for arithmetic services (a reverse proxy internal implementation).

Creating a tracker

In conjunction with Zipkin’s architecture diagram and the need to integrate Reporter components into the application, we used the official go package. The code looks like this (the default zipkin URL is set) :

Var (// Consul environment variable omitted zipkinURL = flag.string ("zipkin.url"."http://192.168.192.146:9411/api/v2/spans"."Zipkin server url")
	)
flag.Parse()

var zipkinTracer *zipkin.Tracer
{
	var (
		err           error
		hostPort      = "localhost:9090"
		serviceName   = "gateway-service"
		useNoopTracer = (*zipkinURL == "")
		reporter      = zipkinhttp.NewReporter(*zipkinURL)
	)
	defer reporter.Close()
	zEP, _ := zipkin.NewEndpoint(serviceName, hostPort)
	zipkinTracer, err = zipkin.NewTracer(
		reporter, zipkin.WithLocalEndpoint(zEP), zipkin.WithNoopTracer(useNoopTracer),
	)
	iferr ! = nil { logger.Log("err", err)
		os.Exit(1)
	}
	if! useNoopTracer { logger.Log("tracer"."Zipkin"."type"."Native"."URL", *zipkinURL)
	}
}
Copy the code

Added link tracing for all requests

The middleware/ HTTP package from Zipkin-Go encapsulates our HTTP. Handler in decorator mode and starts listening, as shown below:

// Create a proxy := NewReverseProxy(consulClient, zipkinTracer, logger) tags := map[string]string{"component": "gateway_server",
}

handler := zipkinhttpsvr.NewServerMiddleware(
	zipkinTracer,
	zipkinhttpsvr.SpanName("gateway"),
	zipkinhttpsvr.TagResponseSize(true),
	zipkinhttpsvr.ServerTags(tags),
)(proxy)
Copy the code

Reverse Proxy Settings

After receiving the request, the Gateway creates a SPAN with the traceId as the unique number of the request. The Gateway must “tell” the traceId to the arithmetic service so that the arithmetic service can keep track of the request.

The only thing that can do this in ReverseProxy is Transport. We can replace the system default HTTP. DefaultTransport with NewTransport provided by Zipkin-Go’s Middleware/HTTP package. The code looks like this:

Func NewReverseProxy(client *api. client, zikkinTracer * zipkin.tracer, Logger log.logger) *httputil.ReverseProxy {// create Director Director := func(req * http.request) {// omit} // add trace logic to ReverseProxy, Use the following RoundTrip instead of the default Transport RoundTrip, _ : = zipkinhttpsvr. NewTransport (zikkinTracer, zipkinhttpsvr. TransportTrace (true))

	return &httputil.ReverseProxy{
		Director:  director,
		Transport: roundTrip,
	}
}
Copy the code

This step is crucial! If this parameter is not set, link tracing will be incomplete. In order to solve this problem, it took a long time, but finally passedzipkin-gotheREADMESolved the puzzle.

Complete the above process, you are ready to compile and run.

Step-3: Modifies the arithmetic service

Creating a tracker

This step is no longer described as the Gateway handles it.

Tracking the Endpoint

Go-kit provides the encapsulation of Zipkin-Go, and you can directly call the middleware TraceEndpoint to set the two endpoints of the arithmetic operation service. The code is as follows:

The endpoint endpoint: = MakeArithmeticEndpoint (SVC) = NewTokenBucketLimitterWithBuildIn (ratebucket) (the endpoint) / / add a tracking, Set the name of span to calculate-endpoint = kitzipkin.traceendpoint (zipkinTracer,"calculate-endpoint")(endpoint) // Create health check endpoint healthEndpoint := MakeHealthCheckEndpoint(SVC) healthEndpoint = NewTokenBucketLimitterWithBuildIn (ratebucket) (healthEndpoint) / / add a tracking, Set the span name to health-endpoint healthEndpoint = kitzipkin.TraceEndpoint(zipkinTracer,"health-endpoint")(healthEndpoint)
Copy the code

Track Transport

Modify thetransports.gotheMakeHttpHandlerMethods. Increase the parameterzipkinTracerAnd then set trace parameters in ServerOption. The code is as follows:

// MakeHttpHandler make http handler use mux
func MakeHttpHandler(ctx context.Context, endpoints ArithmeticEndpoints, zipkinTracer *gozipkin.Tracer, logger log.Logger) http.Handler {
	r := mux.NewRouter()

	zipkinServer := zipkin.HTTPServerTrace(zipkinTracer, zipkin.Name("http-transport")) options := []kithttp.ServerOption{ kithttp.ServerErrorLogger(logger), Kithttp. ServerErrorEncoder (kithttp DefaultErrorEncoder), zipkinServer,} / / omit codereturn r
}
Copy the code

inmain.goIn the callMakeHttpHandler.

// create http.Handler r := MakeHttpHandler(CTX, endpts, zipkinTracer, logger)Copy the code

Now that all the code changes are complete, the next step is to start testing.

Step-4: Run & tests

Verify that Consul, Zipkin, Gateway, and Register services are up and running, and then use Postman to test the request (like before, you can double check your data for convenience).

Open http://localhost:9411 in your browser and click the “Find Traces” button to see the following interface. Details are displayed about the time each request was executed, the number of spans, the service name of the path, and so on.

Open the first request and enter the link tracing interface of the request, as shown in the figure below.

The top part of the command displays:, The execution time of the request is 10.970 ms, the path is 2 services, the link depth is 3, and the number of spans is 3.
Display in the lower part: Span is displayed in a tree form, and information about the path service, execution time, and span name of each span is displayed visually.

On this page, you can know that the time-consuming link in the request link is gateway-service. The reason: Every time a request comes in, the application will query Consul for a service instance and dynamically create a service address.

In addition, click each span in the tree structure to view the description of the span. The description is not expanded here.

conclusion

This article uses the Tracing component of Go-Kit and the Zipkin-Go package to add link tracing functions to gateway services and arithmetic operations services, and demonstrates the integration of Zipkin in Go-Kit with examples. The example is relatively simple, hope useful to you!

In this paper, the reference

This article source @github
Zipkin starts fast
Zipkin’s official website
Drapper paper
zipkin-go

This article is first published in my wechat public account [Xi Yi Ang bar], welcome to scan code attention!

Go-kit microservices: Service link tracking

Link to track

Zipkin

We practice

Step-0: Preparation

Step-1: Docker starts Zipkin

Step-2: Modifies the gateway

Creating a tracker

Added link tracing for all requests

Reverse Proxy Settings

Step-3: Modifies the arithmetic service

Creating a tracker

Tracking the Endpoint

Track Transport

Step-4: Run & tests

conclusion

In this paper, the reference

Related Posts

Talk about Map | Go Theme month in Go

Dubbo log adaptation

Java multi-threaded environment synchronization mechanism