A summary of Jaeger

In microservices, which provide us with modular points, efficient development with low coupling and DevOPS, when multiple service units are requested in a specific business, it is difficult to locate which service is faulty if there are errors or exceptions in the request, so link tracing is needed. You might want to bury points in business system requests, or write logs, but these need to be written in business code, coupled in code, without the extensibility of microservices for easy maintenance lines.

1.1 Jaeger concept

Inspired by Dapper and OpenZipkin, Jaeger is a distributed tracking system released as open source by Uber Technologies. It is used to monitor and diagnose distributed microservices-based systems, including:

  • Distributed context propagation
  • Distributed transaction monitoring
  • Root cause analysis
  • Service dependency analysis performance/latency optimization

1.2 features

1.2.1 High Scalability

The Jaeger backend is designed to have no single point of failure and can be extended to suit business needs. For example, any given Jaeger installation on Uber typically handles billions of spans per day.

1.2.2 Native support for OpenTracing

The Jaeger back end, Web UI, and tool library have been fully designed to support the OpenTracing standard.

  • Representation of traces as directed acyclic graphs (not just trees) by span references

  • Supports strong type span tabs and structured log through luggage

  • Supports a common distributed context propagation mechanism

1.2.3 Multiple Storage Back-end

Jaeger supports two popular open source NoSQL databases as trace storage backends: Cassandra 3.4+ and Elasticsearch 5.x / 6.x / 7.x. Community experiments are underway using other databases, such as ScyllaDB, InfluxDB, Amazon DynamoDB. Jaeger also comes with a simple memory store for testing Settings.

1.2.4 Modern UI

The Jaeger Web UI is implemented in Javascript using popular open source frameworks such as React. Several performance improvements were released in V1.0 to allow the UI to efficiently process large amounts of data and display traces with tens of thousands of spans (for example, we tried traces with 80,000 spans).

1.2.5 Native Cloud deployment

The Jaeger back end is distributed as a collection of Docker images. These binaries support a variety of configuration methods, including command-line options, environment variables, and configuration files in multiple formats (YAML, TOML, etc.). The Kubernetes template and Helm diagram help to deploy it to a Kubernetes cluster.

1.2.6 Observability

By default, all Jaeger backend components expose the Prometheus metric (other metric backends are also supported). Use the structured logging library ZAP to write logs to standard output.

1.2.7 security

Jaeger third-party security audit can be obtained at https://github.com/jaegertracing/security-audits. For a summary of the security mechanisms available in Jaeger, see Issue # 1718.

1.2.8 Backward compatibility with Zipkin

Although we recommend using the OpenTracing API to detect applications and bind to the Jaeger client library to benefit from advanced features not available elsewhere, if your organization has already invested in detection using the Zipkin library, you don’t have to rewrite all the code. Jaeger provides backward compatibility with Zipkin by accepting spans of Zipkin format (Thrift or JSON V1 / v2) over HTTP. Switching from the Zipkin backend is simply a matter of routing traffic from the Zipkin library to the Jaeger backend.

Ii Quick Start

Your application must be checked before it can send trace data to the Jaeger back end. See the “Client Libraries” section for information on how to use the OpenTracing API and how to initialize and configure the Jaeger tracker.

2.1 introduction

Your application must be checked before it can send trace data to the Jaeger back end. See the “Client Libraries” section for information on how to use the OpenTracing API and how to initialize and configure the Jaeger tracker.

2.2 All in One

All-in-one is an executable for fast local testing, with memory storage components that can start Jaeger UI, collector, query and agent. The easiest way to start all-in-one is to use a pre-built image (a single command line) published to DockerHub.

docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \ -p 5775:5775/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 14268:14268 \ -p 9411:9411 \ Jaegertracing /all-in-one:1.14Copy the code

Or run the jaeger-all-in-one(.exe) executable from the binary distribution archives:

jaeger-all-in-one --collector.zipkin.http-port=9411
Copy the code

You can then navigate to http://localhost:16686 to access the Jaeger UI.

Containers need exposed ports

Port Protocol Component Function
5775 UDP agent accept zipkin.thrift over compact thrift protocol (deprecated, used by legacy clients only)
6831 UDP agent accept jaeger.thrift over compact thrift protocol
6832 UDP agent accept jaeger.thrift over binary thrift protocol
5778 HTTP agent serve configs
16686 HTTP query serve frontend
14268 HTTP collector accept jaeger.thrift directly from clients
14250 HTTP collector accept model.proto
9411 HTTP collector Zipkin compatible endpoint (optional)

2.3 Kubernetes and OpenShift

  • Kubernetes templates: github.com/jaegertraci…
  • Kubernetes Operator: github.com/jaegertraci…
  • OpenShift templates: github.com/jaegertraci…

2.4 Example APP: HotROD

HotROD (On-demand Rides) is a demo application that consists of several microservices and illustrates the use of the OpenTracing API. A tutorial/walkthrough is provided in the blog post: OpenTracing API. The way to HotROD. It can run independently, but requires the Jaeger back end to view the trace. Take OpenTracing for a HotROD ride

Against 2.4.1 features

  • Discover the architecture of the entire system through data-driven dependency diagrams
  • View request schedules and errors; Understand how your application works.
  • Find the source of delays and lack of concurrency.
  • High level of cultural logging.
  • To transmit by means of luggage:
    • Diagnose contention (queuing) between requests.
    • Attribute time spent by the service
  • Integrate open source libraries with OpenTracing to use freely available vendor-neutral tools.

2.4.2 Prerequisites

  • You need to have Go 1.11 or higher installed on your computer to run from source code
  • You need a running Jaeger back end to see the trace

2.4.3 run

2.4.3.1 source

mkdir -p $GOPATH/src/github.com/jaegertracing
cd $GOPATH/src/github.com/jaegertracing
git clone [email protected]:jaegertracing/jaeger.git jaeger
cd jaeger
make install
go run ./examples/hotrod/main.go all
Copy the code

2.4.3.2 docker

docker run --rm -it \
  --link jaeger \
  -p8080-8083:8080-8083 \
  -e JAEGER_AGENT_HOST="jaeger"1.14 \ \ jaegertracing/example - hotrod: allCopy the code

2.4.3.3 binary

Run example-hotrod(.exe) executable from the binary distribution archives:

$ example-hotrod all
Copy the code

Then navigate to http://localhost:8080.

2.5 Migration from Zipkin

The collector service exposes the Zipkin-compliant REST API/API/v1 / SPANS, which accepts Thrift and JSON. Also, / API/v2 / SPANS are used for JSON and Proto. By default, it is disabled. You can enable it with –collector.zipkin.http-port = 9411

Zipkin Thrift IDL and Zipkin Proto IDL files can be found in jaegertracing/jaeger-idl repository. They’re compatible with openzipkin/zipkin-api Thrift and Proto.

Three architecture

3.1 the term

3.1.1 Span

Span represents the logical unit of work in Jaeger and has the name of the operation, the start time of the operation, and the duration of the operation. Spans can be nested and ordered to build causal models.

Each Span contains the following objects:

  • Operation name: Indicates the Operation name (also known as Span name).
  • Start timestamp: indicates the Start time.
  • Finish timestamp: Indicates the end time.
  • Span tag: A collection of Span tags formed by a set of key-value pairs. In key-value pairs, the key must be String and the value can be String, Boolean, or numeric.
  • Span log: a collection of Span logs. Each Log operation contains a key-value pair and a timestamp. In a key-value pair, the key must be String and the value can be of any type.
  • SpanContext: Pan context object. Each SpanContext contains the following states:
    • To implement any OpenTracing, you need to rely on a unique Span to transfer the state of the current call chain across process boundaries (for example: Trace and Span ID).
    • Baggage Items is the Trace’s accompanying data. It is a collection of key-value pairs that exist in Trace and also need to be transmitted across process boundaries.
  • References (Relationships between Spans) : Zero or more spans are related (relationships between spans are established using a SpanContext).

3.1.2 Trace

Tracing is a data/execution path through the system, which can be viewed as a directed acyclic graph of span.

3.2 components

Jaeger can be deployed as an all-in-one binary, where all Jaeger back-end components run in a single process, or as a scalable distributed system, as described below. There are two main deployment options:

  • The collector is writing directly to the storage.

  • The collector is writing to Kafka as a preliminary buffer.

This section details the components of Jaeger and their relationships. It is arranged by the order in which your application interacts with it.

3.2.1 Jaeger client libraries

The Jaeger client is a language-specific implementation of the OpenTracing API. They can be used to detect applications for distributed tracing either manually or through various existing open source frameworks integrated with OpenTracing (e.g. Flask, Dropwizard, gRPC, etc.).

The detection service creates the span when it receives a new request and appends context information (trace ID, span ID, and baggage) to the outgoing request. Only ID and luggage are propagated with the request; All the other information that makes up the span, such as operation names, logs, and so on, is not propagated. Instead, the span of samples is transferred asynchronously in the background outside the process of Jaeger Agents.

The instrument has minimal overhead and is designed to always be enabled in production. Note that while all traces are generated, only some are sampled. Sampling the trace marks the trace for further processing and storage. By default, the Jaeger client samples 0.1% of traces (1 out of 1000) and is able to retrieve the sampling policy from the broker.

3.2.2 Agent

The Jaeger agent is a network daemon that listens for spans sent over UDP and then sends them in batches to the collector. It is intended to be deployed to all hosts as an infrastructure component. The agent abstracts the routing and discovery of the collector away from the client.

3.2.3 the Collector

The Jaeger collector receives traces from the Jaeger agent and runs them through the processing pipeline. Currently, our pipe validates traces, indexes them, performs any transformations, and eventually stores them. Jaeger’s storage devices are pluggable components that currently support Cassandra, Elasticsearch and Kafka.

3.2.4 Query

A query is a service that retrieves traces from storage and hosts the UI to display the trace.

3.2.5 Ingester

Ingester is a service that reads from Kafka topic and writes to another storage backend (Cassandra, Elasticsearch).

3.3 from

The Jaeger library implements consistent upfront (or head-based) sampling. For example, suppose we have A simple call diagram where service A calls service B and service B calls service C: A-> B->C. When service A receives A request that does not contain trace information, the Jaeger tracker starts A new trace, assigns it A random trace ID, and then makes A sampling decision based on the currently installed sampling policy. The sampling decision will be propagated along with the request to B and C, so these services will not make the sampling decision again, but will respect the decision made by top-level service A. This approach ensures that if a trace is sampled, all its spans will be recorded on the back end. If each service makes its own sampling decisions, we rarely get a complete trace on the back end.

3.3.1 Client Sampling Configuration

When using configuration objects to instantiate trackers, you can select the sample type through the sampler.type and sampler.param properties. The Jaeger library supports the following samplers:

  • Constant (sampler.type=const) sampler always makes the same decision for all traces. It either samples all traces (sampler.param=1) or none of them (sampler.param=0).
  • Probabilistic (sampler.type=probabilistic) sampler makes a random sampling decision with the probability of sampling equal to the value of sampler.param property. For example, with Sampler. Param = 0.1 approximately 1 in 10 traces will be sampled.
  • Rate Limiting (sampler.type=ratelimiting) sampler uses a leaky bucket rate limiter to ensure that traces are sampled with a certain constant rate. For example, when Sampler. Param = 2.0 it will sample requests with the rate of 2 traces per second.
  • Remote (sampler.type=remote. which is also the default) sampler consults Jaeger agent for the appropriate sampling strategy to use in the current service. This allows controlling the sampling strategies in the services from acentral configuration in Jaeger backend, or even dynamically (see Adaptive Sampling).

3.3.2 Adaptation Example

The adaptive sampler is a composite sampler that combines two functions:

  • It makes sampling decisions on a per-operation basis, that is, based on the span operation name. This is especially useful in API services where the traffic of endpoints can be very different, and using a single probabilistic sampler for the entire service may starve some low QPS endpoints (never sampling).
  • It supports minimum guaranteed sampling rates, such as always allowing a maximum of N traces per second, and then sampling all sampling rates with probability (all for each operation, not each service).

Each operation parameter can be configured statically or extracted periodically from the Jaeger back end with the help of a remote sampler. The adaptive sampler is intended to work with the upcoming adaptive sampling feature on the Jaeger backend.

3.3.3 Collection Example Configuration

The collector can be instantiated using a static sampling strategy (propagated to the corresponding service if the Remote Sampler configuration is used) with the –sampling.strategies-file option. This option requires a JSON file path with the sampling policy defined.

{
  "service_strategies": [{"service": "foo"."type": "probabilistic"."param": 0.8."operation_strategies": [{"operation": "op1"."type": "probabilistic"."param": 0.2}, {"operation": "op2"."type": "probabilistic"."param": 0.4}]}, {"service": "bar"."type": "ratelimiting"."param": 5}],"default_strategy": {
    "type": "probabilistic"."param": 0.5}}Copy the code

Four Sample tests

Visit: http://139.9.163.110:8080/

For code examples, see an example from a Go bighead: github.com/xinliangnot…

Refer to the link

  • Github.com/jaegertraci…
  • jaegertracing.io/docs/
  • Github.com/jaegertraci…