0 x00 the

SOFA is a finance-level distributed middleware independently researched and developed by Ant Financial, which contains various components required for the construction of finance-level cloud native architecture. SOFATracer is a component used for distributed system call tracking.

I have experience with Zipkin before and want to extend it to Opentracing, so I summarized this article based on the source code of SOFATracer’s official blog.

Due to the word limit, this paper is divided into two parts.

0x01 Cause & Problem

1.1 choose

Why did you choose to learn from SOFATracer? The reasoning is simple: with big companies endorsing it (best practices honed in the financial scene), official blogs compiled by developers and communities, live streams, and examples that are easy to debug, why not explore and use them?

1.2 the problem

Let’s use questions to guide our reading.

  • How is spanId generated and what are the rules?
  • How is traceId generated and what are the rules?
  • Where does the client generate the Span?
  • Where does ParentSpan come from?
  • ChildSpan is created by ParentSpan. When was it created?
  • How do Trace messages pass?
  • What does the server do after receiving the request?
  • How is SpanContext handled on the server side?
  • How to collect link information?

1.3 Scope of this article

Full-link tracing is divided into three tracing levels:

  • Cross-process tracing (calling another microservice)
  • Database tracing
  • In-process tracing (tracing within a function)

This article only discusses cross-process, because cross-process tracing is the simplest and easy to use. For cross-process tracing, you can write interceptors or filters to track each request, and it requires very little code.

0x02 Background

2.1 Trends and Challenges

The birth of container and Serverless programming greatly improves the efficiency of software delivery and deployment. During the evolution of the architecture, you can see two changes:

  • The application architecture begins to evolve from monolithic systems to microservices, where the business logic will subsequently become calls and requests between microservices.
  • From the perspective of resources, the traditional server, a physical unit, has gradually become invisible and invisible virtual resources.

The two changes above illustrate the growing complexity of the operational and diagnostic requirements behind this resilient, standardized architecture. How to sort out service dependency calls, how to quickly debug in such an environment, track service processing time, find service performance bottlenecks, and properly evaluate the capacity of the service becomes a tricky task.

2.2 Observability

In response to these problems, the concept of Observability was introduced into software. Traditional monitoring and alarm mainly focus on abnormal conditions and failure factors of the system, while observability focuses more on showing the running status of the system from the system itself, which is more like a self-examination of the system. An observable system is more concerned with the state of the application itself than with indirect evidence such as the machine or network in which it is located. We want direct access to the current throughput and latency of the application, and in order to do that, we need to proactively expose as much of the application as possible. In the current application development environment, our focus on complex systems will gradually shift from point to point, line, surface and body, which will enable us to better understand the system, not only know What, but also answer Why.

Observability currently consists of the following three pillars:

  • Log (Logging) : LoggingIt records discrete events. Applications output logs in well-defined formats to files and use log collectors to collect them for analysis and aggregation. While it is possible to concatenate all log point events over time, it is difficult to show the complete invocation relationship path;
  • Measurement (Metrics) :MetricTend to be some aggregate information compared toLoggingIt loses some concrete information, but takes up much less space than a full log and can be used for monitoring and alerting, where Prometheus has largely become the de facto standard;
  • Distributed tracking (Tracing) : TracingbetweenLogging 和 MetricIn order to keep the necessary information and connect the scattered log events through Span, it helps us better understand the behavior of the system, assist in debugging and troubleshooting performance problems.

The three pillars have the following characteristics:

  • Metric is unique in that it is cumulative. Atomic, each is a logical unit of measurement, or a bar chart over a period of time. For example, the current depth of a queue can be defined as a unit of measurement that is updated as it is written or read; The number of incoming HTTP requests can be defined as a counter for simple accumulation; The execution time of a request can be defined as a bar chart, updated and aggregated over a specified slice of time.
  • Logging is characterized by its ability to describe discrete (discontinuous) events. For example, an application outputs debug or error information in a scrolling file and saves it to Elasticsearch through the log collection system. The approval details are stored in the Kafka database (BigTable). Alternatively, the metadata information for a particular request is stripped from the service request and sent to an exception collection service such as NewRelic.
  • The greatest feature of Tracing is that it processes information within the scope of a single request. Any data, metadata information is bound to a single transaction in the system. For example, an RPC execution that calls a remote service; An actual SQL query; The service ID of an HTTP request.

2.3 Tracing

Distributed tracing, also known as distributed request tracing, is a method for analyzing and monitoring applications, especially those built using microservice architectures; Distributed tracing helps identify where failures are occurring and causes of poor performance, developers can use distributed tracing to help debug and optimize their code, and IT and DevOps teams can use distributed tracing to monitor applications.

The birth of 2.3.1 Tracing

Tracing is a technique that has been around since the 1990s. But what really made the field popular was Google’s paper “Dapper, a Large-scale Distributed Systems Tracing Infrastructure”, Another paper, “Uncertainty in Aggregate Estimates from Sampled Distributed Traces,” contains a more detailed analysis of sampling. After the paper was published, a group of outstanding Tracing software was born.

2.3.2 Tracing features

  • Fault location: You can view the complete path of requests, which is easier to locate problems than discrete logs. (Because the sampling rate is set in the real online environment, you can use the debug switch to achieve full sampling of a specific request.)
  • Dependency carding — Generating service dependency diagrams based on call relationships;
  • Performance analysis and optimization — it is convenient to record and count the time occupation and ratio of different processing units on the system link;
  • Capacity planning and evaluation;
  • Cooperate withLoggingandMetricStrengthen monitoring and alarm.

2.4 OpenTracing

To address the incompatibility of different distributed tracing system apis, OpenTracing emerged. OpenTracing aims to standardize Trace data structures and formats. Its purpose is to:

  • Interoperability of Trace clients developed in different languages. Java/.Net/PHP/Python/NodeJs languages such as the development of the client, as long as you follow OpenTracing standards, can be docking OpenTracing compatible monitoring backend.
  • Tracing monitors back-end interoperability. As long as the OpenTracing standard is followed, enterprises can replace specific Tracing monitoring backends as needed, such as from Zipkin to Jaeger/CAT/Skywalking.

OpenTracing is not a standard. The OpenTracing API provides a standard, vendor-neutral framework that is a highly abstract collection of columns of operations involved in distributed links. This means that if a developer wants to try a different distributed tracking system, the developer can simply change the Tracer configuration without replacing the entire distributed tracking system.

0x03 OpenTracing Data Model

Most of the thought models for distributed tracing systems come from Google’s Dapper paper, and OpenTracing uses similar terminology. There are a few basic concepts that need to be understood in advance:

  • Trace: In a broad sense, a Trace represents the execution of a transaction or process in a (distributed) system. In the OpenTracing standard, a trace is a directed acyclic graph (DAG) composed of multiple spans, each of which represents a continuous execution segment named and timed in the trace.

  • Span: A Span represents a logical operation unit in the system with a start time and execution time. It is a logical operation in the application. A logical causal relationship is established between spans by nesting or ordering them. A SPAN can be understood as a method call, a block call, or an RPC/ database access, as long as a program access has a full time cycle.

  • Logs: Each span can perform multiple Logs operations. For each Logs operation, a timestamp time name is required, and optionally a storage structure of any size.

  • Tags: Each span can have multiple Tags in the form of key: value pairs. Tags are time-stamped and can be easily annotated and supplemented by a span.

  • SpanContext: SpanContext is more of a “concept” than a useful feature of the general OpenTracing layer. SpanContext plays an important role in creating SPANS, injecting into transport protocols, and extracting call chain information from transport protocols.

3.1 Span

Represents a call unit in a distributed call chain, whose boundaries contain a request into the service and out of the current service by some means (HTTP/Dubbo, etc.).

A SPAN typically records some information inside the calling unit, For example, each Span contains an action name, start and end times, A Span Tag to attach additional information to, a Span Log that can be used to record special events within a Span, a SpanContext that can be used to pass a Span context, and References that define relationships between spans.

  • Operation name (An Operation name)
  • Start time (A start timestamp)
  • End time (A Finish timestamp)
  • Label information: Zero or more Span Tags in the form of keys: Values. Key must be string, values can be strings, bool, numeric types
  • Log message: 0 or more Span logs
  • A SpanContext
  • A SpanContext can point to zero or more causally related spans.

3.2 Tracer

Trace describes a “transaction” in a distributed system. A trace is a directed acyclic graph composed of several spans.

Tracer is used to create spans and understand how to inject (serialize) and extract (deserialize) spans across process boundaries. It has the following responsibilities:

  1. Create and open a SPAN
  2. Extract/inject a spanContext from a medium

Traces can be considered spans of DAG from a graphological perspective. In other words, multiple SPANS form a DAG that is a single trace.

For example, here’s a Trace of eight Spans.

A single Trace of the causal relationship between Span in [Span A] please please please (the root Span) | + -- -- -- -- -- - + -- -- -- -- -- -- + | | [Span B] [C] Span please please please (Span C is A ` ChildOf ` [Span Span A) | | D] + - + -- -- -- -- -- -- -- + | | [Span E] [Span F] > > > [Span G] > > > [Span H] write write write (Span G ` FollowsFrom ` Span F)Copy the code

Sometimes it makes more sense to visualize it chronologically. Here’s an example.

A single Trace of the time relationship between Spans in - | -- - | -- - | -- - | -- - | -- - | -- - | -- -- -- - | - > time [Span A. · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·] [Span B · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·] [Span D · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·] [Span C · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·] [Span E · · · · · · ·] [the] Span f. [the] Span g. to [the] Span h.Copy the code

3.3 the References between Spans

A span can have a causal relationship with one or more spans. OpenTracing defines two relationships: ChildOf and FollowsFrom. These two reference types represent a direct causal relationship between child and parent nodes.

ChildOf will become the child of the current Span, while FollowsFrom will become parent. These two relationships establish a direct causal relationship between child span and parent Span.

3.4 SpanContext

Span and spanContext are basically one-to-one correspondence, and the spanContext can be passed down the call chain through some medium and means to do some processing (such as id generation of subspans, inheritance of messages to print logs, and so on).

Context stores some information that needs to be crossed (propagated trace), for example:

  • SpanId: INDICATES the ID of the current span
  • TraceId: The traceId to which the span belongs (that is, the unique ID of the call chain).
    • trace_idandspan_idTo distinguish betweenTraceIn theSpan; Any state associated with the OpenTraceing implementation (such as Trace and SPAN ID) needs to be identified by a cross-process parameterSpanContact.
  • Baggage: Other information that spans multiple calling units, that is, key value pairs across processes.Baggage Items 和 Span TagThe structure is the same, the only difference being:Span TagOnly in the currentSpanExists in, not in the wholetraceIn the transmission, whileBaggage ItemsWill be called by the chain.

A simplified version of the SpanContext data structure looks like this:

SpanContext:
- trace_id: "abc123"
- span_id: "xyz789 - Baggage Items: - special_id: "vsid1738"
Copy the code

The delivery and association of call relationships in transboundary (cross-service or protocol) transport requires the ability to inject SpanContext into and extract SpanContext from downstream transport media.

It is often possible to do this using a mechanism like HTTP Headers provided by the protocol itself, and messaging middleware such as Kafka also has a Headers mechanism that provides this functionality.

The OpenTracing implementation can use the tracer.inject (…) provided in the API. And Tracer. Extract (…). Easy to implement SpanContext injection and extraction.

  • “Extarct ()” gets the trace context from the medium, usually an HTTP header.
  • “Inject ()” puts trace context into the medium to ensure the continuity of trace chain.

3.5 Carrier

Carrier represents a medium that hosts spanContext. For example, there is HttpCarrier in the HTTP call scenario, and there is a DubboCarrier in the dubbo call scenario.

3.6 the Formatter

This interface is responsible for the specific logic of serializing the deserialization context in specific scenarios, such as the usual HttpFormatter for HttpCarrier use. The injection and extraction of Tracer are delegated to Formatter.

3.7 ScopeManager

This class is a new component that was added after version 0.30. The purpose of this component is to get information about the Span enabled in the current thread, as well as enable some of the unenabled spans. In some scenarios, we may create multiple spans in a thread at the same time, but only one span may be enabled on the same thread at a time, and other spans may be in the following states:

  1. Wait for the subspan to complete
  2. Wait for some blocking method
  3. Created but not started

3.8 Reporter

In addition to the above components, in achieving a distributed monitoring framework, all link also needs to have a reporter components, through it to print or to some key link information (such as span to create and end), only the information will be processed before we the full link information visualization and real monitoring.

0x04 SOFATracer

SOFATracer is a component used for distributed system call tracing. It logs all network calls in the invocation link using the unified traceId to achieve the purpose of perspective network calls. These logs can be used for quick fault discovery and service governance.

SOFATracer team has built a complete Tracer framework kernel for us, including data model, encoder, cross-process transparent traceId, sampling, log drop and report and other core mechanisms, and provided extended API and some plug-ins based on open source components. It provides great convenience for us to build our own Tracer platform based on this framework.

SOFATracer does not currently provide data collectors and UI presentation capabilities; There are two main considerations:

  • SOFATracer, as a very lightweight component of SOFA architecture, is intended to log SPAN data to disk so that users can be more flexible with the data
  • In terms of UI display, SOFATracer itself is realized based on the OpenTracing specification, and can achieve seamless docking with some open source products in the model, which can make up for its shortcomings in link visualization to a certain extent.

Therefore, in the reporting model, SOFATracer provides extensions for log output and external reporting, enabling access parties to process reported data in a flexible manner. SOFARPC + SOFATracer + zipKin can quickly build a complete link tracking system, including burial point, collection, analysis and display, etc. Collection and analysis is largely borrowed from zipKin’s capabilities.

SOFATracer currently supports buried support for the following open source components: Spring MVC, RestTemplate, HttpClient, OkHttp3, JDBC, Dubbo(2.6⁄2.7), SOFARPC, Redis, MongoDB, Spring Message, Spring Cloud Stream (Spring Message-based burying point), RocketMQ, Spring Cloud FeignClient, Hystrix.

Opentracing declares all core components as interfaces, such as Tracer, Span, SpanContext, Format (including Scope and ScopeManager in older versions), and so on. SOFATracer uses version 0.22.0, which is mainly the implementation of three conceptual models: Tracer, Span, and SpanContext. The following is an analysis of several components combined with SOFATracer.

4.1 Tracer & SofaTracer

Tracer is a simple, generalized interface that builds and transports SPANS.

SofaTracer implements IO. Opentracing.Tracer interface, and extends sampling and data reporting capabilities.

public class SofaTracer implements Tracer {
    public static final String ROOT_SPAN_ID = "0";
    private final String tracerType;
    private final Reporter clientReporter;
    private final Reporter serverReporter;
    private final Map<String, Object> tracerTags = new ConcurrentHashMap();
    private final Sampler sampler;
}
Copy the code

4.2 Span & SofaTracerSpan

Span is a Span unit. In actual application, Span is a complete packet containing the data to be reported by the current node.

SofaTracerSpan implements the IO. Opentracing.Span interface and extends the ability to handle references, tags, thread asynchronous processing, logTypes required in plug-in extensions, and Tracer types that generate the current Span.

Each span contains two important pieces of information, span ID (the span ID of the current module) and SPAN parent ID (the SPAN ID of the last calling module), which can be used to locate a SPAN in the call chain. This is core information stored in the SpanContext.

public class SofaTracerSpan implements Span {
    public static final char                                ARRAY_SEPARATOR      = '|';
    private final SofaTracer                                sofaTracer;
    private final List<SofaTracerSpanReferenceRelationship> spanReferences;
    /** tags for String */
    private final Map<String, String>                       tagsWithStr          = new LinkedHashMap<>();
    /** tags for Boolean */
    private final Map<String, Boolean>                      tagsWithBool         = new LinkedHashMap<>();
    /** tags for Number */
    private final Map<String, Number>                       tagsWithNumber       = new LinkedHashMap<>();
    private final List<LogData>                             logs                 = new LinkedList<>();
    private String                                          operationName        = StringUtils.EMPTY_STRING;
    private final SofaTracerSpanContext                     sofaTracerSpanContext;
    private long                                            startTime;
    private long                                            endTime              = -1;
}
Copy the code

SOFARPC is divided into ClientSpan and ServerSpan. ClientSpan Records the process of sending a request from a client to a server and receiving the response from the server. ServerSpan is the process between the time when the server receives the response from the client and the time when the server sends the response to the client.

4.3 SpanContext & SofaTracerSpanContext

SpanContext is crucial to the Implementation of OpenTracing. SpanContext can be used to achieve cross-process link passthrough, and the entire link can be connected through the information carried in the SpanContext.

The official documentation reads: “In OpenTracing, we forced the SpanContext instance to be immutable to avoid complex lifecycle issues during the Span finish and Reference operations.” It is understandable that if the SpanContext changes during pass-through, for example by changing the tracerId, the link may break.

SofaTracerSpanContext implements the SpanContext interface, extending new capabilities to build SpanContext, serialize baggageItems, and SpanContext.

public interface SofaTraceContext {
    void push(SofaTracerSpan var1);
    SofaTracerSpan getCurrentSpan(a);
    SofaTracerSpan pop(a);
    int getThreadLocalSpanSize(a);
    void clear(a);
    boolean isEmpty(a);
}
Copy the code

4.3.1 Sending Trace Information

This section answers how Trace information is passed.

In OpenTracing, Trace information is passed through the SpanContext.

SpanContext stores information that needs to cross boundaries, such as trace Id, SPAN Id, Baggage. This information is passed by different components according to their own serialization, such as the serialization into HTTP headers. The current node is then associated to the entire Tracer link using the information carried by the SpanContext.

Simply put, HTTP headers are used as a Carrier to pass traceids. Whether microservices are gRPC or RESTFul, they all use the HTTP protocol. If it is a Message Queue, the traceID is put in the Message header.

The SofaTracerSpanContext class includes and implements “some information that needs to cross boundaries.”

public class SofaTracerSpanContext implements SpanContext {

    //spanId separator
    public static final String        RPC_ID_SEPARATOR       = ".";

    //======= The following is the key for serializing data ========================

    private static final String       TRACE_ID_KET           = "tcid";

    private static final String       SPAN_ID_KET            = "spid";

    private static final String       PARENT_SPAN_ID_KET     = "pspid";

    private static final String       SAMPLE_KET             = "sample";

    /** * The serialization system transparently passes the prefix of the attribute key */
    private static final String       SYS_BAGGAGE_PREFIX_KEY = "_sys_";

    private String                    traceId                = StringUtils.EMPTY_STRING;

    private String                    spanId                 = StringUtils.EMPTY_STRING;

    private String                    parentId               = StringUtils.EMPTY_STRING;

    /** * Default will not be sampled */
    private boolean                   isSampled              = false;

    /** * The system transparently transmits data, * mainly refers to the transparent transmission data of the system dimension. * Note that this field cannot be used for transparent transmission of business. */
    private final Map<String, String> sysBaggage             = new ConcurrentHashMap<String, String>();

    /** * Transparent transmission of data, mainly refers to the transparent transmission data of the business */
    private final Map<String, String> bizBaggage             = new ConcurrentHashMap<String, String>();

    /** * sub-context counter */
    private AtomicInteger             childContextIndex      = new AtomicInteger(0);
}
Copy the code

4.3.2 Thread Storage

In each node of the link, SpanContext is thread-specific and stored in a thread ThreadLocal.

Implementation is SofaTracerThreadLocalTraceContext function. We can see that ThreadLocal is used because the Context is related to the thread Context.

public class SofaTracerThreadLocalTraceContext implements SofaTraceContext {
    private final ThreadLocal<SofaTracerSpan> threadLocal = new ThreadLocal();

    public void push(SofaTracerSpan span) {
        if(span ! =null) {
            this.threadLocal.set(span); }}public SofaTracerSpan getCurrentSpan(a) throws EmptyStackException {
        return this.isEmpty() ? null : (SofaTracerSpan)this.threadLocal.get();
    }

    public SofaTracerSpan pop(a) throws EmptyStackException {
        if (this.isEmpty()) {
            return null;
        } else {
            SofaTracerSpan sofaTracerSpan = (SofaTracerSpan)this.threadLocal.get();
            this.clear();
            returnsofaTracerSpan; }}public int getThreadLocalSpanSize(a) {
        SofaTracerSpan sofaTracerSpan = (SofaTracerSpan)this.threadLocal.get();
        return sofaTracerSpan == null ? 0 : 1;
    }

    public boolean isEmpty(a) {
        SofaTracerSpan sofaTracerSpan = (SofaTracerSpan)this.threadLocal.get();
        return sofaTracerSpan == null;
    }

    public void clear(a) {
        this.threadLocal.remove(); }}Copy the code

4.4 Reporter

The log fall is divided into abstract log fall and statistical log fall;

  • A summary log is a log that lands on disk every time it is called.
  • Statistics logs are generated at regular intervals.

Data reporting is a function implemented by SofaTracer based on OpenTracing Tracer interface extension. The Reporter instance exists as a property of SofaTracer and is initialized when the SofaTracer instance is constructed.

The Reporter interface is designed to provide the ability to retrieve Reporter types in addition to the core reporting function, because SOFATracer’s current buried mechanism solution relies on this implementation.

public interface Reporter {
    String REMOTE_REPORTER = "REMOTE_REPORTER";
    String COMPOSITE_REPORTER = "COMPOSITE_REPORTER";

    // Get Reporter instance type
    String getReporterType(a);
    / / output span
    void report(SofaTracerSpan span);
    // Turn off the ability to output span
    void close(a);
}
Copy the code

Reporter’s implementation class has two, SofaTracerCompositeDigestReporterImpl and DiskReporterImpl:

  • SofaTracerCompositeDigestReporterImpl: Combination log report to realize, when they reported traverse all the Reporter, in the current SofaTracerCompositeDigestReporterImpl transaction by report operation; It can be extended for external users.
  • DiskReporterImpl: Core implementation class of data drop disks and the default reporting class used by SOFATracer.

0x05 Sample code

5.1 RestTemplate

We use the RestTemplate example

import com.sofa.alipay.tracer.plugins.rest.SofaTracerRestTemplateBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.ResponseEntity;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.web.client.AsyncRestTemplate;
import org.springframework.web.client.RestTemplate;

@SpringBootApplication
public class RestTemplateDemoApplication {
    private static Logger logger = LoggerFactory.getLogger(RestTemplateDemoApplication.class);

    public static void main(String[] args) throws Exception {
        SpringApplication.run(RestTemplateDemoApplication.class, args);
        RestTemplate restTemplate = SofaTracerRestTemplateBuilder.buildRestTemplate();
        ResponseEntity<String> responseEntity = restTemplate.getForEntity(
            "http://localhost:8801/rest", String.class);
        logger.info("Response is {}", responseEntity.getBody());

        AsyncRestTemplate asyncRestTemplate = SofaTracerRestTemplateBuilder
            .buildAsyncRestTemplate();
        ListenableFuture<ResponseEntity<String>> forEntity = asyncRestTemplate.getForEntity(
            "http://localhost:8801/asyncrest", String.class);
        //async
        logger.info("Async Response is {}", forEntity.get().getBody());

        logger.info("test finish ......."); }}Copy the code

0 x06 start

Here first to mention the embedded mechanism of SOFATracer, different components have different application scenarios and extension points, so the implementation of the plug-in should also adapt to local conditions, SOFATracer embedded way is generally through the Filter, Interceptor mechanism to achieve. Therefore, the following Client/Server startup is mainly to create a Filter, Interceptor mechanism.

Let’s take a look at the startup of SofaTracer using the RestTemplate as an example.

6.1 Spring SPI

In the code used to SofaTracerRestTemplateBuilder, how can achieve a complete link to track? The original secret was in the POM.xml file.

<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.alipay.sofa</groupId>
            <artifactId>tracer-sofa-boot-starter</artifactId>
        </dependency>
</dependencies>
Copy the code

In the spring.factories file of tracer-SOFA -boot-starter, many classes are defined.

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.alipay.sofa.tracer.boot.configuration.SofaTracerAutoConfiguration,\
com.alipay.sofa.tracer.boot.springmvc.configuration.OpenTracingSpringMvcAutoConfiguration,\
com.alipay.sofa.tracer.boot.zipkin.configuration.ZipkinSofaTracerAutoConfiguration,\
com.alipay.sofa.tracer.boot.datasource.configuration.SofaTracerDataSourceAutoConfiguration,\
com.alipay.sofa.tracer.boot.springcloud.configuration.SofaTracerFeignClientAutoConfiguration,\
com.alipay.sofa.tracer.boot.flexible.configuration.TracerAnnotationConfiguration,\
com.alipay.sofa.tracer.boot.resttemplate.SofaTracerRestTemplateConfiguration
org.springframework.context.ApplicationListener=com.alipay.sofa.tracer.boot.listener.SofaTracerConfigurationListener
Copy the code

Spring Boot has a very decoupled extension mechanism: Spring Factories. This extension mechanism is actually modeled after the SPI extension mechanism in Java.

SPI, whose full name is Service Provider Interface, is a Service discovery mechanism that finds Service implementations for an Interface. Services can be dynamically specified when modules are assembled. It’s kind of like the IOC idea of moving control of assembly out of the program.

Spring Factories are the implementation class names of interfaces configured in meta-INF/spring.Factories, then read and instantiate those configuration files in the program. This custom SPI mechanism is the basis for the Spring Boot Starter implementation.

For the SpringBoot project, after tracer-SOFA -boot-starter is introduced, the Spring program directly reads the classes in the Spring. Factories file of tracer-SOFA -boot-starter and instantiates them. Users will be able to use many of SOFA’s functions directly in the program.

Take Reporter, for example. Will automatically configure class SofaTracerAutoConfiguration all current SpanReportListener save type of bean instance to SpanReportListenerHolder List object. While SpanReportListener type of Bean in ZipkinSofaTracerAutoConfiguration automatically injected into the current in the Ioc container configuration class. When invokeReportListeners are called, they can then retrieve the Zipkin reporting class so that they can be reported.

Support for non SpringBoot application report, is essentially need to instantiate ZipkinSofaTracerSpanRemoteReporter object, and put the object in SpanReportListenerHolder List object. So SOFATracer provides a ZipkinReportRegisterBean in the zipkin plug-in and implements the spring-provided bean lifecycle interface InitializingBean, In ZipkinReportRegisterBean build a ZipkinSofaTracerSpanRemoteReporter instance after initialization, and given to SpanReportListenerHolder class management.

6.2 the Client startup

This is part of the code SofaTracerRestTemplateConfiguration. The main function is to generate a RestTemplateInterceptor.

The purpose of the RestTemplateInterceptor is to process the request first.

First SofaTracerRestTemplateConfiguration role is to generate a SofaTracerRestTemplateEnhance.

@Configuration
@ConditionalOnWebApplication
@ConditionalOnProperty(prefix = "com.alipay.sofa.tracer.resttemplate", value = "enable", matchIfMissing = true)
public class SofaTracerRestTemplateConfiguration {

    @Bean
    public SofaTracerRestTemplateBeanPostProcessor sofaTracerRestTemplateBeanPostProcessor(a) {
        return new SofaTracerRestTemplateBeanPostProcessor(sofaTracerRestTemplateEnhance());
    }

    @Bean
    public SofaTracerRestTemplateEnhance sofaTracerRestTemplateEnhance(a) {
        return newSofaTracerRestTemplateEnhance(); }}Copy the code

Second, SofaTracerRestTemplateEnhance generates a RestTemplateInterceptor, so that you can do before the request processing.

public class SofaTracerRestTemplateEnhance {

    private final RestTemplateInterceptor restTemplateInterceptor;

    public SofaTracerRestTemplateEnhance(a) {
        AbstractTracer restTemplateTracer = SofaTracerRestTemplateBuilder.getRestTemplateTracer();
        this.restTemplateInterceptor = new RestTemplateInterceptor(restTemplateTracer);
    }

    public void enhanceRestTemplateWithSofaTracer(RestTemplate restTemplate) {
        // check interceptor
        if (checkRestTemplateInterceptor(restTemplate)) {
            return;
        }
        List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>(
            restTemplate.getInterceptors());
        interceptors.add(0.this.restTemplateInterceptor);
        restTemplate.setInterceptors(interceptors);
    }

    private boolean checkRestTemplateInterceptor(RestTemplate restTemplate) {
        for (ClientHttpRequestInterceptor interceptor : restTemplate.getInterceptors()) {
            if (interceptor instanceof RestTemplateInterceptor) {
                return true; }}return false; }}Copy the code

6.3 Starting the Server

This is part of the code OpenTracingSpringMvcAutoConfiguration. The main effect is registered SpringMvcSofaTracerFilter. When Spring Filter is used to intercept a Servlet program, it can decide whether to continue the request to the Servlet program and whether to modify the request and response messages.

@Configuration
@EnableConfigurationProperties({ OpenTracingSpringMvcProperties.class, SofaTracerProperties.class })
@ConditionalOnWebApplication
@ConditionalOnProperty(prefix = "com.alipay.sofa.tracer.springmvc", value = "enable", matchIfMissing = true)
@AutoConfigureAfter(SofaTracerAutoConfiguration.class)
public class OpenTracingSpringMvcAutoConfiguration {

    @Autowired
    private OpenTracingSpringMvcProperties openTracingSpringProperties;

    @Configuration
    @ConditionalOnWebApplication(type = ConditionalOnWebApplication.Type.SERVLET)
    public class SpringMvcDelegatingFilterProxyConfiguration {
        @Bean
        public FilterRegistrationBean springMvcDelegatingFilterProxy(a) {
            FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean();
            SpringMvcSofaTracerFilter filter = new SpringMvcSofaTracerFilter();
            filterRegistrationBean.setFilter(filter);
            List<String> urlPatterns = openTracingSpringProperties.getUrlPatterns();
            if (urlPatterns == null || urlPatterns.size() <= 0) {
                filterRegistrationBean.addUrlPatterns("/ *");
            } else {
                filterRegistrationBean.setUrlPatterns(urlPatterns);
            }
            filterRegistrationBean.setName(filter.getFilterName());
            filterRegistrationBean.setAsyncSupported(true);
            filterRegistrationBean.setOrder(openTracingSpringProperties.getFilterOrder());
            returnfilterRegistrationBean; }}@Configuration
    @ConditionalOnWebApplication(type = ConditionalOnWebApplication.Type.REACTIVE)
    public class WebfluxSofaTracerFilterConfiguration {
        @Bean
        @Order(Ordered.HIGHEST_PRECEDENCE + 10)
        public WebFilter webfluxSofaTracerFilter(a) {
            return newWebfluxSofaTracerFilter(); }}}Copy the code

Stay tuned for more information about the burying mechanism and the overall request process.

0xEE Personal information

★★★★ Thoughts on life and technology ★★★★★

Wechat official account: Rosie’s Thoughts

If you want to get a timely news feed of personal articles, or want to see the technical information of personal recommendations, please pay attention.

0 XFF reference

Distributed tracing system — Opentracing

Introduction to Open Distributed Tracing with Jaeger implementation

Description of OpenTracing semantics

Overview of distributed tracking systems and comparison of mainstream open source systems

Skywalking Distributed Tracking and Monitoring: The beginning

Distributed Full link Monitoring – OpenTracing Demo

Opentracing of actual combat

Go microservices full link tracing details

OpenTracing Java Library Tutorial (3) — Passing SpanContext across services

OpenTracing Java Library Tutorial (1) introduction to Trace and Span

Ant gold uniform distributed link tracking component SOFATracer overview | anatomy

Analysis on the principle of transparent link transmission of Ant Financial open source distributed link tracking component SOFATracer and the extended capability of SLF4J MDC

Ant Financial open source distributed link tracking component SOFATracer sampling strategy and source code analysis

Github.com/sofastack-g…

The OpenTracing Semantic Specification

Ant Financial distributed link tracking component SOFATracer data reporting mechanism and source code analysis

Ant Financial open source distributed link tracking component SOFATracer buried point mechanism analysis

Analysis of distributed link component SOFATracer buried point mechanism

【 analyze | SOFARPC framework 】 SOFARPC link of tracking