Author: Zhu Letao, software architect, with many years of Java development and architecture design experience, good at micro services

The author’s blog: https://blog.csdn.net/zlt2000


background

Development problem with the most popular means of screening system is to check the system log, ELK are generally used in the distributed environment to collect the logs, but use the log when complicated with large positioning problem is more troublesome, due to a large number of other users/other threads log output through together also lead to difficult to filter out all of the specified request related log, And logs corresponding to downstream threads/services.

solution

Each request uses a unique identifier to trace all links and is displayed in logs. The original printing mode is not changed (no intrusion code). Add the traceId identifier to the log template of the MDC mechanism of Logback.

MDC (Mapped Diagnostic Context) is a feature provided by LOG4J and LogBack to facilitate logging in multithreaded conditions. MDC can be thought of as a Map bound to the current thread to which you can add key-value pairs. The content contained in MDC can be accessed by code executing in the same thread. Children of the current thread inherit MDC content from their parent thread. To record logs, you only need to obtain the required information from the MDC. The MDC contents are saved by the program at the appropriate time. For a Web application, this data is usually saved at the very beginning of the request being processed.

Plan implementation

Since MDC uses ThreadLocal internally, only the local thread is valid, and values in child threads and downstream service MDC are lost. Therefore, the main difficulty of the scheme is to solve the problem of value transfer, which mainly includes the following parts:

  • How is MDC data in the API gateway passed to downstream services

  • How does the service receive data and continue to pass it on when calling other remote services

  • How does asynchrony (thread pool) pass to child threads

Modifying a Log Template

Logback Configuration file template format Add id %X{traceId}

Gateway Add Filter

Generate a traceId and pass the header to the downstream service

@Componentpublic class TraceFilter extends ZuulFilter {    @Autowired    private TraceProperties traceProperties;    @Override    public String filterType() {        return FilterConstants.PRE_TYPE;    }    @Override    public int filterOrder() {        return FORM_BODY_WRAPPER_FILTER_ORDER - 1;    }    @Override    public boolean shouldFilter() {// Enable the filter according to the configurationreturn traceProperties.getEnable();    }    @Override    public Object run() {// Link tracing ID String traceId = idUtil.fastSimpleUuid (); MDC.put(CommonConstant.LOG_TRACE_ID, traceId); RequestContext ctx = RequestContext.getCurrentContext(); ctx.addZuulRequestHeader(CommonConstant.TRACE_ID_HEADER, traceId);return null;    }}Copy the code

Downstream services add Spring interceptors

Receives and saves the traceId value interceptor

public class TraceInterceptor implements HandlerInterceptor {    @Override    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {        String traceId = request.getHeader(CommonConstant.TRACE_ID_HEADER);        if (StrUtil.isNotEmpty(traceId)) {            MDC.put(CommonConstant.LOG_TRACE_ID, traceId);        }        return true;    }}Copy the code

Register interceptors

public class DefaultWebMvcConfig extends WebMvcConfigurationSupport { @Override protected void AddInterceptors (InterceptorRegistry registry) {// log link tracking InterceptorRegistry. AddInterceptor (new) TraceInterceptor()).addPathPatterns("/ * *");    super.addInterceptors(registry);  }}Copy the code

Added feIGN interceptor for downstream services

Continue passing the traceId value of the current service to the downstream service

public class FeignInterceptorConfig {    @Bean    public RequestInterceptor requestInterceptor() {RequestInterceptor RequestInterceptor = template -> {// Pass logs traceId String traceId = MDC.get(CommonConstant.LOG_TRACE_ID);if(StrUtil.isNotEmpty(traceId)) { template.header(CommonConstant.TRACE_ID_HEADER, traceId); }};return requestInterceptor;    }}Copy the code

Solve parent-child thread passing problem

Thread pools are used primarily for business purposes (asynchronous, parallel processing), and Spring itself has the @async annotation to use thread pools. To solve this problem, there are two steps

Rewrite the LogbackMDCAdapter for logback

Since logback’s MDC implementation internally uses ThreadLocal and cannot transmit child threads, we need to rewrite ali’s TransmittableThreadLocal

TransmittableThreadLocal is Alibaba’s open source InheritableThreadLocal extension that solves the problem of “passing ThreadLocal when using components that cache threads such as thread pools.” If you want the TransmittableThreadLocal to be transmitted between the thread pool and the main thread, use TtlRunnable and TtlCallable.

TtlMDCAdapter class

package org.slf4j; import com.alibaba.ttl.TransmittableThreadLocal; import org.slf4j.spi.MDCAdapter; Public class implements MDCAdapter {/** * private final ThreadLocal<Map<String, String>> copyOnInheritThreadLocal = new TransmittableThreadLocal<>(); private static TtlMDCAdapter mtcMDCAdapter; static { mtcMDCAdapter = new TtlMDCAdapter(); MDC.mdcAdapter = mtcMDCAdapter; } public static MDCAdaptergetInstance() {        return mtcMDCAdapter;    }Copy the code

Other code with ch. Qos. Logback. Classic. Util. LogbackMDCAdapter, instead of just call copyOnInheritThreadLocal variables

The TtlMDCAdapterInitializer class is used to load its own mdcAdapter implementation at startup

public class TtlMDCAdapterInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> { @ Override public void the initialize (ConfigurableApplicationContext applicationContext) {/ / loaded TtlMDCAdapter instance TtlMDCAdapter.getInstance(); }}Copy the code

Extend thread pool implementation

Added TtlRunnable and TtlCallable extensions to implement TTL

public class CustomThreadPoolTaskExecutor extends ThreadPoolTaskExecutor {    @Override    public void execute(Runnable runnable) {        Runnable ttlRunnable = TtlRunnable.get(runnable);        super.execute(ttlRunnable);    }    @Override    public <T> Future<T> submit(Callable<T> task) {        Callable ttlCallable = TtlCallable.get(task);        returnsuper.submit(ttlCallable); } @Override public Future<? > submit(Runnable task) { Runnable ttlRunnable = TtlRunnable.get(task);returnsuper.submit(ttlRunnable); } @Override public ListenableFuture<? > submitListenable(Runnable task) { Runnable ttlRunnable = TtlRunnable.get(task);return super.submitListenable(ttlRunnable);    }    @Override    public <T> ListenableFuture<T> submitListenable(Callable<T> task) {        Callable ttlCallable = TtlCallable.get(task);        return super.submitListenable(ttlCallable);    }}Copy the code

Scenario testing

The test code is as follows

Logs generated by the API gateway

The gateway generated traceId value of 13 d9800c8c7944c78a06ce28c36de670

A log that is printed when a request is made to jump to the file service

The traceId displayed is the same as that displayed on the gateway. The exception scenario is simulated

ELK aggregation log You can use traceId to query logs of the entire link

When the system is abnormal, you can query all the requested logs in the log center based on the traceId value of the exception log

Download the source code

Attached is my open source microservices framework (including the code in this article), welcome to Star

https://gitee.com/zlt2000/microservices-platform