SpringBoot logs HTTP requests
1. Demand interpretation
Requirements:
The framework needs to record the information of each HTTP request, including the request path, request parameters, response status, return parameters, request time and other information.
Demand interpretation:
The Springboot framework provides a variety of ways to intercept HTTP requests and responses. As long as the corresponding request and response can be obtained, the required information can be obtained through the corresponding API.
It should be noted that request parameters can be divided into two parts, one is the GET request, request parameters are sent to the back end through URL stitching, and the other is the POST request to submit Json format parameters, which will be sent to the back end in the request body. This is not available via Request. getParameterMap.
2, Spring Boot Actuator
2.1 introduction and Use
The key feature of Spring Boot Actuator is that it provides many Web interfaces in the application, through which it can understand the internal conditions of the application while running, and monitor and measure Spring Boot applications.
To use Spring Boot Actuators, you need to import a dependency package
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Copy the codeThen, you need to enable the port access permission
management.endpoints.web.exposure.include=httptrace
Copy the codeWhen the Spring Boot application is started, the following information is displayed on the console, indicating that the port access is enabled
A browser visiting /acutator/ httpTrace will see the HTTP request
2.2. Default HttpTraceRepository
Spring Boot physical default to 100 times recently HTTP request record into memory, the corresponding implementation class is InMemoryHttpTraceRepository
public class InMemoryHttpTraceRepository implements HttpTraceRepository {
private int capacity = 100;
private boolean reverse = true;
private final List<HttpTrace> traces = new LinkedList<>();
/**
* Flag to say that the repository lists traces in reverse order.
* @param reverse flag value (default true)
*/
public void setReverse(boolean reverse) {
synchronized (this.traces) {
this.reverse = reverse; }}/**
* Set the capacity of the in-memory repository.
* @param capacity the capacity
*/
public void setCapacity(int capacity) {
synchronized (this.traces) {
this.capacity = capacity; }}@Override
public List<HttpTrace> findAll(a) {
synchronized (this.traces) {
return Collections.unmodifiableList(new ArrayList<>(this.traces)); }}@Override
public void add(HttpTrace trace) {
synchronized (this.traces) {
while (this.traces.size() >= this.capacity) {
this.traces.remove(this.reverse ? this.capacity - 1 : 0);
}
if (this.reverse) {
this.traces.add(0, trace);
}
else {
this.traces.add(trace); }}}}Copy the codeSynchronized is used in the add method here. By default, only the latest 100 entries are stored. If the number of concurrent entries is large, the performance will be affected
2.3. Custom HttpTraceRepository
We can implement our own HttpTraceRepository interface, override the Add method and trace logs
@Slf4j
public class RemoteHttpTraceRepository implements HttpTraceRepository {
@Override
public List<HttpTrace> findAll(a) {
return Collections.emptyList();
}
@Override
public void add(HttpTrace trace) {
String path = trace.getRequest().getUri().getPath();
String queryPara = trace.getRequest().getUri().getQuery();
String queryParaRaw = trace.getRequest().getUri().getRawQuery();
String method = trace.getRequest().getMethod();
long timeTaken = trace.getTimeTaken();
String time = trace.getTimestamp().toString();
log.info("path: {}, queryPara: {}, queryParaRaw: {}, timeTaken: {}, time: {}, method: {}", path, queryPara, queryParaRaw, timeTaken, time, method); }}Copy the codeRegister the implementation class with Spring’s container
@Configuration
@ConditionalOnWebApplication
@ConditionalOnProperty(prefix = "management.trace.http", name = "enabled", matchIfMissing = true)
@EnableConfigurationProperties(HttpTraceProperties.class)
@AutoConfigureBefore(HttpTraceAutoConfiguration.class)
public class TraceConfig {
@Bean
@ConditionalOnMissingBean(HttpTraceRepository.class)
public RemoteHttpTraceRepository traceRepository(a) {
return newRemoteHttpTraceRepository(); }}Copy the code2.4 and disadvantages
Currently, this implementation can record the request path, request time, response status, request Header, response Header, etc. ** has no way to record request and response parameters. ** Someone raised an issue on Github, and the author replied that the design was designed to be compatible with Spring MVC and WebFlux modes. For details, please refer to github.com/spring-proj…
3. Spring Boot Filter
3.1, HttpTraceFilter
Since httptrace can’t meet the needs of existing, we can follow the InMemoryHttpTraceRepository this default implementation up find, see who call the implementation class. It turns out that the HttpTraceFilter interceptor (in servlet mode) is called.
public class HttpTraceFilter extends OncePerRequestFilter implements Ordered {
// Not LOWEST_PRECEDENCE, but near the end, so it has a good chance of catching all
// enriched headers, but users can add stuff after this if they want to
private int order = Ordered.LOWEST_PRECEDENCE - 10;
private final HttpTraceRepository repository;
private final HttpExchangeTracer tracer;
/**
* Create a new {@link HttpTraceFilter} instance.
* @param repository the trace repository
* @param tracer used to trace exchanges
*/
public HttpTraceFilter(HttpTraceRepository repository, HttpExchangeTracer tracer) {
this.repository = repository;
this.tracer = tracer;
}
@Override
public int getOrder(a) {
return this.order;
}
public void setOrder(int order) {
this.order = order;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
if(! isRequestValid(request)) { filterChain.doFilter(request, response);return;
}
TraceableHttpServletRequest traceableRequest = new TraceableHttpServletRequest(
request);
HttpTrace trace = this.tracer.receivedRequest(traceableRequest);
int status = HttpStatus.INTERNAL_SERVER_ERROR.value();
try {
filterChain.doFilter(request, response);
status = response.getStatus();
}
finally {
TraceableHttpServletResponse traceableResponse = newTraceableHttpServletResponse( (status ! = response.getStatus()) ?new CustomStatusResponseWrapper(response, status)
: response);
this.tracer.sendingResponse(trace, traceableResponse,
request::getUserPrincipal, () -> getSessionId(request));
this.repository.add(trace); }}... Omit some code}Copy the codeThe HTTP request time is recorded in tracer
3.2. Customize HttpTraceFilter to get request parameters
HttpTraceFilter inherits OncePerRequestFilter. We can copy this filter and define our own filter to inherit OncePerRequestFilter. Get HttpServletRequest, HttpServletResponse in the doFilterInternal method, and then get the corresponding request parameters and return parameters.
Parameters of the GET request can be obtained in the following ways:
String parameterMap = request.getParameterMap()
Copy the codeThe POST request puts parameters into the request body, which is retrieved as follows:
String requestBody = IOUtils.toString(request.getInputStream(), Charsets.UTF_8);
Copy the codeUnfortunately, code execution throws an exception
The reason is that the body character is transferred through the getInputStream() byte stream in HttpServletRequest; The byte stream ceases to exist after it has been read once.
Solution: use ContentCachingRequestWrapper request packet layer on it, the class will a copy of an inputstream to own a byte array, this is not an error. After reading the body, you need to call
wrappedResponse.copyBodyToResponse();
Copy the codeRestore the request.
3.3, complete custom HttpTraceFilter
@Slf4j
public class HttpTraceLogFilter extends OncePerRequestFilter implements Ordered {
private static final String NEED_TRACE_PATH_PREFIX = "/api";
private static final String IGNORE_CONTENT_TYPE = "multipart/form-data";
private final MeterRegistry registry;
public HttpTraceLogFilter(MeterRegistry registry) {
this.registry = registry;
}
@Override
public int getOrder(a) {
return Ordered.LOWEST_PRECEDENCE - 10;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
if(! isRequestValid(request)) { filterChain.doFilter(request, response);return;
}
if(! (requestinstanceof ContentCachingRequestWrapper)) {
request = new ContentCachingRequestWrapper(request);
}
if(! (responseinstanceof ContentCachingResponseWrapper)) {
response = new ContentCachingResponseWrapper(response);
}
int status = HttpStatus.INTERNAL_SERVER_ERROR.value();
long startTime = System.currentTimeMillis();
try {
filterChain.doFilter(request, response);
status = response.getStatus();
} finally {
String path = request.getRequestURI();
if(path.startsWith(NEED_TRACE_PATH_PREFIX) && ! Objects.equals(IGNORE_CONTENT_TYPE, request.getContentType())) { String requestBody = IOUtils.toString(request.getInputStream(), Charsets.UTF_8); log.info(requestBody);//1. Record logs
HttpTraceLog traceLog = new HttpTraceLog();
traceLog.setPath(path);
traceLog.setMethod(request.getMethod());
long latency = System.currentTimeMillis() - startTime;
traceLog.setTimeTaken(latency);
traceLog.setTime(LocalDateTime.now().toString());
traceLog.setParameterMap(JsonMapper.INSTANCE.toJson(request.getParameterMap()));
traceLog.setStatus(status);
traceLog.setRequestBody(getRequestBody(request));
traceLog.setResponseBody(getResponseBody(response));
log.info("Http trace log: {}", JsonMapper.INSTANCE.toJson(traceLog)); } updateResponse(response); }}private boolean isRequestValid(HttpServletRequest request) {
try {
new URI(request.getRequestURL().toString());
return true;
} catch (URISyntaxException ex) {
return false; }}private String getRequestBody(HttpServletRequest request) {
String requestBody = "";
ContentCachingRequestWrapper wrapper = WebUtils.getNativeRequest(request, ContentCachingRequestWrapper.class);
if(wrapper ! =null) {
try {
requestBody = IOUtils.toString(wrapper.getContentAsByteArray(), wrapper.getCharacterEncoding());
} catch (IOException e) {
// NOOP}}return requestBody;
}
private String getResponseBody(HttpServletResponse response) {
String responseBody = "";
ContentCachingResponseWrapper wrapper = WebUtils.getNativeResponse(response, ContentCachingResponseWrapper.class);
if(wrapper ! =null) {
try {
responseBody = IOUtils.toString(wrapper.getContentAsByteArray(), wrapper.getCharacterEncoding());
} catch (IOException e) {
// NOOP}}return responseBody;
}
private void updateResponse(HttpServletResponse response) throws IOException {
ContentCachingResponseWrapper responseWrapper = WebUtils.getNativeResponse(response, ContentCachingResponseWrapper.class);
Objects.requireNonNull(responseWrapper).copyBodyToResponse();
}
@Data
private static class HttpTraceLog {
private String path;
private String parameterMap;
private String method;
private Long timeTaken;
private String time;
private Integer status;
private String requestBody;
privateString responseBody; }}Copy the code@Configuration
@ConditionalOnWebApplication
public class HttpTraceConfiguration {
@ConditionalOnWebApplication(type = Type.SERVLET)
static class ServletTraceFilterConfiguration {
@Bean
public HttpTraceLogFilter httpTraceLogFilter(MeterRegistry registry) {
returnnew HttpTraceLogFilter(registry); }}}Copy the code4, Spring AOP
The Spring AOP approach requires a custom annotation, which is required on every Controller method for interception and is mandatory for business code to be written, so this approach is not adopted.