Introduction to the

The article is divided into four parts

  1. The first part will briefly introduce the data of some real projects, including the time consuming data before and after optimization, so as to let friends know how much room there is for integration test optimization and highlight why integration test optimization is needed.
  2. In Part 2, we will introduce some examples of how integration test execution time is structured so that you can see where your time is being spent.
  3. In part 3, I will break through the source code of the Spring testing framework and introduce you to the principles of Spring integration testing.
  4. Finally, I’ll give some useful suggestions for optimizing the execution time of Spring integration tests.

Integration testing

Just to be brief, we have been pursuing automated testing for a long time. The fundamental reason is that automated testing is more reliable and cheaper than people, and integrated testing is one of the components of automated testing.

Doing integration testing well in a team can give the team confidence, for example

  • Integration tests cover more areas than unit tests cover
  • You can be more comfortable refactoring boldly

In order to do integration testing well, we also need to pay some costs, such as

  • Integration tests execute slowly (compared to unit tests)
  • Construct test data to “cry”…

Why reduce test execution time?

Gradle generates an execution report for each Gradle task in reverse chronological order.

The data are all in the chart. You can take a look and ask yourself if the time is acceptable.

001 code base: 650+ total tests, including unit tests and integration tests

002 code base: Total number of tests 1250+, including unit tests and integration tests

Some of you might think, that’s it? I’ll get a glass of water, and I’ll come back from the bathroom just fine, but that’s probably not enough time…

However, as a Devloper pushing the limits, this time is unacceptable, and if there is room for optimization, it should be done.

Next, I optimized the 001 code base in the first stage, where P0 refers to the elapsed time data before optimization

P1 has a more general decrease in time consumption

  • Test time: decreased from 6’34s(P0) to 5’31s(P1), a total loss of 1’03s
  • Total time: from 8’16s(P0) to 6’48s(P1), a total loss of 1’28s

But that wasn’t enough, so I did a second phase of optimization

P2 has a significant decrease in time consumption

  • Test time: decreased from 6’34s(P0) to 2’53s(P2), a total loss of 3’41s
  • Total time: from 8’16s(P0) to 4’3s(P2), a total loss of 4’13s

In fact, I spent very little energy on optimization, but I got good results. Now, can you still accept the time of P0 stage?

In actual projects, many integration tests were not written properly and took much longer time than the P0 phase, which resulted in delayed feedback and slow CI efficiency.

By now you should know how much room integration testing has for optimization, which is why we need to optimize integration testing.

Where is the time spent on integration testing?

Let’s move on to the second part — where does the testing go?

The test report output by Gradle Build shows the time of each test file. For example, I have captured part of the test report of the 001 code base:

Duration refers to the time spent on the tests themselves, which does not include the time spent preparing, running, and destroying the Spring context, so the total time spent on the tests actually looks pretty small (866 tests were unit and integration tests)

If we go into an integration test file and select Standard Output, we might see two types of logs

  1. The first type of log clearly initiates the Spring context, as shown below

  1. The second type of logging has no Spring context, as shown below

At this point, the question of where the time is spent has been roughly concluded. Here, I roughly summarize the time into two parts:

  1. The test itself (the test code we wrote)
  2. Build the environment required for testing (such as preparing, starting, and destroying the Spring container; Or starting and destroying in-memory databases.)

The test itself takes very little time (unless sleep or very large disk I/O operations are used); However, building the environment required by the test is very time-consuming, which is related to the size and complexity of the code base. This friend can run several integration tests on his own project and probably get the time-consuming data, which usually takes more than 10s.

Imagine how time consuming it would be if many integration tests had to restart the context.

What is the principle of Spring integration testing?

Now that we know where the integration testing time goes, how can we optimize it?

In fact, I explicitly say how to optimize, or there will be a small partner there are various doubts, simply I take you to see the source code, understand the principle of Spring integration testing, once clear principle, naturally know how to optimize, teach people to fish is better than teach people to fish.

Due to the limited space, and the article is mainly concerned with optimization, so the third part I ignore a lot of details, only show the main context of integration testing, as for the principle of finer granularity, I will consider writing an article, friends can also look down on their own source code.

I’m using JUnit5 and Spring Boot 2.4.5, so I’m going to assume that you have some knowledge of JUnit5.

As you can see from the figure above, JUnit5 Test Engine is responsible for executing the tests, the Spring Test Framework is responsible for building the environment needed for integration testing, and we just need to focus on what the Spring Test Framework is doing.

Build TestContext

After we start the integration Test, the Spring framework will first build a Test Context. The entry is @SpringBooTtest. The corresponding source code is as follows

We focus on @ BootstrapWith (SpringBootTestContextBootstrapper. Class), While @extendWith (SpringExtension.class) is helpful in understanding the principles of Spring integration testing, it is not the focus of this article and will be ignored for now.

SpringBootTestContextBootstrapper provides a lot of ways, one of our most concern is buildTestContext (), and the corresponding flow chart of the Build Test Context steps, this method provides a Test Context, Provide input to build the Spring ApplicationContext later.

Find/build the ApplicationContext

Here is the core code!

Here is the core code!

Here is the core code!

With Spring TestContext ready, you can then build Spring ApplicationContext based on Spring TestContext to prepare for the subsequent launch of Spring Application. This operation in DefaultTestContext. GetApplicationContext (), the source code is as follows

As you can see from the code, caching the ApplicationContext is preferred, Then we enter into cacheAwareContextLoaderDelegate. LoadContext (enclosing mergedContextConfiguration);

Then we look at the cache implementation logic, source code corresponding to the DefaultCacheAwareContextLoaderDelegate. LoadContext (mergedContextConfiguration)

Which do to contextCache synchronization locks, the purpose is to avoid loadContext (mergedContextConfiguration) method performs many times, the repeated start Spring Application

From this we can draw two conclusions:

  1. Spring will preferentially fetch from the cacheApplicationContext
  2. If the cache is not found, a new one will be builtApplicationContextAnd put it in the cache

Then we have two more questions:

  1. MergedContextConfigurationWhat is?
  2. What are the factors that affect the change fromContextCacheTo takeApplicationContext?

To the first question, we can open the MergedContextConfiguration. Java look at description

We can see from the description, when we perform a test class, Spring will configurations are combined to use the integration test, put in MergedContextConfiguration management, Then cache the ApplicationContext as the key in ContextCahce.

What are the specific configuration will be merged friends can see the implementation code, description and the class implementation code in AbstractTestContextBootstrapper. BuildMergedContextConfiguration ()

For your second question, we can see MergedContextConfiguration. HashCode ()

As you can see, there are a number of factors that affect retrieving the ApplicationContext from ContextCache, including the following:

  1. Different configurations are customized between integration test classes using @ContextConfiguration
  2. Integration test classes use different @ActiveProfiles configurations
  3. Different @TestPropertysource configurations are used between integration test classes
  4. Integration tests whether a class has an inherited parent class
  5. Whether different custom contexts are used between integration test classes is mainly maintained inSet<ContextCustomizer>You can go and see for yourself

I’ll focus on point 5, one of the ContextCustomizer implementation classes, MockitoContextCustomizer, which refers to @MockBean and @SpyBean, two annotations that are often used in integration testing. If two test classes use different @MockBean and @SpyBean, then the ApplicationContext cannot be reused between the two test classes, resulting in a rebuild of the ApplicationContext, Because hashCode() is different!!

So don’t mess with it@Mockbean 和 @SpyBean!!!!!!!!!

So don’t mess with it@Mockbean 和 @SpyBean!!!!!!!!!

So don’t mess with it@Mockbean 和 @SpyBean!!!!!!!!!

Running Spring Application

With the ApplicationContext built above, it’s time to start the Spring application and finally build the environment required for integration testing.

Code in SpringBootContextLoader. LoadContext (mergedContextConfiguration) and SpringApplication. Run (args).

Since it is not the focus of this article, I will not waste words, interested partners to have a look

Storage ApplicationContext

Caching the Spring ApplicationContext was mentioned above to find/build the Spring ApplicationContext, so it won’t be repeated here.

Long winded so much, the third part is finished, can be roughly summarized as three points:

  1. When starting each integration test class, Spring prioritizes execution efficiencyContextCacheTo takeApplicationContext
  2. If fromContextCacheIn less thanApplicationContext, a new one will be builtApplicationContext, then start Spring Application, and willApplicationContextThe cache toContextCache
  3. When you start each integration test class, Spring consolidates the configuration used by the integration test and places it inMergedContextConfigurationTo manage, and then use it as a keyApplicationContextcachedContextCahceOnce the configuration of each integration test class is different, it cannot be reusedApplicationContext, resulting in the need to reload the entire Spring context

How to reduce integration testing time?

If you understand the general principles of Spring integration testing, you will have a basic optimization strategy (the core goal is to reuse the ApplicationContext to reduce execution time).

Establish a standardized

We first need to reach some consensus within the team — what is the scope of integration testing?

These teams may be different, but I will sort them out based on personal empiricism and basically reach four consensus:

  1. What needs integration testing?

Code that does not involve external dependencies should be considered in the context of integration testing.

For example, Controller, Service, DB(in-memory database can be used instead)

  1. What doesn’t need integration testing?

External dependencies that make testing unstable should not be included in integration testing.

For example, MQ and FeignClient

  1. How do I isolate code that does not require integration tests?

Using the @ MockBean

  1. How do I organize custom configurations of integration tests?

Priority should be given to global reuse, as little as possible for special test classes, and then unified to manage in the common base class (including @MockBean). When writing integration tests, inherit directly from the base class.

Such as:

Refactoring @ MockBean

Careful @ MockBean!!!!!!

The frequent misuse of @MockBean in integration testing is one of the main causes of slow integration testing. Many people use @MockBean heavily for convenience and end up writing a bunch of meaningless tests and making integration testing very slow.

Most of the integration testing time is spent building data. If it doesn’t work, consider other options, such as declaring it @SpyBean, moving it to the base class, creating a technical debt card, and scheduling time to refactor the code.

Sink unit test

Consider sinking some integration tests into unit tests.

Unit tests do not require as much context preparation as integration tests, so execution times are very short.

Practice,

Practice, and then draw conclusions in practice, friends have any questions can leave a message.

conclusion

At the start of this article, through some key to highlight the importance of optimizing integration test data, and then analyzes the time-consuming part of then roughly combed the Spring with friends to the principle of integration test, a deep understanding of the root cause of the slow Spring integration test, finally gives the Suggestions of how to optimize the integration test, I hope it works.

Welcome to follow my wechat subscription number, I will continue to output more technical articles, I hope we can learn from each other.