The core points

  • Microservices typically belong to cross-functional, autonomous teams that are responsible for their implementation, testing, and release.
  • There should be no separate testing phase when implementing a microservice architecture.
  • Everyone is responsible for the quality of the products.
  • The complexity of microservice integration requires careful determination of the appropriate integration testing approach.
  • Continuous testing means a shift to the left to ensure that the team remains autonomous, and a shift to the right to be exploratory and experimental.

Architecture can have a significant impact on testing. The way you test a monolithic system is different from the way you test a loosely coupled system. We use microservices as a typical example of a loosely coupled system. As we’ll see below, they challenge our previous definition of testing.

Conway’s law

Melvin Conway observes that the structure of organizations has a significant impact on the products they create: “When any organization designs a system […] , delivered designs that are structurally consistent with the organization’s communication structure.”

Conway’s Law states that a company’s software architecture usually reflects the organization’s structure. On the other hand, how we organize our teams has a huge impact on our architecture and testing methods.

The interdependencies between Architecture and organizational structure are also reflected in BAPO, which assumes that in software engineering, four concerns need to be addressed: Business, Architecture, Process, and Organization. Fundamentally, the idea is to define architecture based on business requirements. However, as Jan Bosch (2017) said, most companies do not conform to BAPO, but to OPAB, which defines architecture based on organizational structure.

If an organization wants to change the architecture of a software product, its organizational structure (including how testing is organized) can be either an enabler or a hindrance. Let’s look at the different types of organizational structure:

  1. A function-oriented organization can optimize costs. These organizations typically have hierarchical structures that contain teams of experts responsible for their functional areas.
  2. Market-oriented organizations can optimize speed. These organizations tend to be flat, with cross-functional and autonomous teams responsible for implementing and delivering products, but this can lead to redundancies across the organization.
  3. Matrix-oriented organizations attempt to combine function-oriented and market-oriented ideas.

For more information on this, see: Gene Kim (2016).

Test microservices

How can a loosely coupled architecture like microservices be tested?

A loosely coupled system follows the principle of service autonomy because its architecture is based on the decomposition of autonomous components.

More and more organizations are adopting microservices to improve team autonomy and speed up change. Microservice applications consist of small, version-independent, extensible services that are customer-centric and communicate with each other over standard protocols with well-defined interfaces.

Micro service

  • Is autonomous and truly loosely-coupled because each microservice is physically separated from the other microservices.
  • Responsible for only one function (functional breakdown).
  • Can be independently deployed and published.
  • Dependencies should be embedded to ensure that the number of integration points is manageable.
  • Resilience and fault isolation should be possible through autonomy.

What does this mean for testing?

If we are domain-oriented and vertically shard for each business function, we can still have a layered architecture, but layers are a secondary organizational mechanism. Autonomous product development teams are responsible for implementing, testing, and delivering business functionality:

Microservices often come with DevOps: In Agile and DevOps, there is no single design phase where the architect defines the overall architecture before the development phase begins. Instead, architecture is defined in a more federated way, across multiple projects, and owned by the entire team.

What will happen to the way these systems are tested? Again, testing is not done in a single testing phase. On the contrary, everyone is responsible for quality.

There are no separate testing phases:

DevOps and Continuous Integration & Continuous Delivery (CI & CD) address the need to deliver functionality to customers quickly and with high quality. This means that testing must provide quick and meaningful feedback. To release quickly and confidently, you need immediate feedback from Continuous Testing:

  • Almost all testing is automated. Enabling continuous testing is critical for CI & CD as it provides timely feedback on quality. At the same time, it allows the team to learn and respond quickly.
  • However, this does not mean that continuous testing is only about automated testing. As we will see, it is a comprehensive approach to getting timely feedback on quality:

Image credit: Dan Ashby (2016)

Testing is not a separate activity, but is integrated into a comprehensive set of practices to improve quality: “There are still many companies that are trying to change processes to improve quality, from ‘test or check quality’ to ensuring quality in the first place, through culture, design, craftsmanship and leadership” (Ben Linders 2017).

Developers play an important role in ensuring high quality, and software process principles can also help improve quality. Software craftsmanship refers to the professionalism of the software development process: “Well-crafted software means that applications, no matter how old, can be easily understood by developers. Its side effects are well known and can be managed. With very high and reliable test coverage, the design is clear and simple, and the code can well express the business language “(Sandro Mancuso, 2015).

Continuous testing (usually) means turning left and turning right:

What does it mean to turn left and to turn right?

Tests are usually executed in a specific order, starting with the unit tests (left) because they provide quick feedback, and subsequent tests take longer to execute, but they can increase our confidence in the release candidate:

There is a left-turn trend here, where automated unit and component-level testing is performed in the Continuous integration & Continuous Delivery (CI & CD) pipeline owned by the product development team. However, this comes at a cost: integration testing becomes more challenging given the complexity and dynamics of integration scenarios.

If we want to contrast isolation with cooperation-based testing, it is important to distinguish between the solitary (solitary) and the social (solitary) testing:

  • In stand-alone testing activities, software units under test are isolated and collaborators (upstream dependencies) are replaced with test doubles, such as MOCks or STUBS.
  • In social testing activities, software units under test are tested with collaborators.

Following the test pyramid concept, microservice testing focuses on having many unit tests and a comprehensive set of component tests. Microservices are first tested in isolation from each other, followed by a small amount of integration, and finally end-to-end testing. Many end-to-end tests are often seen as a problem because they take longer to execute than unit and component tests and are more fragile. However, it is interesting to note that there is a debate about the value of the test pyramid theory for microservice testing and whether it is still reasonable to focus on unit and component testing and use them instead of integration testing (see examples Cindy Sridharan 2017 and Andre Schaffer 2018). This is a question of legitimacy, because if the integration of microservices is the most challenging, can we focus less on integration testing and more on social testing? I think the answer is yes and no: if the embedded dependencies of microservices do not provide independent business functionality, then it makes sense to test microservices with those dependencies. In microservices, this type of social testing should be done prior to integration testing with other microservices that encapsulate other business functions. Without adequate unit – and component-level testing, analyzing integration test failures is more complicated. In addition to traditional integration testing, API and contract integration testing are also important because they help ensure that microservices collaboration pipelines are independent of each other. Therefore, it is important to test at the appropriate test level.

Another aspect is also important: how many integration tests are necessary for microservices, where integration scenarios are complex and dynamic? There is a trade-off between putting more effort into testing and quickly probing problems in a production environment. In addition to turning left, there is a trend to more closely link testing with deployment and monitoring of production environments. Controlled experiment in production can help quickly identify problems and get quick feedback from real users.

What are the differences between different types of flow testing in production environments?

That means there is also a shift to the right in testing.

As mentioned earlier, market-oriented organizations are optimized for speed, and a right turn in the testing area helps achieve their goals. Explore and learn as part of an ongoing experiment. With the help of verification experiments, we were able to get customer feedback when using the application in the real world.

Verification experiments can also support elastic architectures. As we discussed earlier, microservices should achieve reliability and fault isolation through autonomy. But what is elasticity? “We often think of weak systems as the opposite of robust systems, because weak systems tend not to pay much attention (in other words, not to like or dislike) to stress. In fact, robust systems are only less vulnerable. The real opposite of a fragile system is one that benefits from stress. Nassim Taleb calls such a system antifragile “(Dave Zwieback 2014, p. 3). The traditional way of testing is to test the system before release to ensure stability. However, these tests support robustness, but not anti-vulnerability. A different approach to testing is to introduce failures into the production environment and run active controlled chaos experiments. This well-designed chaos can guide product development teams in thinking about failure tolerance and isolating the possibility of failure.

conclusion

For loosely coupled systems such as microservices, testing should not be carried out by a dedicated test team in a single test phase, but rather by cross-functional product development teams. In testing, there was a shift to the left to ensure the team remained autonomous, and a shift to the right that emphasized exploration and experimentation.

Continuous Testing, which includes both automated Testing in CI & CD and an experimental and exploratory culture, ensures the rapid and stable release of loosely coupled services.

reference

  • Ashby, Dan (10/2016): Continuous Testing in DevOps.
  • Bosch, Jan (11/26/2017): Structure Eats Strategy.
  • Conway, Melvin E. (1968): How do Committees Invent?
  • Kim, Gene (11/16/2016): How to Design With Conway’s Law in Mind.
  • Linders, Ben (2017): What Drives Quality: A Deep Dive into Software Quality with Practical Solutions for Delivering High-Quality Products. Ben Linders Publishing.
  • Mancuso, Sandro (2015): The Software Craftsman. Professionalism, Pragmatism, Pride. Pearson Education.
  • Schaffer, André (01/11/2018): Testing of Microservices.
  • Sridharan, Cindy (12/30/2017): Testing Microservices, the sane way.
  • Zwieback, Dave (2014): Antifragile Systems and Teams. O ‘Reilly Media.

About the author

Stefan Friese is HERE’s head of testing and development, currently based in Berlin. In his current role, he leads the team building continuous integration & delivery test capabilities for a number of real-time location Web services hosted on the HERE cloud platform. Stefan has 14 years of experience as a test lead, performance tester, software engineer, and quality engineer. You can follow Stefan on Twitter via @y3key.

Architecturally Aligned Testing