In this paper, starting from vivo Internet technology WeChat public links: https://mp.weixin.qq.com/s/Z3uJhxJGDif3qN5OlE_woA author: wenbo zhang

The Path to Domain-Driven Design Practice

Domain-driven Design (DDD) Road to Practice (I) focuses on strategic DDD principles.

This second article in the “Path to Domain-driven Design Practices” series examines how events can be applied to decouple core software complexity. Explore why CQRS is widely used in DDD projects and how to implement CQRS framework. Of course, we should also be alert to the lessons of some failures, and analyze the pros and cons before choosing the right way to deal with them.

I. Preface: Start with logistics details

Everyone is familiar with logistics tracking, which keeps a detailed record of what happened at what time, and data is immutable as important evidence. I understand the value behind it in the following aspects: the business side can control each sub-process and know the current link; On the other hand, when it is necessary to trace back, the entire historical process can be replayed simply by recording each step.

In my previous article, I pointed out that “software projects are also a category of productive relations in human society, but the fruits of our labor are invisible.” Therefore, we can learn from the idea of logistics tracking to develop software projects and break down complex processes into steps, sub-processes and states, which is consistent with our event division, which is a typical case of event-driven.


Ii. Domain events

Domain Events is a concept in Domain Driven Design (DDD) that captures what happens in the Domain we are modeling.

Domain events themselves are also part of the Ubiquitous Language used by all project members, including domain experts.

For example, in the case of cross-border logistics mentioned above, it is necessary to assign staff for sorting and subcontracting after the goods arrive at the bonded warehouse, so “the goods have arrived at the bonded warehouse” is a field event.

First, from the business logic, this event is related to the success or failure of the whole process; At the same time, subsequent subprocesses are triggered; For the business side, this event is also a symbolic milestone, representing their goods will soon be delivered to their hands.

So generally speaking, a domain event has the following characteristics: high business value, helps to form a complete business loop, and leads to further business operations. It is also important to note that domain events have clear boundaries.

For example, if you are modeling a restaurant checkout system, then “the customer has arrived” is not your concern, because you cannot immediately ask for money when the customer arrives, and “the customer has placed an order” is a useful event for the checkout system.

Modeling domain events

When modeling domain events, we should name events and attributes according to the common language in the bounded context. If the event is generated by a command action on an aggregation, we usually name the domain event after the name of that action method.

For the example “goods arrived in bonded warehouse” above, we will publish the corresponding domain event

GoodsArrivedBondedWarehouseEvent (of course in the boundaries of clear context can also remove the name of the polymerization, modeling for ArrivedBondedWarehouseEvent directly, this is the habit of naming).

The name of the event indicates what happens after the command method on the aggregation is successfully executed; in other words, pending items and uncertain states are not domain events.

An effective way to do this is to draw a state flow diagram of the current business, including pre-actions and resulting state changes. In this case, the state is expressed as the state that has been changed so we do not use the past tense, such as delete or cancel, which means deleted or canceled.

Event modeling is then performed for the nodes. The following figure shows the business of cloud storage of files. We model “past simple” events, PreUploadedEvent, ConfirmUploadedEvent and RemovedEvent, for pre-upload, confirmation of upload completion and deletion respectively.


2. Domain event code interpretation

package domain.event; import java.util.Date; import java.util.UUID; /** * @Description: * @Author: zhangwenbo * @Since: 2019/3/6 */ Public class DomainEvent {/** * Domain events also contain unique ids, * but this ID is not an Entity level ID concept, * is mainly used for event tracing and logging. * For database storage, this field is usually a unique index. */ private final String id; /** * The creation time is used for traceability. On the other hand, regardless of which event store is used, event delays are likely to be encountered. */ private final Date occurredOn; publicDomainEvent() { this.id = String.valueOf(UUID.randomUUID()); this.occurredOn = new Date(); }}Copy the code


There are two things to note when creating domain events:

  • Domain events themselves should be Immutable;

  • Domain events should carry contextual data information that is relevant when the event occurs, but not state data for the entire aggregation root. For example, an order can be created with the basic information of the order, while the AddressUpdatedEvent event, in which the user updates the order’s shipping address, only needs to contain the order, the user, and the new address.

Public Class AddressUpdatedEvent extends DomainEvent {public class AddressUpdatedEvent extends DomainEvent { private String userId; private String orderId; // New Address private Address Address; // omit specific business logic}Copy the code


Storage of domain events

The immutability and traceability of events dictate that they must be persisted, so let’s look at some common scenarios.

3.1 Separate EventStore

Some business scenarios create a separate event storage center, which could be Mysql, Redis, Mongo, or even file storage. The following uses Mysql as an example. Business_code and event_code are used to distinguish different events of different services. Specific naming rules can be based on actual requirements.

Be aware of situations where the data source is inconsistent with the business data source. We want to make sure that events are accurately recorded when the business data is updated. In practice, try to avoid using distributed transactions, or try to avoid cross-library scenarios, otherwise you will have to figure out how to compensate. The AddressUpdatedEvent event fails to save when the user updates the shipping address.

The general rule is to Say No to distributed transactions, however, I believe there are more methods than problems, and in practice we can always come up with a solution. The difference is whether the solution is simple and decoupled.

# Consider whether you need a separate table, the event storage suggestion logic is simpleCREATE TABLE `event_store` ( `event_id` int(11) NOT NULL auto increment, `event_code` varchar(32) NOT NULL, 'occur_name' vARCHar (64) NOT NULL, 'occurred_body' varchar(4096) NOT NULL, 'occurred_on' datetime NOT NULL,  `business_code` varchar(128) NOT NULL, UNIQUE KEY (`event id`) ) ENGINE=InnoDB COMMENT'Event Store Table';Copy the code


3.2 Storage together with Service Data

In a distributed architecture, each module is made relatively small, to be exact, “autonomous.” If the current amount of service data is small, you can store the event together with the service data and use related identifiers to distinguish real service data from event records. Alternatively, the current service database can create its own event store, but considering that the magnitude of event store must be larger than the real service data, consider whether to separate tables.

Advantages of this scheme: data autonomy; Avoid distributed transactions; No additional event storage center is required. The disadvantage, of course, is that it cannot be reused.

4. How to release domain events

4.1 Send domain events by domain aggregation

/* * an example of a hyperemic model for a Match * * the anemia model will construct a MatchService, we use the model to trigger the corresponding event * * this example omitted the business details */ public class Match {public voidstart() {// construct Event.... MatchEvent matchStartedEvent = new MatchStartedEvent(); / / out specific business logic DefaultDomainEventBus. The publish (matchStartedEvent); } public voidfinish() {// construct Event.... MatchEvent matchFinishedEvent = new MatchFinishedEvent(); / / out specific business logic DefaultDomainEventBus. The publish (matchFinishedEvent); } // omit the Match object basic attributes}Copy the code


4.2 Event Bus vs. message middleware

Domain events within microservices can achieve business collaboration between different aggregations through the event bus or by leveraging application services. That is, when domain events occur in microservices, it is not necessary to introduce message-oriented middleware because the integration of most events takes place in the same thread. However, if an event updates multiple aggregate data at the same time, according to the DDD principle of “one transaction updates only one aggregate root”, we can consider introducing message-oriented middleware to adopt different transactions for different aggregate roots in microservices by means of asynchronization

Saga distributed transactions

1. Saga Overview

Let’s see how the Saga pattern can be used to maintain data consistency.

Saga is a mechanism for maintaining data consistency in a microservice architecture that avoids the problems associated with distributed transactions.

A Saga represents one of several services that need to be updated, i.e. a Saga consists of a series of local transactions. Each local transaction is responsible for updating the private database of its service, and these operations still rely on the familiar ACID transaction framework and library.

Pattern: the Saga

Maintain data consistency across multiple services by using asynchronous messages to coordinate a series of local transactions.

Please refer to the (strongly recommended) : microservices. IO/patterns/da…

Saga has one less Try operation than TCC, which requires two interactions with transaction participants regardless of whether the transaction succeeds or fails. Saga, on the other hand, only needs to interact with transaction participants once in the event of a successful transaction, and an additional compensation rollback is required if the transaction fails.

  • Each Saga consists of a series of sub-Transaction Ti;

  • Each Ti has a corresponding compensation action Ci, which is used to cancel the result caused by Ti.

As you can see, Saga has no “reserve” action compared to TCC, and its Ti simply commits directly to the library.

There are two sequences of Saga execution:

  • Success: T1, T2, T3… Tn;

  • Failure: T1, T2… , Tj, Cj,… , C2, C1, where 0 < j < n;

So we can see that Saga undo is critical, and arguably the difficulty with Saga is how to design your rollback strategy.

2. Saga implementation

Through the above example, we have a preliminary body sense of Saga. Now let’s discuss how to achieve it in depth. When a Saga is started by a system command, the coordination logic must select and notify the first Saga participant to perform a local transaction. Once the transaction completes, the Saga coordinator selects and invokes the next Saga participant.

This process continues until Saga has completed all the steps. If any local transactions fail, Saga must execute the compensation transactions in reverse order. There are several different approaches that can be used to build Saga’s coordination logic.

2.1 Choreography

The decision and execution sequence logic of Saga is distributed among each Saga participant, who communicates by exchanging events.

(Quoted in the relevant section of Microservice Architecture Design Patterns)

  1. The Order service creates an Order and publishes the OrderCreated event.

  2. The Consumer service consumes the OrderCreated event, verifies that the Consumer can place an order, and issues the ConsumerVerified event.

  3. The Kitchen service consumes the OrderCreated event, validates the order, creates a fault ticket in the CREATE_PENDING state, and publishes the TicketCreated event.

  4. The Accounting service consumes the OrderCreated event and creates a Credit CardAuthorization in the PENDING state.

  5. Accounting Services consume TicketCreated and ConsumerVerified events, charge consumers’ credit cards and issue credit card authorization failure events.

  6. The Kitchen service uses the credit card authorization failure event and changes the status of the fault ticket to REJECTED.

  7. The order service consumes a credit card authorization failure event and changes the order status to rejected.

2.2 Orchestration

Centralize Saga decision and execution order logic in a Saga choreographer class. The Saga choreographer issues imperative messages to Saga participants instructing those participating services to complete specific operations (local transactions). Similar to a state machine, when the participant service completes an operation it sends a status directive to the choreographer to decide what to do next.

(Quoted in the relevant section of Microservice Architecture Design Patterns)

Let’s analyze the execution process

  1. The Order Service first creates an Order and a create Order controller. After that, the path flows as follows:

  2. The Saga Orchestrator sends the Verify Consumer command to the Consumer Service.

  3. The Consumer Service replies to the Consumer Verified message.

  4. The Saga orchestrator sends the Create Ticket command to the Kitchen Service.

  5. The Kitchen Service replies with the Ticket Created message.

  6. The Saga coordinator sends authorization card messages to the Accounting Service.

  7. The Accounting service replies with the card authorization message.

  8. The Saga Orchestrator sends the Approve Ticket command to the Kitchen Service.

  9. The Saga Orchestrator sends the approve order command to the order service.

2.3 Compensation Strategy

In the previous description we said that the most important thing about Saga is how it handles exceptions, and that the state machine defines many exception states. The 6 above failure occurs, triggering AuthorizeCardFailure, at which point we terminate the order and roll back the previously committed transaction. It is important to distinguish between checksum transactions and those that require compensation.

A Saga consists of three different types of transactions: compensable transactions (which can be rolled back and therefore have a compensation transaction); Critical transactions (this is the key to Saga’s success or failure, such as 4 account withholdings); And repeatable transactions that do not need to be rolled back and are guaranteed to complete (such as 6 status updates).

In Create Order Saga, the createOrder(), createTicket() steps are compensable transactions with compensable transactions that undo their updates.

The verifyConsumerDetails() transaction is read-only, so no compensation transaction is required. The authorizeCreditCard() transaction is the key transaction of this Saga. If the consumer’s credit card can be authorized, then this Saga guarantees completion. The approveTicket() and approveRestaurantOrder() steps are repeatable transactions following critical transactions.

It is important to carefully unpack each step and then evaluate its compensation strategy, as you can see that each type of transaction plays a different role in the strategy.

Four, CQRS

Having covered the concept of events and examined how Saga solves complex transactions, we now look at why CQRS is widely used in DDD. In addition to the read-write separation features, we can effectively reduce business complexity by implementing Command logic in an event-driven manner.

When you understand how to model events, how to circumvent complex transactions, and when to use message-oriented middleware and when to use the event bus, you can understand why CQRS is used and how to apply it correctly.

(Photo from Internet)

Here is the design in our project, why there is a Read/Write Service, is to encapsulate the call, the Service internally sends events based on aggregation. Because I find that in real projects, many people immediately ask me for XXXService instead of the XXX model, I recommend this center strategy for projects where DDD is not fully available. This is also consistent with our decoupling, the other side depends on my abstraction capabilities, and it is not transparent whether I am internally based on DDD or traditional process code.

Let’s start by looking at events and the timing of the processor.

Here is the file cloud storage service as an example, the following is the core code of some processors. The comment lines explain the functionality, usage, and extension of the code, so read them carefully.

package domain; import domain.event.DomainEvent; import domain.handler.event.DomainEventHandler; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; /** * @description: event registration logic * @author: zhangwenbo * @since: 2019/3/6 */ public class DomainRegistry { private Map<String, List<DomainEventHandler>> handlerMap = new HashMap<String, List<DomainEventHandler>>(); private static DomainRegistry instance; privateDomainRegistry() {
    }

    public static DomainRegistry getInstance() {
        if (instance == null) {
            instance = new DomainRegistry();
        }
        return instance;
    }

    public Map<String, List<DomainEventHandler>> getHandlerMap() {
        return handlerMap;
    }

    public List<DomainEventHandler> find(String name) {
        if (name == null) {
            return null;
        }
        returnhandlerMap.get(name); } // Event register is divided into several scenarios according to the service. // This is the core of the service flow. Public void register(Class<?) public void register(Class<? extends DomainEvent> domainEvent, DomainEventHandler handler) {if (domainEvent == null) {
            return;
        }
        if(handlerMap.get(domainEvent.getName()) == null) { handlerMap.put(domainEvent.getName(), new ArrayList<DomainEventHandler>()); } handlerMap.get(domainEvent.getName()).add(handler); // Sort event handlers by priority... }}Copy the code


This is an example of a file upload event.

package domain.handler.event; import domain.DomainRegistry; import domain.StateDispatcher; import domain.entity.meta.MetaActionEnums; import domain.event.DomainEvent; import domain.event.MetaEvent; import domain.repository.meta.MetaRepository; import org.springframework.stereotype.Component; import javax.annotation.PostConstruct; import javax.annotation.Resource; /** * @description: an event handler * We use a mixture of Saga's two modes, the outer event interaction; * Internal implementation of state flow for a single complex event. * @Author: zhangwenbo * @Since: 2019/3/6 */ @Component public class MetaConfirmUploadedHandler implements DomainEventHandler { @Resource private MetaRepository metaRepository; public void handle(DomainEvent event) { //1. We define a ThreadLocal variable in the current context // to hold the aggregate root information (shared by threads) //2. Of course, if you need any additional information, A Specification can be constructed from a repository based on the information carried by the event // code example // metaRepository.queryBySpecification(SpecificationFactory.build(event)); DomainEvent domainEvent = metaRepository.load(); // Here is our logic... // For complex single operations, you can use state flow to further split domainEvent.setStatus(nextState); // After the event is triggered, a state tracker is still needed to resolve large transaction issues //Saga choreographed stateDispatcher.dispatch (); } @PostConstruct public voidautoRegister() {// This can be further subdivided, which kind of scenario is registered in, which is also the power and flexibility of event-driven. // Avoid if... The else. We can realize that once your logic is filled with a lot of //switch,ifTo see if their registration scene can continue subdivision DomainRegistry... getInstance (). The register (MetaEvent. Class, this); } public StringgetAction() {
        returnMetaActionEnums.CONFIRM_UPLOADED.name(); } // Applies to dependent events. The execution sequence is specified by prioritygetPriority() {
        returnPriorityEnums.FIRST.getValue(); }}Copy the code


Event bus logic

package domain;

import domain.event.DomainEvent;
import domain.handler.event.DomainEventHandler;
import java.util.List;

/**
 * @Description:
 * @Author: zhangwenbo
 * @Since: 2019/3/6
 */

public class DefaultDomainEventBus {

    public static void publish(DomainEvent event, String action,
                               EventCallback callback) {

        List<DomainEventHandler> handlers = DomainRegistry.getInstance().
            find(event.getClass().getName());
        handlers.stream().forEach(handler -> {
            if(action ! = null && action.equals(handler.getAction())) { Exception e = null; boolean result =true;
                try {
                    handler.handle(event);
                } catch (Exception ex) {
                    e = ex;
                    result = false; // Custom exception handling... } finally { //write into event store saveEvent(event); } // Handle callback scenarios based on the actual business, DefaultEventCallback can returnif(callback ! = null) { callback.callback(event, action, result, e); }}}); }}Copy the code


Autonomous services and systems

DDD emphasizes the autonomy of bounded context. In fact, from a smaller granularity, objects still need to have these four characteristics of autonomy, that is, minimum completeness, self-fulfillment, stable space, independent evolution. Among them, self-fulfillment is the key point, which is stable because it is not strongly dependent on external factors, and independent evolution is possible only because of stability. This is why hexagonal architectures are common in DDD.

(Photo from Internet)


Six, the concluding

The events, Saga, and CQRS scenarios described in this article can all be used individually and can be applied to one of your methods or your entire package. We don’t have to practice a whole set of CQRS in our project, just some of the ideas that solve a problem in our project are enough.

Maybe now you’re ready to practice these skills on a project. However, we need to understand that “every coin has two sides”, and we should not only see the advantages of high scaling, decoupling, and easy orchestration, but also understand the problems it brings. After analyzing the pros and cons, it is the right way to decide how to achieve it.

  • This type of programming has a learning curve;

  • The complexity of message-based applications;

  • Dealing with the evolution of events is difficult;

  • It is difficult to delete data;

  • Querying the event repository can be challenging.

However, it is important to recognize that hexagonal architectures and DDD tactics will speed up our domain modeling process when appropriate and force us to explain a domain from a strictly generic language perspective, rather than individual requirements. Any approach that places more emphasis on core domains than technology implementation adds value to the business and gives us a greater competitive advantage.

​​​​​​​


Attached: References

  1. Pattern: Saga

  2. Distributed transactions: Saga mode

  3. Book: Design Patterns for Microservices Architecture

For more content, please pay attention to vivo Internet technology wechat public account

Note: To reprint the article, please contact our wechat account: Labs2020.