In order to satisfy different users in terms of price, the experience of differentiation, drops provides more and more rich category, these processes are roughly similar category, have differences in some details of the experience, a set of architecture how to juggle isolation and reuse, while supporting the category, and drops the server technology platform for the Gulf Stream to do.

1. Project background

In Didi taxi-hailing business, the API of the service end receives requests from the service end, assemes and returns them, and connects orders, pricing, cashier and other business systems of the service center to complete the whole taxi-hailing process. Gulfstream Platform project expects to create a “travel center” at the API layer of the service end. Why do we need to create a “travel center” in the business “front desk” API layer when there is already a travel center? This is inseparable from the traffic mode of the travel business.

1.1 Traffic Mode

1.1.1 Traditional and industry conventional “cone” traffic allocation model

The traditional typical middle platform architecture is “big middle platform and small front desk”. The reason for this architecture is related to the traffic distribution mode. In the field of e-commerce, for example, the middle platform abstracts the business entities related to e-commerce, such as orders, cash registers, commodities, etc., while the flow inlet between different business lines is separated, and different BU can be closed loop in the way of small front desk. The architecture of “big middle stage and small front stage” can support the rapid construction of prototype products for trial and error exploration. The middle stage provides basic capabilities of e-commerce standards, and the front stage is closed to achieve B2C, C2C and other businesses without affecting each other.

The traffic distribution mode is as follows: the traffic access from the front desk of multiple services (or categories) is transferred to the unified and limited service middle platform, and finally falls to the basic platform.

1.1.2 Unique “diamond” traffic distribution mode of online ride-hailing

The core of Didi’s ride-hailing business is taxi-hailing. In order to meet the needs of different users (pricing, response rate, experience, etc.), it provides differentiated service capabilities through category differentiation, extending from the initial taxi, private car and express to dozens of categories today. The entrance is always around the two ends of the vehicle, open platform.

The traffic distribution mode is as follows: multiple categories are connected to the API by the unified end and the main service logic processing of each category is completed, and then transferred to the unified service center and finally to the basic platform.

The “diamond” traffic distribution mode of online car hailing is destined to: on the one hand, the server API needs to support some cross-BU platform demands, such as The Spring Festival service fee, epidemic service shutdown, etc.; On the other hand, differentiated demands between different BU should also be supported, such as taxi using meter pricing rather than online pricing.

As the categories become more and more diverse, the logic of these differences also increases, resulting in the system becoming more and more bloated, more and more complex, and the efficiency of iteration decreases. The Gulf Stream platform project was created by sinking the common parts of the server API and opening up the mechanism for differential customization while simultaneously taking into account isolation and reuse.

1.2 Responsibilities of the Server API Before you start, you need to define the core responsibilities of the server API.

Thecountry.Core responsibilities

  • Traffic dyeing: Identifies and defines categories, scenarios, and functions in access traffic, and escapes them as unified service features.

  • Process concatenation: Invokes downstream services according to logic and process to accomplish specific functions based on different events/requests.

  • Data rendering: render the resulting data into corresponding data views according to different end or category/scene requirements.

Thecountry.The ultimate question

  • Reuse: what (reuse what, reuse to what level), how (how to achieve reuse)

  • Isolation: What (what is isolation), how (what is isolation mechanism)

1.3 Evolution of the Gulf Stream Platform

This is a brief overview of the first two iterations of the Gulfstream platform project.

1.3.1 Gulfstream Platform 1.0 (2017-2018)

Phase 1.0 mainly solved the problem of rapidly adding new categories and code isolation between different BU, which was solved by configuration and plug-in. Configuration is to unify the upstream and downstream product description protocols, form n-tuples of product description, and abstract a set of general n-tuples to function mapping rules. Plug-in using plug-in package to isolate the code between different BU, run time plug-in selector is distributed to the corresponding plug-in package according to traffic characteristics.

Thecountry.legacy

  1. Configuration relies on functional abstraction and requires a unified approach to abstraction

  2. Plug-ins rely on solid processes and clear functional boundaries. Opening plug-in points according to differences will lead to problems such as unclear definition of plug-in, uncontrollable granularity, unstable insertion points, and long-term maintenance difficulties.

1.3.2 Gulfstream Platform 2.0 (2018)

On the one hand, 2.0 has to solve the problems of function abstraction and process solidification left over from 1.0, and on the other hand, it has to face the increasingly complex server API system. To this end, we borrowed DDD’s ideas and started the Transformation of gulfstream platform 2.0.

  • Macroscopically, services are split according to core data & functions, and a large module is divided into multiple vertical closed loop sub-modules, namely, domain. Through the divide-and-conquer method, the overall complexity is reduced, and the online conflicts and queuing situations caused by the development of all team members in one module are also solved.

  • Microscopically, domain services are further analyzed according to processes and functions, and functions are aggregated and abstracted, that is, componentization. Improve reusability and solve the problems of business scalability and development efficiency.

Thecountry.legacy

  1. Without systematic framework constraints, iteration is easy to destroy the original structure

  2. It emphasizes divide-and-conquer and abstraction, and does not consider the isolation between categories synchronously

**2.** Detailed proposal for Gulfstream Platform 3.0

2.1 General Idea

As mentioned earlier, the Gulfstream platform core addresses isolation and reuse issues. The overall idea of 3.0 is to divide the business logic of the server API into two layers: one layer is the process layer used to concatenate state flow, and the other layer is the capability component layer used to complete the functions of each vertical class. The process layer includes not only the macro taxi hailing process from estimation, order issuing to order completion, but also the execution process of each interface. The basic categories of the macro taxi hailing process are unified, and so are the interaction protocols with the terminal. The execution process of each interface may be partially inconsistent among different categories due to the difference in business form. We abstract the implementation process of the interface to form steps one by one to precipitate a set of common implementation steps of interface standards. Categories can overload STEP according to their differences.

Capability component abstraction aggregates some vertical functions, and implements component behavior according to different category scenarios using different policies according to policy patterns. For example, the unicast module provides modes such as delayed unicast, real-time unicast, and round unicast. The special bus reservation adopts delayed unicast, which is realized through configuration in the unicast module. If something is very different, for example, a taxi wants to implement a new broadcast order mode and does not have universality, the taxi can also implement the new mode and mount it in the form of plug-ins.

After such transformation, some platform products were born at the same time to help improve the development efficiency. For example, the process choreography center can arrange the interface process according to different categories and scenarios. Feature management platform, unified control of business characteristics, to maintain uniform business description; Category configuration center can configure behavior modes of different abilities from the perspective of category scenarios, and quickly launch new category scenarios.

2.2 Framework Introduction

In order to achieve the overall idea, we developed a set of code running framework, named DuKang, how to solve the problem, only DuKang! Relying on DuKang framework, solve our isolation and reuse problems.

DuKang framework implements scheduling for each interface according to the following flow, involving core concepts such as InputSource, Transport, TransportFactor, StepRuntime, Step, and Ability.

The core point is that Transport, as a process bearer, provides a base process realization. Different categories can inherit BaseTransport, and then reload step of different processes. However, the whole process is scheduled by process-driven engine, and all categories are consistent. Ability is a capability component, which provides a set of common modes within the component. Different types of scenarios reuse these modes in a configurational way, and meanwhile open the mechanism of customized modes to businesses. Businesses can use Biz to customize their own unique modes and mount them under ABILITY to achieve differentiated functions.

The server API language stack is mainly PHP and Golang. The old services are mainly written in PHP, and the whole is gradually migrating to Golang. The new services are directly using Golang, and the Dukang framework supports both PHP and Golang versions.

Next, some core concepts of Dukang will be explained

2.2.1 Transport – Service Transport/bearer

Thecountry.define

  • A process carrier that is abstracted for different capacity (or category) in a single interface

  • Any business must have a process and order of execution

Thecountry.The constraint

  • BaseTransport overrides the generic process of the current interface business

  • There is only one BaseTransport in any interface

  • XxxTransport covers the differential implementation of different capacity (or category)

  • The differentiated XxxTransport within any interface must inherit from BaseTransport

  • Any Transport contains at least one Step Transport <=> [N]Step (N >= 1)

2.2.2 Step – Process

Thecountry.define

  • A link carrier that abstracts the business within a single interface

  • A single Step is a specific implementation of a business segment or function

Thecountry.features

  • A single Step is an effective means of large-grained differentiation (e.g., luxury cars/taxis) and can be achieved by Override Step

  • The communication between steps is connected in series by the unified runtime data bus StepRuntime, which can theoretically achieve hot swap

Thecountry.legend

  • Business process-driven interface: mainly to connect various processing processes

  • Data-driven interface: mainly builds business characteristic data

2.2.3 StepRuntime – Runtime data bus

Thecountry.define

  • The process cascades the runtime data bus

Thecountry.The constraint

  • StepRuntime can only be used as the input of Ability, and the business characteristics and data of StepRuntime cannot be modified in Ability and the underlying logic

Thecountry.legend

2.2.4 InputSource – InputSource

Thecountry.define

  • External input data source to prepare data for TransportFactor

  • Used to eliminate external execution environment differentiation and isolate external input parameters

Thecountry.The constraint

  • After Step is entered, it has the read-only attribute, which is referenced by StepRuntime. Subsequent service logic cannot modify all features and data contents contained in Step

Thecountry.legend

2.2.5 Transport Factor – Indicates the Transport Factor

Thecountry.define

  • The Transport decision factor for the service Transport

  • Different interfaces may take different factors to make Transport decisions

  • The current implementations are product_id for the order dimension and car_level for the driver dimension

  • The end source can be selected as the decision factor, such as Didi Chuxing App, open platform, Lichen Private car app, etc

Thecountry.The constraint

  • Factor must be several types of factors that can be selected and cannot be written freely by RD students

  • In principle, the TransportFactor of the same service interfaces must be consistent

2.2.6 Ability – Ability Components Affectative Description

It was not defined

  • From the perspective of characteristic data, the focused business is refined and abstracted to form capability components

◎ Design principles

  • Design for reuse

  • Design for extensible differentiation

2.2.6.2 Service Features

Thecountry.Summarize the concept of business characteristics

  • Business Expression: Unicast plan = sStartAddDuseTime+sStartAddDuseTime +sStartAddDuseTime +bIsDelayBroadcast +bIsRepeatAssign+ bIsRepeatAssign+ bIsRepeatAssign+iBroadcastAssignType + $iBroadcastExpire

Thecountry.Business characteristics solve the problem

  • Standardize business attribute or field semantics to avoid ambiguity and unknown semantics

  • Definition: Business = ordered process * control (Feature X, feature Y, feature Z…) , copy control = dyeing filling | | | | tag

2.2.6.3 Capability component extraction process

For a set of service characteristics, the production and modification operations of these service characteristics are aggregated to form capability components. Such as around the unicast feature of the unicast components, pricing components of the price characteristics and so on

Thecountry.Ability scalability

Support Addtional business extension Ability Mode, namely the biz form mentioned above, to customize Mode, so as to achieve the isolation of different categories in Ability.

Thecountry.Ability+ Category scenario configuration final idea

Reuse: Based on category + scenario (n-tuple) configuration, mode selector Execution mode of flexible decision making components

Differentiation: Biz implements mode customization and configures dynamic loading

2.3 Application Framework Directory Structure

Example PHP module directory structure, golang module as a whole similar

// Module root Directory ├─ Dukang // New introduced content, separate old code, Future will replace hermes │ ├ ─ ─ Ability / / Ability to │ │ ├ ─ ─ Common / / public Ability │ │ │ ├ ─ ─ DispatchOrder │ │ │ │ └ ─ ─ DispatchOrderComponent. PHP │ │ │ └ ─ ─ VirtualPhone │ │ │ └ ─ ─ VirtualPhoneComponent. PHP │ │ └ ─ ─ Express / / category specific capacity extension │ │ └ ─ ─ DispatchOrder │ │ └ ─ ─ DispatchOrderComponent. PHP │ ├ ─ ─ the Config / / interface configuration │ │ └ ─ ─ ConfirmOrder. Json │ ├ ─ ─ InputSource / / interface input │ │ └ ─ ─ ConfirmOrderInputSource. PHP │ ├ ─ ─ StepRuntime / / global data bus │ │ └ ─ ─ ConfirmOrderStepRuntime. PHP │ ├ ─ ─ Model / / utility Model │ │ ├ ─ ─ Dao │ │ ├ ─ ─ Driver │ │ │ └ ─ ─ DriverModel. PHP │ │ └ ─ ─ the Order │ │ └ ─ ─ OrderModel. PHP │ ├ ─ ─ Service / / public Service │ │ └ ─ ─ Driver │ │ └ ─ ─ DriverService. PHP │ ├ ─ ─ TransportFactor / / Transport factor │ │ └ ─ ─ ConfirmOrderTransportFactor. PHP │ └ ─ ─ Transport / / process concatenated │ ├ ─ ─ Base │ │ └ ─ ─ ConfirmOrderBaseTransport. PHP │ └ ─ ─ Taxi │ └ ─ ─ ConfirmOrderTaxiTransport. PHP └ ─ ─ ├─ ├─ SRC // exercises (Dimensions, dTO) ├─ SRC //Copy the code

Drops Logi

Didi Logi log service suite has been polished in Didi for more than 7 years, aiming at log collection, log storage, log computing, log retrieval and log analysis. It has carried out targeted optimization in component capability PAAS construction and engine stability and expansibility.

Currently, this suite has opened source Didi Logi-Kafkamanager, and will open source logi-Agent, Logi-LogX, logi-ElasticSearchManager and other PAAS suites in the future.

1. Github: z.didi.cn/4newP

2, rapid experience address: http://117.51.150.133:8080/kafka account password admin/admin

3, daily FAQ: github.com/didi/Logi-K…

4. Manual: github.com/didi/Logi-K…

5. Didi Logi-Kafkamanager Cloud platform Construction Summary:

Mp.weixin.qq.com/s/9qSZIkqCn…

6, series of video tutorials: mp.weixin.qq.com/s/9X7gH0tpt…

Drops the nightingale

Didi Nightingale is a set of distributed and highly available operation and maintenance monitoring system. Its biggest feature is hybrid cloud support, which can support both traditional physical machine virtual machine scenarios and K8S container scenarios. Meanwhile, Didi Nightingale is not only capable of monitoring, but also of CMDB and automatic operation and maintenance. Many companies develop their own operation and maintenance platforms based on Didi Nightingale.

Making: z.d idi. Cn / 4 wurz

Official document: n9e.didiyun.com

Gocn. VIP /topics/1081…

The voice answer: m.ximalaya.com/keji/450958…

Video tutorial: m.bilibili.com/space/44253…

Secondary development: xie.infoq.cn/article/30d…

If you have problems using didi Logi-KafkaManager and nightingale, or have any questions you need to communicate with the developers, you can scan the qr code below to enter didi Logi and nightingale’s open source user group, and ask questions in the group.

The group has Didi Logi-KafkaManager and nightingale project leader: Didi senior expert engineer — Zhang Liang, Qin Xiaohui and other technology giants, online for you to answer questions, welcome everyone to pay attention to [Didi cloud Obsuite] public number reply Kafka or nightingale plus small assistant into the group. (Note Kafka or nightingale)