Financial products rules engine from refactoring to constant iterative optimization, around the purpose is how to design more flexible and more scalable scheme, research and development of final design for product operation test role multi-level user groups, so the cost of platform lower learning and ease of use as the goal of our continuous efforts, we in order to achieve this goal in reconstruction in the course of a optimization efforts.
Warm tips: the full text is a total of more than 3800 words, it is not easy to write code words on the way to work, thank you for your attention. Draft on 23 November 2020, final draft on 28 November 2020.
Car finance | financial product center of incarnations
Financial | car GPS incarnations of the audit system
Car financial | basic data platform of incarnations
Car incarnations of financial center | contract system
Car financial | financial product rules engine incarnations (last)
Car financial | financial product rules engine incarnations (medium)
Car financial | financial product rules engine incarnations (next)
Car financial | I M in the two years
A, takeaway
The business rules of the financial product platform can be divided into three categories: financial scheme fee rules, business scene pre-and-post rules and capital access rules, which constitute the core design of the rule engine of the financial products of the auto finance business line. Financial plan cost rules provide more flexible and more financial products have extensibility of laying a foundation for financial plan, and set the rules before and after the business scenario is in order to solve the inverse calculation of specific business scenarios refused to deduction, which make up the lack of financial cost rules, the relationship between the supplement each other, common casting the core of financial product rule engine. At the same time, the capital access rules are the further abstract set of rules of the financial scheme from the capital dimension, and the common rule feature set of the differentiation of the financial scheme, which lay the core foundation for supporting the business development of the company.
The pre-rules of financial products are implemented from the original stored procedures, then transformed into Java implementation, and finally redesigned in the financial products system. Postset rules are designed to provide unified and standardized user interaction for product operators like pre-set rules, migrating from audit systems to financial products platforms. This paper mainly introduces the development process of the financial product before-and-after rule engine of auto finance business line and the realization of relevant technical solutions. Through this paper, I hope you can gain enlightenment and help you get inspiration on the road at the moment.
This article will cover the rules of the three categories of rules before and after, showing you the core details behind the evolution and introduction to the technical solution design.
The era of stored procedures
Early entry, my car’s financial business line of credit team has 13 deal with different business scenarios of the stored procedure, one of the stored procedure provides rules of RR (front-facing rules) business processing, cron timing to scheduling execution call stored procedures based on the server, screening and conform to the state of the order batch run batch, through several rule after verification, Perform pass or reject processing.
Later, I was in charge of financial products and participated in a business requirement of the capital, which required adjustment of the pre-rules. At this moment, I clearly remember that we put the adjusted stored procedure script in the test environment after verification, when the regression verification found that the execution of the batch task processing exception. In view of the objective conditions at that time, our back-end research and development could not track and troubleshoot MYSQL stored procedure abnormalities, so we had to settle for the second best and add processing to postposition rules. After all, postposition rules were implemented in The Java server, so we could barely go online to meet business requirements.
Later, the boss also realized the disadvantages of stored procedures and believed that such a technology stack would be untenable for subsequent business development. Therefore, the R&D team had to overcome all difficulties and carry out technical reconstruction in a short period of time.
In view of the mission entrusted by the boss, I led the team to start the technical transformation of 13 stored procedures successively, and realized them again by Java language (build a SpringBoot application lyQC-sieg), and then launched research and development and testing one by one, thus completely eliminating stored procedures and opening a new milestone in the reconstruction of financial product team.
Third, the necessity of real-time processing
Financial business in the second half of 2018, the car from the whole process in the related optimization efficiency of the improve, given the order entry limitation of examination and approval, expect access link can know whether to pass, because the day into the next run to batch produced results, which requires us to the front can rule by timing task run group, converted into real-time call and return immediately access the results.
Because of the initial and front rules together run batch of two other (repeatability test and channel state approval), the three run batch has the sequentiality and relevance, if one of them run batch not by words, can’t to run another batch, the temporal dependence makes us want to optimization, performance optimization for time reached a bottleneck.
Later, we communicated with the boss and the product for many times about this matter, and found that one of the longest stored procedures (repetitive inspection) can be put after the business process, without the need for access to run batch. In this way, the remaining two stored procedures are merged through the interface service logic and notified through the message mechanism. In this way, real-time processing can be realized and scheduled tasks can be retained. In case of message loss or abnormal service logic processing, batch processing can be scheduled again.
4. Rule configuration trend
This technical optimization has further improved the timeliness of business process approval, but there is still room for optimization in the technical architecture design of pre-rules. Due to the dependence of business logic processing on order-related data, we still need to modify the existing business logic code every time our business requirements involve pre-rule adjustment.
In the second half of 2018, my boss also mentioned our financial product team many times. Can the pre-rules be configured and delivered to the product operation students for use? Can we complete the requirement alignment without having to modify the code in each requirement iteration?
We did technical research and found that it could be done. We need to further clarify the system responsibilities of the rules engine, which only focuses on the definition of scene rule characteristics and interface input parameters for the scenario use of various business systems in the line of business. The business chooses according to the scenario, defines the input parameters according to our rule engine specification, and then invokes the rule engine service to perform the rule validation and return the results. In the technical scheme design, we put the pre-rule test interface required parameters, parameter coding, parameter type and a series of metadata definition, and then the interface passed in the required legal type parameters, through such data process definition, can realize the pre-rule configuration and online release free.
In the technical design, we extended a series of custom functions based on Google Aviator, an open source computing engine package, to support the needs of our business scenarios. For example, a sum of multiple numbers will run abnormally if the package itself does not pass a value for the sum associated with the number. Therefore, we have extended compatibility, and if no parameter is passed, we treat it as zero.
For the financial industry, there are many business scenarios in the round and round of numbers. We also expanded and customized related functions to assist our product operation students to use them.
In consideration of the use and learning costs of product operation students, functional interaction was optimized on the financial product platform to reduce their learning costs and avoid their configuration errors.
4. The inevitable migration of postposition rules
The post-rule was originally in the audit system maintenance, the initial technical implementation is nothing more than the use of a large number of if and else condition statement control, compared with the initial pre-rule, the advantage is that after all, it is Java language implementation, problems are easy to check, requirements change is easier to expand.
Since the front rules to complete migration to financial product configuration, an operating system platform during our many times communication with product operation classmates, interaction of platform function optimization for many times, they gradually familiar with related functions, they expect the rear can also like front rules so that they can be operated, demand demand again and again, to research and development, cycle is long, Unable to support adjustment responses to business policies.
Although the post-rule absorbed the configuration technical solution design idea of the pre-rule, it underwent a technical reconstruction to further adapt to the business development trend. The Financial Products system provides engine packages for audit teams to use, and this has been the case for quite some time.
Later, our financial products team further considered that since the audit system already relies on the financial products engine package, why not directly transplant the configuration of post-rules to the financial products platform, and the audit system only calls the interface. Therefore, after communication and discussion, we are willing to help students in product operation to realize this wish. We decide to migrate post-rules from the audit system, reuse the table structure of pre-rules and the engine logic processing module, and finally realize the visualization and operationalization of post-rules for students in product operation.
Five, the core design of the technical scheme
When we realize the design of pre-rule configuration, because it belongs to our technical team’s self-developed design, we can give full play to our innovative advantages to highlight our thinking and imagination. To be honest, I prefer independent innovation projects, free and unfettered.
There are three execution strategies for pre-rules: delete, cancel, and reject. In fact, after refactoring, we only used the rejection business scenario strategy.
The business scenario requirement logic of pre-rule, in short, is that the information of the order master and borrower or the information related to the vehicle, and the rule strategy is implemented when the pre-conditions are met and the rule characteristic conditions are not met. However, when a number of rules are hit, not only one but multiple rules are sometimes hit. In this case, the rule priority or weight is needed to solve the business scenario.
In fact, our business scenarios only have rejection scenarios, and other scenarios do not require pre-processing of rules. From the perspective of business processes, risk control rules will be pre-processing, so we only focus on the business logic test in the access link.
In this way, we don’t have to worry about the concept of priority, the internal design of the interface is to take all the rule sets from the database, and then loop through them one by one, if not satisfied, directly out of the loop and return the rule prompt text. If so, proceed to the next test. If a run-time exception occurs in any of these items, such as regular expression syntax parsing errors, or zero as a divisor, the exception is ignored and the loop continues.
In the internal design of our interface, we also provide several policy modes when parsing rules are abnormal. In this design, we absorb the rejection strategy design of Java thread pool to meet the expansion requirements of our business scenarios.
However, while we provide several policy mechanisms that we don’t actually use, we do provide extensibility. In terms of interface design, we follow a basic principle that runtime exceptions caused by product operation configuration rules will not affect the call results when the business side calls, so the simple solution is to ignore the exceptions.
Parameter metadata definition
We define the parameter metadata through the configuration center configuration as shown in the following example:
[{
"fieldName": "appCode"."fieldType": "String"."fieldDesc": "Order Number"."mustExpression": "true"."belongGroup": [
"baseInfo"] {},"fieldName": "carLoanAmount"."fieldType": "Number"."fieldDesc": "Auto loan Amount"."mustExpression": "true"."belongGroup": []}, {"fieldName": "loanRate"."fieldType": "Number"."fieldDesc": "Loan rate"."mustExpression": "true"."belongGroup": []}, {"fieldName": "evaCarPriceChange"."fieldType": "Number"."fieldDesc": "Used Car Adjustment Price"."mustExpression": "isOld ! = nil && string.contains(isOld,'1')"."belongGroup": []}]Copy the codePlatform interaction
Interface design
External API interface
@RestController
@RequestMapping("/position")
public class PositionController{
@Resource
private PositionFacadeProvider positionFacadeProvider;
@apiOperation (" Pre-rule execution interface ")
@PostMapping("/processPrePosition")
public Result<RuleCheckRe> processPrePosition(@RequestBody Map<String,Object> map){
return this.positionFacadeProvider.processPrePosition(map);
}
@apiOperation (" Post-rule execution interface ")
@PostMapping("/processPostPosition")
public Result<RuleCheckRe> processPostPosition(@RequestBody Map<String,Object> map) {
return this.positionFacadeProvider.processPostPosition(map); }}Copy the codeAs can be seen from the above, the interface input parameter is a Map, that is, a JSON object, where key is the parameter and value is the parameter value. However, the parameter and parameter value must comply with configuration constraints.
The core code
AbstractPositionRuleHandler
The core method ruleCheck
/** * external call, actuator entry *@param context
*/
public void ruleCheck(PositionRuleContext context){
try {
// 1. Initialize the object
init(context);
// 2. Verify parameters
boolean paramCheck = paramCheck();
if(! paramCheck) {return;
}
context.setAuth(true);
// 3. Load the check rule
loadCheckRule();
if(CollectionsTools.isEmpty(ruleList)) {
ruleCheckRe.setMessage("There are no rules to verify.");
return;
}
// 4. Verify rulesPositionContext positionContext = PositionContext.builder().strategy(PositionStrategy.THROWS_EXCEPTION).env(params).ruleList(ruleList).ruleCheckRe(ruleChe ckRe).classifyEnum(classifyEnum).build(); AbstractRuleCheckExecutor executor =new PositionExecutor(positionContext);
executor.execute();
} catch (Exception e) {
log.error("{} actuator entry call exception, context={}", LOG_TITLE, JSON.toJSONString(context), e);
ruleCheckRe.setAccess(false);
ruleCheckRe.setMessage(MessageFormat.format("{0} system exception", LOG_TITLE));
ruleCheckRe.setErrorRuleDesc(JSON.toJSONString(context));
context.setMessage(LOG_TITLE + "Execution exception");
} finally{ context.setRuleCheckRe(ruleCheckRe); saveLog(); }}Copy the codeThe core method checkParamsTypeLegal
/** * Check whether all input parameter types are valid *@param params
* @param checkParamsEntity
*/
private boolean checkParamsTypeLegal(Map<String, Object> params, List<AutoPositionEntity> checkParamsEntity) {
if(CollectionsTools.isEmpty(checkParamsEntity)){
return true;
}
for(AutoPositionEntity entity : checkParamsEntity) {
// Get the verification specification type
PositionParamTypeEnum positionParamTypeEnum = PositionParamTypeEnum.getByName(entity.getFieldType());
if(positionParamTypeEnum == null) {
context.setMessage(LOG_TITLE + "Parameter configuration is incorrect. Please contact technical support.");
return false;
}
Object paramsValue = params.get(entity.getFieldName());
AviatorContext ctx = AviatorContext.builder().expression(entity.getMustExpression()).env(params).build();
try {
boolean must = AviatorExecutor.executeBoolean(ctx);
if(must && paramsValue == null) {
context.setMessage(MessageFormat.format("Request parameters: {0}, cannot be null", entity.getFieldName()));
return false; }}catch (Exception e) {
context.setMessage(LOG_TITLE + "Abnormal configuration of mandatory field attribute, please contact technical support.");
log.error("{} , appCode:{}", LOG_TITLE, appCode, e);
return false;
}
if(paramsValue ! =null) {
String paramsType = paramsValue.getClass().getTypeName();
boolean checkType = positionParamTypeEnum.check(paramsValue);
if(! checkType) { context.setMessage(MessageFormat.format("{0} request parameter is invalid, parameter name: {1}, input parameter type: {2}, verification type: {3}", LOG_TITLE, entity.getFieldName(), paramsType, positionParamTypeEnum));
return false;
}
// If the value is an integer, the calculation engine will have the problem of rounding down, the conversion will be converted to a decimal, the String type can not be converted, for the sake of unityparams.put(entity.getFieldName(), positionParamTypeEnum.convert(paramsValue)); }}return true;
}
Copy the codeAbstractRuleCheckExecutor
Core method Call
/** * call processing */
protected void call(a) {
AutoApprConstant.OpIdnEnum opIdnEnum;
try {
for (AutoApprRule ruleEntity : ruleList) {
this.executeRule = false;
this.ruleEntity = ruleEntity;
List<AutoApprRuleProp> propList = ruleEntity.getProps();
for (AutoApprRuleProp propEntity : propList) {
try {
boolean currentMatch;
opIdnEnum = AutoApprConstant.OpIdnEnum.getByName(propEntity.getOpIdn());
if (Objects.equals(AutoApprConstant.PropFieldExtendEnum.EXPRESSION.getName(), propEntity.getPropCode())) {
currentMatch = access(propEntity.getPropValue());
} else {
AutoApprConstant.PropFieldTypeEnum valueTypeEnum = AutoApprConstant.PropFieldTypeEnum.getByIndex(propEntity.getPropValueType());
currentMatch = OpSymbols.builder().env(env).opIdnEnum(opIdnEnum).valueTypeEnum(valueTypeEnum)
.propCode(propEntity.getPropCode()).propValue(propEntity.getPropValue()).build().access();
}
if(! currentMatch) { executeRule =false;
break;
} else {
executeRule = true; }}catch (Exception e) {
errorRuleDesc.add(MessageFormat.format("Rule [{0}] property configuration error", ruleEntity.getRuleId()));
executeRule = false;
strategyHandle.handle(e, this, ruleEntity);
break; }}boolean complete = after();
if(complete) {
return; }}if(! ruleCheckRe.isAccess()) { ruleCheckRe.setCalculateResult(calculateResult);if(AutoApprConstant.BelongNameEnum.getByIndex(ruleCheckRe.getBelongType()) == AutoApprConstant.BelongNameEnum.COMPUTE && CollectionsTools.isEmpty(calculateResult)) {
ruleCheckRe.setBelongType(AutoApprConstant.BelongNameEnum.CHECK.getIndex());
}
}
ruleCheckRe.setErrorRuleDesc(errorRuleDesc.stream().collect(Collectors.joining(",")));
} catch (PositionStrategyException e) {
log.error({} exception, ruleIdList={}", LOG_TITLE, JSON.toJSONString(errorRuleDesc), e);
ruleCheckRe.setAccess(false);
ruleCheckRe.setErrorRuleDesc(errorRuleDesc.stream().collect(Collectors.joining(","))); ruleCheckRe.setBelongType(AutoApprConstant.BelongNameEnum.CHECK.getIndex()); ruleCheckRe.setMessage(e.getMessage()); }}Copy the codeCore method after
/** ** post-processing */
protected boolean after(a) throws PositionStrategyException {
AutoApprConstant.BelongNameEnum belongNameEnum = Optional.ofNullable(AutoApprConstant.BelongNameEnum.getByName(ruleEntity.getBelongName()))
.orElseThrow(() -> new PositionStrategyException(MessageFormat.format("Rule [{0}] type configuration error, rule type ={1}", ruleEntity.getRuleId(), ruleEntity.getBelongName())));
if (executeRule) {
ruleCheckRe.setAccess(false);
ruleCheckRe.setRuleId(ruleEntity.getRuleId());
ruleCheckRe.setBelongType(belongNameEnum.getIndex());
switch (belongNameEnum) {
case CHECK:
ruleCheckRe.setMessage(ruleEntity.getMsgTemplate());
return true;
case COMPUTE:
calculateResult = Optional.ofNullable(calculateResult).orElse(Maps.newHashMapWithExpectedSize(10)); AutoApprFormulaBo autoApprFormulaBo = AutoApprFormulaBo.builder().ruleId(ruleEntity.getRuleId()) .type(AutoApprFormulaTypeEnum.BACK_CALCULATION.getIndex()).params(env).messageTemplate(ruleEntity.getMsgTemplate()).clas sifyEnum(classifyEnum).build(); List<RuleCalculateRe> calculateReList = autoApprFormulaComponent.calculateResult(autoApprFormulaBo);if(CollectionsTools.isNotEmpty(calculateReList)) {
for(RuleCalculateRe calculateRe : calculateReList){
RuleCalculateRe ruleCalculateRe = calculateResult.get(calculateRe.getFieldName());
BigDecimal calculateReValue = calculateRe.getValue();
if(ruleCalculateRe == null || calculateReValue.compareTo(ruleCalculateRe.getValue()) < 0) { ruleCheckRe.setMessage(ruleEntity.getMsgTemplate()); calculateResult.put(calculateRe.getFieldName(), calculateRe); }}}else {
if(StringTools.isEmpty(ruleCheckRe.getMessage())) {
ruleCheckRe.setMessage(ruleEntity.getMsgTemplate());
}
errorRuleDesc.add(MessageFormat.format("Rule [{0}] missing backcalculation configuration", ruleEntity.getRuleId()));
}
return false;
default:
throw new PositionStrategyException(MessageFormat.format("Rule [{0}] type configuration error, rule type ={1}", ruleEntity.getRuleId(), belongNameEnum.getName())); }}return false;
}
Copy the codePositionStrategyHandle
PositionStrategyHandle
public interface PositionStrategyHandle {
/** * handle *@paramThrowable exception *@paramExecutor Executor object *@paramRuleEntity ruleEntity object */
void handle(Throwable throwable, AbstractRuleCheckExecutor executor, AutoApprRule ruleEntity) throws PositionStrategyException;
}
Copy the codeThrowsExceptionStrategyHandle
/ * * *@description: When multiple rules are executed, if a runtime exception occurs in the current rule execution, the exception is thrown *@Date: 2018/12/12 2:11 PM *@Author: Shi Dongdong -Seig Heil */
@Slf4j
public class ThrowsExceptionStrategyHandle implements PositionStrategyHandle {
final String pattern = "Internal error in rule [{0}], rule name ={1}";
@Override
public void handle(Throwable throwable, AbstractRuleCheckExecutor executor, AutoApprRule ruleEntity) throws PositionStrategyException {
String message = MessageFormat.format(pattern, ruleEntity.getRuleId(), ruleEntity.getRuleName());
log.error("{0},ruleEntity={1}", message, JSON.toJSON(ruleEntity), throwable);
throw newPositionStrategyException(message, throwable); }}Copy the codeStrictExecuteStrategyHandle
/ * * *@description: Strictly enforce all rules *@Date: 2018/12/12 2:11 PM *@Author: Shi Dongdong -Seig Heil */
public class StrictExecuteStrategyHandle extends ThrowsExceptionStrategyHandle implements PositionStrategyHandle {
@Override
public void handle(Throwable throwable, AbstractRuleCheckExecutor executor, AutoApprRule ruleEntity) throws PositionStrategyException {
super.handle(throwable,executor,ruleEntity); }}Copy the codeIgnoreExceptionStrategyHandle
/ * * *@description: When multiple rules are executed, if a runtime exception occurs during the current rule execution, the execution is ignored@Date: 2018/12/12 2:11 PM *@Author: Shi Dongdong -Seig Heil */
@Slf4j
public class IgnoreExceptionStrategyHandle implements PositionStrategyHandle {
@Override
public void handle(Throwable throwable, AbstractRuleCheckExecutor executor, AutoApprRule ruleEntity) {
log.warn(RuleEntity ={0}" ruleEntity={0}", JSON.toJSON(ruleEntity),throwable); }}Copy the codePositionStrategyHandleFactory
/ * * *@description: Put the rule policy processing interface factory class *@Date: 2018/12/11 4:08 PM *@Author: Shi Dongdong -Seig Heil */
public final class PositionStrategyHandleFactory {
/** * factory method *@param strategy
* @return* /
public static PositionStrategyHandle create(PositionStrategy strategy){
switch (strategy){
case STRICT_EXECUTE:
return new StrictExecuteStrategyHandle();
case IGNORE_EXCEPTION:
return new IgnoreExceptionStrategyHandle();
case THROWS_EXCEPTION:
return new ThrowsExceptionStrategyHandle();
default:
throw new IllegalArgumentException("UNKNOWN strategy="+strategy.toString()); }}}Copy the code6. Backcalculation mechanism of postposition rules
Later, with the development of business scenarios, in order to improve the user interaction experience, we developed and designed a reverse calculation mechanism.
To be honest, if several input parameters match a configuration, we expect not to return a message, but to guide the user to adjust the size of the relevant input to meet the rules of the business scenario. The purpose of this is to keep your business process friendly.
Seven, tail
After several reconstructions, the configuration of pre-rules and post-rules in the financial product platform is realized. Not only do we free up developers from developing for stored procedures or SQL; Easier for product people, who can quickly support business adjustments. We have realized the configuration and visualization of the pre-and-post rules of financial products, which is a qualitative change.
