In the second-hand car business line, the application of vehicle, person and dealer information in the business audit process cannot be realized at the present stage. Therefore, in order to achieve this goal, the business side comes up with the relationship mapping service based on data collection and data query.
This paper is divided into two parts because of too many words.
Technical scheme design of relational Graph service – Part I
3.2 Data check
Above is a UML class diagram of the entire code design, as can be seen from the above diagram:
- 1. The “API” layer of the first layer provides HTTP interfaces externally.
- The second layer of “query executor” wraps the API layer’s requests and hands them off to the query handler.
- 3, the third layer of “query processor”, abstract class only derived a business scenario multi-condition query processor
MultiSearchHandler. - 3. The “hit query processor” of the fourth layer is derived from three subclasses according to business scenarios
PersonInfoHitQuerier,VehicleInfoHitQuerier,DealerInfoHitQuerier, and provides two data hit modes at the same timeExactSearchMode,SimilarSearchMode.
3.2.1, MultiSearchRequest
/ * * *@description: Multi-condition replay query condition DTO *@Date : 2020/12/15 3:17 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class MultiSearchRequest {
@APIModelProperty (value=" request ID(can be generated using UUID)",dataType=" java.lang.string ")
@notnull (message = "requestId [requestId] not empty ")
private String requestId;
@apiModelProperty (value=" List
",dataType="List
")
@notnull (message = "conditions NotNull ")
private List<Condition> conditions;
/ * * *@description: Query condition *@Date : 2020/12/17 3:42 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class Condition{
@APIModelProperty (value=" query field type ",dataType="SourceTypeEnum")
private SourceTypeEnum sourceType;
@APIModelProperty (value=" query field name ",dataType=" java.lang.string ")
@notnull (message = "Query field name [searchFieldName] cannot be empty!" )
@notempty (message = "Query field name [searchFieldName] cannot be empty!" )
private String searchFieldName;
@APIModelProperty (value=" fuzzy query score interval ",dataType=" java.lang.double ")
@size (Max = 2, message = "query field name [scoreRange] list length should be 0-2")
private List<Double> scoreRange;
@APIModelProperty (value=" query field input value ",dataType=" java.lang.string ")
@notnull (message = "searchFieldValue cannot be empty!" )
@notempty (message = "searchFieldValue cannot be empty!" )
private String searchFieldValue;
@APIModelProperty (value=" query field description ",dataType=" java.lang.string ")
@notnull (message = "Query field description [searchFieldDesc] cannot be empty!" )
@notempty (message = "Query field description [searchFieldDesc] cannot be empty!" )
privateString searchFieldDesc; }}/ * * *@description: Type of source (1- principal lender; 2 - sales; 3- Sales supervisor; 4 - spouse; 5- Guarantor; 6- Used car seller; 7- Emergency contact 1; 8- Emergency contact 2; 9- Legal person of automobile dealer; 10- Person in charge of car dealer; 11- Payee of vehicle dealer; 12 - vehicles;) *@Date : 2020/11/27 4:08 PM
* @Author: Shi Dongdong -Seig Heil */
public enum SourceTypeEnum implements EnumValue {
CREDITOR_INFO(1."Principal lender"),
SELLER_INFO(2."Sales"),
SALES_MANAGER_INFO(3."Sales executive"),
MATE(4."Spouse"),
GUARANTOR_INFO(5."Guarantor"),
SELLER_INFO_OF_OLD_CAR(6."Used Car seller"),
EMERGENCY_CONTACT_ONE(7."Emergency Contact 1"),
EMERGENCY_CONTACT_TWO(8."Emergency Contact 2."),
CAR_DEALER_LEGAL_PERSON(9."Car dealer corporation."),
CAR_DEALER_PRINCIPAL(10."Car dealer in Charge."),
CAR_DEALER_PAYEE(11."Car dealer payee"),
VEHICLE_INFO(12."Vehicle"),;private int index;
private String value;
SourceTypeEnum(int index, String value ){
this.value = value;
this.index = index;
}
@Override
public int getIndex(a) {
return index;
}
@Override
public String getName(a) {
return value;
}
/** * get object * by index@param index
* @return* /
public static SourceTypeEnum getByIndex(int index){
return Stream.of(SourceTypeEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
}
/** * get the name by index *@param index
* @return* /
public static String getNameByIndex(int index){
SourceTypeEnum find = Stream.of(SourceTypeEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
return null == find ? "": find.getName(); }}Copy the code3.2.2, SearchResultRe
/ * * *@description: Query result *@Date : 2020/12/15 3:29 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class SearchResultRe<E extends SearchResultRe.Record>{
@APIModelProperty (value=" prompt ",dataType=" java.lang.string ")
private String message;
@APIModelProperty (value=" query field name ",dataType=" java.lang.string ")
private String searchFieldName;
@APIModelProperty (value=" query field input value ",dataType=" java.lang.string ")
private String searchFieldValue;
@APIModelProperty (value=" query field description ",dataType=" java.lang.string ")
private String searchFieldDesc;
@APIModelProperty (value=" number of hits ",dataType=" java.lang.integer ")
private int hitCount;
@apiModelProperty (value=" hit Record ",dataType="List
")
private List<E> hitRecords;
/** * Add the number of hits *@param delta
*/
public synchronized void increaseHitCount(int delta){
this.hitCount += delta;
}
/** * Add hit records *@param hits
*/
public synchronized void addHits(List<E> hits){
if(! isEmpty(hits)){if(isEmpty(this.hitRecords)){
this.hitRecords = new ArrayList<>(hits.size());
}
this.hitRecords.addAll(hits); }}/** * determines whether the collection element is empty *@param collection
* @return* /
boolean isEmpty(Collection
collection){
return null == collection || collection.isEmpty();
}
/ * * *@description: Query result *@Date : 2020/12/15 3:29 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@SuperBuilder
@NoArgsConstructor
@AllArgsConstructor
public static class Record{
@ ApiModelProperty (value = "primary key ID", dataType = "Java. Lang. Integer")
private Integer id;
@APIModelProperty (value=" external data ID ",dataType=" java.lang.string ")
private String externalId;
@APIModelProperty (value=" order number ",dataType=" java.lang.string ")
private String appCode;
@APIModelProperty (value=" dataType ",dataType=" java.lang.string ")
private String dataCode;
/ * * * {@link SceneEnum#getIndex()}
*/
@ ApiModelProperty (value = "scene", dataType = "Java. Lang. Integer")
private Integer scene;
private String sceneDesc;
/ * * * {@link SourceTypeEnum#getIndex()}
*/
@APIModelProperty (value=" source type ",dataType=" java.lang.integer ")
private Integer sourceType;
private String sourceTypeDesc;
}
/ * * *@description: Natural person information *@Date : 2020/12/15 3:29 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@SuperBuilder
@NoArgsConstructor
public static class PersonInfoRecord extends Record{
@ ApiModelProperty (value = "name", dataType = Java. Lang. "String")
private String name;
@APIModelProperty (value=" ID number ",dataType=" java.lang.string ")
private String idNo;
@APIModelProperty (value=" phone number ",dataType=" java.lang.string ")
private String mobile;
@APIModelProperty (value=" bank card number ",dataType=" java.lang.string ")
private String creditCardNo;
@ ApiModelProperty (value = "province", dataType = "Java. Lang. String")
private String provinceAddress;
@ ApiModelProperty (value = "city", dataType = Java. Lang. "String")
private String cityAddress;
@ ApiModelProperty (value = "area", dataType = "Java. Lang. String")
private String districtAddress;
@APIModelProperty (value=" detailed address ",dataType=" java.lang.string ")
private String detailAddress;
}
/ * * *@description: Vehicle information *@Date : 2020/12/15 3:29 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@SuperBuilder
@NoArgsConstructor
public static class VehicleInfoRecord extends Record{
@ ApiModelProperty (value = "VIN code", dataType = Java. Lang. "String")
private String vin;
@APIModelProperty (value=" km ",dataType=" java.lang.integer ")
private Integer mileage;
@APIModelProperty (value=" evaluation remarks ",dataType=" java.lang.string ")
privateString evaluateRemark; }}Copy the code3.2.3, SearchController
/ * * *@description: Query data *@Date : 2020/12/15 4:03 PM
* @Author: Shi Dongdong -Seig Heil */
@RestController
@api (description = "query data ", tags =" query data ")
@RequestMapping("/search")
public class SearchController {
@Autowired
SearchQueryExecutor searchQueryExecutor;
/** **@return* /
@PostMapping("/multiQuery")
@apiOperation (value = "ApiOperation ", notes =" ApiOperation ")
@NoAuthRequired
@ovalvalidator (value = "multiQuery ")
public Result<List<SearchResultRe>> multiQuery(@RequestBody MultiSearchRequest request) {
returnsearchQueryExecutor.execute(SearchQueryExecutor.Type.MULTI,request); }}Copy the code3.2.4, SearchQueryExecutor
/ * * *@description: Queries the actuator *@Date : 2020/12/16 11:27 AM
* @Author: Shi Dongdong -Seig Heil */
@Component
public class SearchQueryExecutor {
final Map<Type,AbstractSearchHandler> HANDLER_MAP = Maps.newHashMap();
@Resource
MultiSearchHandler multiSearchHandler;
@PostConstruct
void init(a){
HANDLER_MAP.put(Type.MULTI, multiSearchHandler);
}
public enum Type{
/** ** */
SIMPLE,
/** ** ** /
MULTI
}
/** * execute *@param type
* @param request
* @param <T>
*/
public <T> Result<List<SearchResultRe>> execute(Type type, T request){
if(Type.MULTI == type){
SearchQueryContext<MultiSearchRequest> context = SearchQueryContext.<MultiSearchRequest>builder().param((MultiSearchRequest)request).build();
HANDLER_MAP.get(type).execute(context);
return Result.suc(context.getResults());
}
returnResult.suc(); }}Copy the code3.2.5, SearchQueryContext
/ * * *@description: data lookup context object *@Date : 2020/12/16 11:06 AM
* @Author: Shi Dongdong -Seig Heil */
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class SearchQueryContext<T> {
/** * request ID */
private String requestId;
/** ** query condition */
private T param;
/** * Query result */
private List<SearchResultRe> results;
/** * Check whether the query is successful */
private boolean success;
/** * Business information */
private String message;
public SearchQueryContext withSuccess(boolean success,String message){
this.setSuccess(success);
this.setMessage(message);
return this; }}Copy the code3.2.6, AbstractSearchHandler
/ * * *@descriptionAbstract query handler *@Date : 2020/12/16 11:08 AM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
public abstract class AbstractSearchHandler<T> {
/** * context container */
final ThreadLocal<SearchQueryContext<T>> CONTEXT = new ThreadLocal<>();
@Resource
DiamondConfigProxy diamondConfigProxy;
@Resource
HitQuerierManager hitQuerierManager;
@Resource
ExecutorService searchQueryPoolExecutor;
/** * External call execution method *@param context
*/
public void execute(SearchQueryContext<T> context){
try {
CONTEXT.set(context);
doQuery(context);
} catch (Exception e) {
log.error("[execute],requestId={},ctx={}",context.getRequestId(),JSONObject.toJSONString(context),e);
}finally{ CONTEXT.remove(); }}/** * Execute query *@param context
*/
abstract void doQuery(SearchQueryContext<T> context);
/** * Build an exception query result *@param condition
* @param message
* @return* /
SearchResultRe buildExceptionSearchResult(MultiSearchRequest.Condition condition,String message){
returnSearchResultRe.builder() .searchFieldName(condition.getSearchFieldName()) .searchFieldValue(condition.getSearchFieldValue()) .searchFieldDesc(condition.getSearchFieldDesc()) .message(message) .hitRecords(Collections.emptyList()) .build(); }}Copy the code3.2.7, MultiSearchHandler
Multi-condition query processor, here the use of multithreading, to the dimension of conditions, to the child thread processing, the main thread blocking waiting for all child threads processing, and the hit data results unified encapsulation return.
/ * * *@descriptionAbstract query handler *@Date : 2020/12/16 11:08 AM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class MultiSearchHandler extends AbstractSearchHandler<MultiSearchRequest> {
@Override
void doQuery(SearchQueryContext<MultiSearchRequest> context) {
context.setRequestId(context.getParam().getRequestId());
String requestId = context.getRequestId();
// List of conditions
List<MultiSearchRequest.Condition> conditions = context.getParam().getConditions();
// Result list
List<SearchResultRe> results = Lists.newArrayListWithExpectedSize(conditions.size());
//Future task list
List<Future<HitQuerierContext>> futureList = new ArrayList<>(conditions.size());
for(MultiSearchRequest.Condition condition : conditions){
// Validate parameters
if(! checkCondition(condition)){ results.add(super.buildExceptionSearchResult(condition, MessageFormat.format(Abnormal | "query parameter is illegal, field = {0}",condition.getSearchFieldName())));
continue;
}
Future<HitQuerierContext> future = searchQueryPoolExecutor.submit(() -> {
HitQuerierContext hitQuerierContext = HitQuerierContext.builder()
.requestId(requestId)
.condition(condition)
.build();
hitQuerierManager.execute(hitQuerierContext);
return hitQuerierContext;
});
futureList.add(future);
}
int count = 0;
for (Future<HitQuerierContext> f : futureList) {
try {
HitQuerierContext hitQuerierContext = f.get();
results.add(hitQuerierContext.getHitResult());
} catch (InterruptedException | ExecutionException e) {
log.error("[doQuery][InterruptedException|ExecutionException],requestId={},context={}",requestId, JSONObject.toJSON(context),e);
results.add(super.buildExceptionSearchResult(conditions.get(count), MessageFormat.format("Query exception | a runtime exception, a message = {0}",e.getMessage())));
continue;
}
count++;
}
context.setResults(results);
context.withSuccess(Boolean.TRUE,"Query successful");
}
/** * Parameter validation *@param condition
*/
boolean checkCondition(MultiSearchRequest.Condition condition){
Set<String> configFields = diamondConfigProxy.searchFieldConfig().keySet();
log.info("[checkCondition],configFields={}", configFields.toArray());
String searchFieldName = condition.getSearchFieldName();
returnconfigFields.contains(searchFieldName); }}Copy the code3.2.8, HitQuerierManager
Hit the query manager, because a query condition will go to multiple tables for data query, and then data merge, so here we register the query to the hitList, each request will call the execute method loop to iterate over all the query processing.
/ * * *@description: Matches the query manager *@Date : 2020/12/17 4:21 PM
* @Author: Shi Dongdong -Seig Heil */
@Component
public class HitQuerierManager {
static final List<AbstractHitQuerier> hitList = new ArrayList<>();
@Resource
PersonInfoHitQuerier personInfoHitQuerier;
@Resource
DealerInfoHitQuerier dealerInfoHitQuerier;
@Resource
VehicleInfoHitQuerier vehicleInfoHitQuerier;
@PostConstruct
void init(a){
hitList.add(personInfoHitQuerier);
hitList.add(dealerInfoHitQuerier);
hitList.add(vehicleInfoHitQuerier);
}
/** * Execute query *@param context
*/
public void execute(HitQuerierContext context){
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = SearchResultRe.builder()
.searchFieldValue(condition.getSearchFieldValue())
.searchFieldName(condition.getSearchFieldName())
.searchFieldDesc(condition.getSearchFieldDesc())
.message("Query successful")
.build();
for(AbstractHitQuerier querier : hitList){ SearchResultRe hit = querier.execute(context); hitResult.increaseHitCount(hit.getHitCount()); hitResult.addHits(hit.getHitRecords()); } context.setHitResult(hitResult); }}Copy the code3.2.9, HitQuerierContext
/ * * *@description: Data query context *@Date : 2020/12/17 3:38 PM
* @Author: Shi Dongdong -Seig Heil */
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class HitQuerierContext {
/** * request ID */
private String requestId;
/** ** query condition */
private MultiSearchRequest.Condition condition;
/** * Query result */
private SearchResultRe hitResult;
}
Copy the code3.2.10, AbstractHitQuerier
/ * * *@description: Abstract match query *@Date : 2020/12/17 3:38 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
public abstract class AbstractHitQuerier<E extends SearchResultRe.Record> {
/** * handle query table */
protected final SearchFieldConfig.Table table;
@Resource
DiamondConfigProxy diamondConfigProxy;
public AbstractHitQuerier(SearchFieldConfig.Table table) {
this.table = table;
}
/** * query *@param context
* @return* /
abstract SearchResultRe<E> doQuery(HitQuerierContext context);
/** * external execution method *@param context
* @return* /
public SearchResultRe execute(HitQuerierContext context){
SearchResultRe searchResultRe;
try {
if(! executeCurrent(context)){ searchResultRe = buildDefaultSearchResult(context.getCondition());return searchResultRe;
}
searchResultRe = doQuery(context);
} catch (Exception e) {
log.error("[execute],requestId={},ctx={}",context.getRequestId(),JSONObject.toJSONString(context),e);
searchResultRe = buildDefaultSearchResult(context.getCondition());
searchResultRe.setMessage("Query exception | message =" + e.getMessage());
}
return searchResultRe;
}
/** * Whether to execute the current query *@param context
* @return* /
protected boolean executeCurrent(HitQuerierContext context){
return getSearchFieldConfig(context.getCondition().getSearchFieldName()).getTables().contains(table);
}
/** * Build the default query result *@param condition
* @return* /
SearchResultRe buildDefaultSearchResult(MultiSearchRequest.Condition condition){
String fieldName = condition.getSearchFieldName();
String fieldValue = condition.getSearchFieldValue();
SearchResultRe hitResult = SearchResultRe.builder()
.searchFieldName(fieldName)
.searchFieldValue(fieldValue)
.hitCount(0)
.hitRecords(Collections.emptyList())
.build();
return hitResult;
}
/** * Get the configuration field *@param fieldName
* @return* /
SearchFieldConfig getSearchFieldConfig(String fieldName){
Map<String,SearchFieldConfig> mapping = diamondConfigProxy.searchFieldConfig();
SearchFieldConfig searchFieldConfig = mapping.get(fieldName);
log.info("[getSearchFieldConfig],fieldName={},config={}",fieldName, JSONObject.toJSONString(searchFieldConfig));
return searchFieldConfig;
}
/** * process hit data *@param context
* @param queryForm
* @param recordsCaller
* @return* /
List<E> hitRecords(HitQuerierContext context, Object queryForm, BiFunction<String,Object,List<E>> recordsCaller){
MultiSearchRequest.Condition condition = context.getCondition();
SearchFieldConfig searchFieldConfig = getSearchFieldConfig(condition.getSearchFieldName());
SearchFieldConfig.SearchMode searchMode = searchFieldConfig.getSearchMode();
returnAbstractSearchMode.getAbstractSearchMode(searchMode).execute(searchFieldConfig, context, queryForm, recordsCaller); }}/ * * *@description: Auto dealer information matching query *@Date : 2020/12/17 3:51 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class DealerInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.PersonInfoRecord> {
public DealerInfoHitQuerier(a) {
super(SearchFieldConfig.Table.DEALER_INFO);
}
@Resource
DealerInfoService dealerInfoService;
@Override
SearchResultRe<SearchResultRe.PersonInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
DealerInfoForm queryForm = DealerInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.PersonInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<DealerInfo> recordList = dealerInfoService.queryList((DealerInfoForm)searchForm);
SearchFieldConfig config = getSearchFieldConfig(condition.getSearchFieldName());
if(SearchFieldConfig.SearchMode.SIMILAR == config.getSearchMode() && condition.getSearchFieldName().contains("Address")){
String provinceAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String cityAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String districtAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String detailAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
return BeanConverter.convertFromDealer(recordList,
each -> BeanTool.getObjectValue(each, provinceAddressFiledName),
each -> BeanTool.getObjectValue(each, cityAddressFiledName),
each -> BeanTool.getObjectValue(each, districtAddressFiledName),
each -> BeanTool.getObjectValue(each, detailAddressFiledName));
}
return BeanConverter.convertFromDealer(recordList,
null.null.null.null);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
returnhitResult; }}/ * * *@description: Natural person information matches the query *@Date : 2020/12/17 3:51 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class PersonInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.PersonInfoRecord> {
/** * Enter */ for the mobile number field
static final String MOBILE_FIELD = "mobile";
/** * Address class field name suffix */
static final String ADDRESS_FIELD_SUFFIX = "Address";
public PersonInfoHitQuerier(a) {
super(SearchFieldConfig.Table.PERSON_INFO);
}
@Resource
PersonInfoService personInfoService;
@Override
SearchResultRe<SearchResultRe.PersonInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
PersonInfoForm queryForm = PersonInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.PersonInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<PersonInfo> recordList = personInfoService.queryList((PersonInfoForm)searchForm);
if(MOBILE_FIELD.equals(condition.getSearchFieldName())){
return BeanConverter.convertFromPerson(recordList, each -> BeanTool.getObjectValue(each, fieldName),
null.null.null.null);
}
SearchFieldConfig config = getSearchFieldConfig(condition.getSearchFieldName());
if(SearchFieldConfig.SearchMode.SIMILAR == config.getSearchMode() && condition.getSearchFieldName().contains(ADDRESS_FIELD_SUFFIX)){
String provinceAddressFiledName = config.getMapping().get(0);
String cityAddressFiledName = config.getMapping().get(1);
String districtAddressFiledName = config.getMapping().get(2);
String detailAddressFiledName = config.getMapping().get(3);
return BeanConverter.convertFromPerson(recordList, null,
each -> BeanTool.getObjectValue(each, provinceAddressFiledName),
each -> BeanTool.getObjectValue(each, cityAddressFiledName),
each -> BeanTool.getObjectValue(each, districtAddressFiledName),
each -> BeanTool.getObjectValue(each, detailAddressFiledName));
}
return BeanConverter.convertFromPerson(recordList);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
returnhitResult; }}/ * * *@description: Vehicle information matching query *@Date : 2020/12/17 3:51 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class VehicleInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.VehicleInfoRecord> {
public VehicleInfoHitQuerier(a) {
super(SearchFieldConfig.Table.VEHICLE_INFO);
}
@Resource
VehicleInfoService vehicleInfoService;
@Override
SearchResultRe<SearchResultRe.VehicleInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
VehicleInfoForm queryForm = VehicleInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.VehicleInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<VehicleInfo> recordList = vehicleInfoService.queryList((VehicleInfoForm)searchForm);
return BeanConverter.convertFromVehicle(recordList);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
returnhitResult; }}Copy the code3.2.11, AbstractSearchMode
/ * * *@description: Matching query mode *@see com.creditease.horus.core.search.querier.mode.impl.ExactSearchMode
* @see com.creditease.horus.core.search.querier.mode.impl.SimilarSearchMode
* @Date : 2020/12/24 3:27 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
public abstract class AbstractSearchMode {
/** * Policy set */
public static Map<SearchFieldConfig.SearchMode, AbstractSearchMode> abstractSearchModeMap = new HashMap<>();
/** * Select policy *@param modeEnum
* @return* /
public static AbstractSearchMode getAbstractSearchMode(SearchFieldConfig.SearchMode modeEnum){
return abstractSearchModeMap.get(modeEnum);
}
/** * policy abstract method *@param searchFieldConfig
* @param context
* @param queryForm
* @param recordsCaller
* @param <E>
* @return* /
public abstract <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig, HitQuerierContext context, Object queryForm, BiFunction
> recordsCaller)
,>;
/** * Copy configured extension parameters to entity class *@param searchFieldConfig
* @param queryForm
*/
public void copyExclude (SearchFieldConfig searchFieldConfig, Object queryForm){
Map<String,Object> exclude = searchFieldConfig.getExclude();
if(MapUtils.isNotEmpty(exclude)){ BeanTool.copyFromOneMap(exclude,queryForm); }}}/ * * *@description: Matching query mode - Precise processing mode *@Date : 2020/12/24 3:28 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class ExactSearchMode extends AbstractSearchMode {
{
abstractSearchModeMap.put(SearchFieldConfig.SearchMode.EXACT, this);
}
/** **@param searchFieldConfig
* @param context
* @param queryForm
* @param recordsCaller
* @return* /
@Override
public <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig, HitQuerierContext context, Object queryForm, BiFunction
> recordsCaller)
,> {
// Copy the configured extension parameters to the entity class
copyExclude (searchFieldConfig, queryForm);
// Add parameters
String requestId = context.getRequestId();
String fieldName = context.getCondition().getSearchFieldName();
String fieldValue = context.getCondition().getSearchFieldValue();
/ / the result set
List<E> hitRecords;
// Valid query parameters
Map<String,Object> sourceValues = Maps.newHashMap();
/ / into the and a parameter mapping query two parameters (into the corresponding database primaryMobile mobile, SecondMobile)
List<String> mapping = searchFieldConfig.getMapping();
if(null! = mapping && ! mapping.isEmpty()){ hitRecords = Lists.newArrayList();for(String fieldNameAlias : mapping){
sourceValues.put(fieldNameAlias,fieldValue);
// copy the map key-value pair to the entity class
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId, JSONObject.toJSONString(queryForm));
List<E> hitRecordsTemp = recordsCaller.apply(fieldNameAlias,queryForm);
log.info("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecordsTemp.size());
if(CollectionsTools.isNotEmpty(hitRecordsTemp)){
hitRecords.addAll(hitRecordsTemp);
}
// Delete this parameter (set this parameter to null and copy it to the query entity class)
sourceValues.put(fieldNameAlias,null); }}else{
sourceValues.put(fieldName,fieldValue);
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId,JSONObject.toJSONString(queryForm));
hitRecords = recordsCaller.apply(fieldName,queryForm);
log.info("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecords.size());
}
returnhitRecords; }}/ * * *@description: Matching query mode - Similarity processing mode *@Date : 2020/12/24 3:28 PM
* @Author: Shi Dongdong -Seig Heil */
@Slf4j
@Component
public class SimilarSearchMode extends AbstractSearchMode {
/** ** ** /
final String SIMILAR_DETAIL_FIELD = "detail";
/** * output model, address field name */
final String OUT_MODEL_PROVINCE_ADDRESS = "provinceAddress";
final String OUT_MODEL_CITY_ADDRESS = "cityAddress";
final String OUT_MODEL_DISTRICT_ADDRESS = "districtAddress";
final String OUT_MODEL_DETAIL_ADDRESS = "detailAddress";
/** * Similarity query type */
final String SIMILAR_TYPE = "m:organization.organization.name";
{
abstractSearchModeMap.put(SearchFieldConfig.SearchMode.SIMILAR, this);
}
/** * similarity query *@param context
* @param queryForm
* @param recordsCaller
* @return* /
@Override
public <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig, HitQuerierContext context, Object queryForm, BiFunction
> recordsCaller)
,> {
// Copy the configured extension parameters to the entity class
copyExclude (searchFieldConfig, queryForm);
// Add parameters
String requestId = context.getRequestId();
String fieldName = context.getCondition().getSearchFieldName();
String fieldValue = context.getCondition().getSearchFieldValue();
// Split the address into province, city, region, and details.
List<String> mapping = searchFieldConfig.getMapping();
String[] fieldValues = fieldValue.split("\ \ |");
// The name of the field in which the word is compared
String similarField = mapping.get(mapping.size()-1);
// The detailed address sent by the user
String addressDetail = fieldValues[fieldValues.length-1];
HsmmAddressNormalizer anm = new HsmmAddressNormalizer();
String addressDetailFormat = fieldValues[0] + fieldValues[1] + fieldValues[2]
+ ( (HashMap<String,String>)anm.splitAddress(addressDetail) ).get(SIMILAR_DETAIL_FIELD);
// Valid query parameters
Map<String,Object> sourceValues;
/ / the result set
List<E> hitRecords = Collections.emptyList();
if(CollectionUtils.isEmpty(mapping)){
log.warn("[execute]diamond mapping is null,requestId={},fieldName={}",requestId,fieldName);
return hitRecords;
}
sourceValues = Maps.newHashMap();
String[] mappingValues = mapping.toArray(new String[mapping.size()]);
// The last item is not used as a query condition
for (int i = 0; i < mappingValues.length - 1; i++) {
sourceValues.put(mappingValues[i],fieldValues[i]);
}
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId, JSONObject.toJSONString(queryForm));
List<E> hitRecordsTemp = recordsCaller.apply(fieldName,queryForm);
log.debug("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecordsTemp.size());
if(CollectionsTools.isEmpty(hitRecordsTemp)){
return hitRecords;
}
[Score interval is empty - no score requirement; score interval length is 1- minimum score requirement; score interval length is 2- score interval requirement]
List<Double> scoreRangeList = context.getCondition().getScoreRange();
Double[] scoreRange = CollectionUtils.isEmpty(scoreRangeList) ? new Double[0] : scoreRangeList.toArray(new Double[scoreRangeList.size()]);
if(scoreRange.length == 0) {// Score range is empty - there is no score requirement
return hitRecordsTemp;
}else{
String provinceAddressFromDb;
String cityAddressFromDb;
String districtAddressFromDb;
String detailAddressFromDb;
String detailAddressFromDbFormat;
Double score;
hitRecords = Lists.newArrayListWithExpectedSize(hitRecordsTemp.size());
for (E item : hitRecordsTemp) {
// Similar scores that meet the criteria are added to the result set
provinceAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_PROVINCE_ADDRESS);
cityAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_CITY_ADDRESS);
districtAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_DISTRICT_ADDRESS);
detailAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_DETAIL_ADDRESS);
detailAddressFromDbFormat = provinceAddressFromDb + cityAddressFromDb + districtAddressFromDb
+ ( (HashMap<String,String>)anm.splitAddress(detailAddressFromDb) ).get(SIMILAR_DETAIL_FIELD);
score = NLPUtil.getUtil().similarity(SIMILAR_TYPE, detailAddressFromDbFormat, addressDetailFormat);
log.info("[execute][hitRecords][similarScore],requestId={},addressDetailFormat={},addressDetailFromDb={},score={}",
requestId, addressDetailFormat, detailAddressFromDbFormat, score);
// The length of the score range is 1- the minimum score required
if(scoreRange.length == 1 && score >= scoreRange[0]){
hitRecords.add(item);
}
// Score interval length is 2- score interval requirement
if(scoreRange.length == 2 && score >= scoreRange[0] && score <= scoreRange[1]){ hitRecords.add(item); }}}returnhitRecords; }}Copy the code4. Extension
4.1 Check service request parameters
Example request parameters
{
"requestId":"1"."conditions": [{
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Master lender id number."."searchFieldName": "idNo"."searchFieldValue": "350 * * * * 8114118"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Master lender's phone number."."searchFieldName": "mobile"."searchFieldValue": "186 * * * * 2901"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Selling mobile phone numbers"."searchFieldName": "mobile"."searchFieldValue": "182 * * * * 4023"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Used Car Seller's ID Number"."searchFieldName": "idNo"."searchFieldValue": "3522 * * * * 2138"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Emergency contact number one, mobile number."."searchFieldName": "mobile"."searchFieldValue": "139 * * * * 603"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Emergency contact 2, cell number."."searchFieldName": "mobile"."searchFieldValue": "18 * * * * 637"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Guarantor's ID Number"."searchFieldName": "idNo"."searchFieldValue": "350 * * * * 38"
}, {
"sourceType": "CREDITOR_INFO"."searchFieldDesc": "Guarantor's mobile phone number."."searchFieldName": "mobile"."searchFieldValue": "15 * * * * 020"
}, {
"sourceType": "VEHICLE_INFO"."searchFieldDesc": "Vehicle VIN"."searchFieldName": "vin"."searchFieldValue": "LFV****3721"}}]Copy the codeSample response results
Part of the data desensitization display, such as mobile phone number, bank card number, ID number.
{
"code": 0."data": [{"hitCount": 1."hitRecords": [{"appCode": "F2009111915000180101"."creditCardNo": "-"."dataCode": "P20121700551635"."externalId": "100021309"."id": 243677."idNo": "350122 * 4118"."mobile": "186 * * 01"."name": "Zheng * *"."scene": 1003."sourceType": 1}]."message": "Query successful"."searchFieldDesc": "Master lender id number."."searchFieldName": "idNo"."searchFieldValue": "3501 * 14118"
},
{
"hitCount": 2."hitRecords": [{"appCode": "F2009151915000180101"."creditCardNo": "-"."dataCode": "P20121700538028"."externalId": "100022351"."id": 219723."idNo": "-"."mobile": "186 * 901"."name": "Zheng * *"."scene": 1002."sourceType": 8
},
{
"appCode": "F2009111915000180101"."creditCardNo": "-"."dataCode": "P20121700551635"."externalId": "100021309"."id": 243677."idNo": "3501 * 118"."mobile": "186 * * 01"."name": "Zheng * *"."scene": 1003."sourceType": 1}]."message": "Query successful"."searchFieldDesc": "Master lender's phone number."."searchFieldName": "mobile"."searchFieldValue": "186 * 2901"
},
{
"hitCount": 1."hitRecords": [{"appCode": "F2009151915000180106"."creditCardNo": "-"."dataCode": "P20121700537447"."externalId": "100022605"."id": 218697."idNo": "3505 * 5550"."mobile": "182 * 4023"."name": "O * *"."scene": 1003."sourceType": 1}]."message": "Query successful"."searchFieldDesc": "Selling mobile phone numbers"."searchFieldName": "mobile"."searchFieldValue": "182 * 023"
},
{
"hitCount": 6."hitRecords": [{"appCode": "F2011091915000180104"."creditCardNo": "-"."dataCode": "P20121700417419"."externalId": "100038555"."id": 7575."idNo": "35223 * * 38"."mobile": "-"."name": "Nguyen * *"."scene": 1002."sourceType": 6
},
// omit others]."message": "Query successful"."searchFieldDesc": "Used Car Seller's ID Number"."searchFieldName": "idNo"."searchFieldValue": "3522 * 32138"
},
{
"hitCount": 3."hitRecords": [{"appCode": "F2010191915000180107"."creditCardNo": "-"."dataCode": "P20121700481894"."externalId": "100032121"."id": 121031."idNo": "-"."mobile": "139 * 603"."name": "Chen * *"."scene": 1002."sourceType": 7
},
// omit others]."message": "Query successful"."searchFieldDesc": "Emergency contact number one, mobile number."."searchFieldName": "mobile"."searchFieldValue": "139 * 603"
},
{
"hitCount": 3."hitRecords": [{"appCode": "F2007281915000180103"."creditCardNo": "-"."dataCode": "P20121700447417"."externalId": "100010141"."id": 60345."idNo": "-"."mobile": "1810 * * 37"."name": "Lin * *"."scene": 1002."sourceType": 8
},
// omit others]."message": "Query successful"."searchFieldDesc": "Emergency contact 2, cell number."."searchFieldName": "mobile"."searchFieldValue": "181 * 637"
},
{
"hitCount": 3."hitRecords": [{"appCode": "F2009281915000180104"."creditCardNo": "-"."dataCode": "P20121700442684"."externalId": "100026661"."id": 52033."idNo": "35012 * 938"."mobile": "152 * * 20"."name": "Xu * *"."scene": 1003."sourceType": 1
},
// omit others]."message": "Query successful"."searchFieldDesc": "Guarantor's ID Number"."searchFieldName": "idNo"."searchFieldValue": "35012 * * * * 195938"
},
{
"hitCount": 4."hitRecords": [{"appCode": "F2009281915000180104"."creditCardNo": "-"."dataCode": "P20121700442684"."externalId": "100026661"."id": 52033."idNo": "3501 * * 8"."mobile": "152050 * *"."name": "Xu * *"."scene": 1003."sourceType": 1
},
// omit others]."message": "Query successful"."searchFieldDesc": "Guarantor's mobile phone number."."searchFieldName": "mobile"."searchFieldValue": "152 * *"
},
{
"hitCount": 6."hitRecords": [{"appCode": "F2011091915000180104"."dataCode": "V20121700417457"."evaluateRemark": "Normal. 1. Take photos of spare tire groove again, and ensure that they are complete and clear. \n2. Add left and right rear leaf plate flume photos \ N3. Add left and right front stringer photos \ N4. Add photos of main passenger seat slide track \ N5. Add photos of frame rubbing number (on iron sheet) if there are replacement records."."externalId": "100038555"."id": 1045."mileage": 136029."scene": 1002."sourceType": 12."vin": "LFV4A24F7A30837**"
},
// omit others]."message": "Query successful"."searchFieldDesc": "Vehicle VIN"."searchFieldName": "vin"."searchFieldValue": "LFV4A24F7A3083**"}]."msg": "Operation successful"."success": true
}
Copy the code4.2 Configuration of data query replay field
In order to improve the scalability of data query interface, metadata configuration is based on configuration.
Value Contains the following parameters
desc: Parameter description, no service logic; Use as a field description only.searchMode: Indicates the query mode. Currently, only two query modes are supportedEXACT(Accurate query),SIMILAR(Similarity query).tables: json string array, suitable for this domain query table, currently total three tables (PERSON_INFO,DEALER_INFO,VEHICLE_INFO).mapping: Query field mappings, string data, query form in or as conditions, result sets to merge.exclude: Queries the filter criteria. This parameter is used as the filter criteria in the query result set. The key field is used as the query condition. The value field is used as the query condition.
{" idNo ": {" desc" : "id", "searchMode" : "EXACT" and "mapping" : [], "tables" : [" PERSON_INFO, "" DEALER_INFO"]}, "name" : {" desc ":" name ", "searchMode" : "EXACT" and "mapping" : [], "tables" : [" PERSON_INFO, "" DEALER_INFO"]}, "mobile" : {" desc ": "The mobile phone number (primaryMobile SecondMobile)", "searchMode" : "EXACT", "mapping" : [" primaryMobile ", "SecondMobile"], "tables" : [ "PERSON_INFO", "DEALER_INFO" ], "exclude": { "sourceTypeScopeExclude": [ 2, 3 ] } }, "creditCardNo": { "desc": "Bank card", "searchMode" : "EXACT", "mapping" : [], "tables" : [" PERSON_INFO, "" DEALER_INFO"]}, "companyAddress" : {" desc ": "SearchMode ": "SIMILAR", "mapping": [ "companyAddressProvince", "companyAddressCity", "companyAddressDistrict", "companyAddressDetail" ], "tables": ["PERSON_INFO", "DEALER_INFO"]}, "censusAddress": {"desc": "registered address ", "searchMode": "SIMILAR", "mapping": [ "censusAddressProvince", "censusAddressCity", "censusAddressDistrict", "censusAddressDetail" ], "tables": ["PERSON_INFO"]}, "residenceAddress": {"desc": "residenceAddress", "searchMode": "SIMILAR", "mapping": [ "residenceAddressProvince", "residenceAddressCity", "residenceAddressDistrict", "residenceAddressDetail" ], "tables": "PERSON_INFO"}, "vin" : {" desc ":" vehicle vin ", "searchMode" : "EXACT" and "mapping" : [], "tables" : [" VEHICLE_INFO "]}}Copy the code5, summary
The overall design uses related design patterns and is divided into multiple modules, each responsible for its own business logic responsibilities. In the design of data replay interface, considering the large amount of query data, based on the input of multiple conditions, using parallel processing, and merge the query results of multiple processors, so as to improve the performance of the interface.