preface
In the process of iteration of product version, the test team mostly designed use cases according to product design documents or requirements documents, and mined product defects by means of automated testing or manual testing. The following problems exist:
- It is not easy to know if the use case covers all of the changes in the code;
- Although there is the coverage of jacoco’s ready-made full code, it is not intuitive for this code change, so we cannot confirm how much use cases are covered by this change.
The advent of incremental code coverage addresses this pain point, enabling teams to test accurately.
The so-called incremental code coverage refers to comparing the diFF of the code of the two branches (generally the test branch and the master branch) to reduce the scope of the full code coverage to the granularity of all function methods involved in the diFF, and the function methods not designed by the code diff are excluded from the statistics.
The project design
Due to the code modification of the Jacoco Core module, Jacoco-CLI.jar needs to be repackaged
- Based on git diff function, the difference code lines of two different branches are calculated.
- Diff line number is converted to diFF function method level by JDT.
- In jacoco core module, the function method range of statistics is filtered according to diff function method information.
Differential code details
1. Get the difference code details
This part is mainly to compare the differences between the two branches and format them into the format we want through gitLab’s own Git diff interface. Git Diff parses the difference line information of files and also needs to parse the code text information into abstract syntax tree through Eclipse JDT AST. To put it simply, the difference line is converted to the difference method, and the row information is corresponding to the function method.
public Map<String, List<MethodInfo>> run_diff(){
// Use the gitlab interface to extract the difference row data of the two branches
JSONArray diff_src = git.get_commit_diff(from_commit,to_commit);
// Serialize to Java objects
List<DiffInfo> diff = DiffParse.parse(diff_src);
// No file changes in the past
filter(diff);
// Convert diFF row information to DIff difference information
parse_diff_method(diff);
return diff_class_exist_method;
}
/* * Get diff details for both branches from gitlab API */
public JSONArray get_commit_diff(String from, String to){
String uri = "/api/v4/projects/" + this.project_id + "/repository/compare";
Map<String, String> params = new HashMap<>();
params.put("from",from);
params.put("to",to);
JSONObject res = get(uri,params,this.headers);
return (JSONArray) res.get("diffs");
}
/* * serialize data */
public static List<DiffInfo> parse(JSONArray data_array){
List<DiffInfo> diffInfos = new ArrayList<>();
for (Object data:data_array){
JSONObject data1 = (JSONObject) data;
DiffInfo diffInfo = new DiffInfo();
// Set the diff file address
diffInfo.setFile_path(data1.get("new_path").toString());
// Set the diff type
diffInfo.setType(parseDiffType(data1));
// Parse the number of diff lines
diffInfo.setLines(parseDiffContent(data1.get("diff").toString()));
diffInfos.add(diffInfo);
}
return diffInfos;
}
/** * Filter type is blank, delete diff * Filter is not Java end diff *@param data
*/
private void filter(List<DiffInfo> data){ data.removeIf(file -> file.getType() == DiffType.DELETED || file.getType() == DiffType.EMPTY); data.removeIf(file->! file.getFile_path().endsWith(".java"));
}
/** * Convert the number of lines out of diff to method *@param data
*/
private void parse_diff_method(List<DiffInfo> data){
data.forEach(item ->{
String file_path = item.getFile_path();
String java_text = git.get_file_content(file_path, to_commit);
//ast parses Java source code
ASTParse ast_parse = new ASTParse(file_path,java_text);
List<Integer> diff_lines = item.getLines();
// Parse the class corresponding to the trip
List<ClassInfo> diff_tree = ast_parse.parse_code_class();
lines_to_method(diff_tree,diff_lines);
});
}
Copy the code2. Jacoco changes the source code details
This section is mainly for adding incremental code processing logic to jacoco’s source code.
- Determine whether incremental code coverage is enabled;
- Calculate variance data;
- Narrow the statistical range to within the range of differential data;
// org/jacoco/cli/internal/commands/Report.java
/** * check for incremental code coverage *@return* /
private boolean isDiff(PrintWriter out){
List<String> stringList = new ArrayList<>(Arrays.asList(gitlabHost,gitlabToken,gitlabProjectId,fromCommit,toCommit));
return stringList.stream().noneMatch(StringUtils::isEmptyOrNull);
}
private IBundleCoverage analyze(final ExecutionDataStore data,
final PrintWriter out) throws IOException {
final CoverageBuilder builder;
// Determine whether incremental code coverage is enabled
if (isDiff(out)){
builder = new CoverageBuilder(gitlabHost,gitlabProjectId,gitlabToken,fromCommit,toCommit);
out.println("[!!!INFO] === start deal with Incremental code coverage ===");
}else{
builder = new CoverageBuilder();
}
final Analyzer analyzer = new Analyzer(data, builder);
for (final File f : classfiles) {
analyzer.analyzeAll(f);
}
printNoMatchWarning(builder.getNoMatchClasses(), out);
return builder.getBundle(name);
}
// org/jacoco/core/analysis/CoverageBuilder.java
/** * Incremental code new Builder * receives the relevant input parameter data of the incoming difference information ** */
public CoverageBuilder(String host, String project_id, String token, String from_commit, String to_commit) {
this.classes = new HashMap<String, IClassCoverage>();
this.sourcefiles = new HashMap<String, ISourceFileCoverage>();
if (classInfos == null || classInfos.isEmpty()){
DiffMain diffMain = newDiffMain(host, project_id, token, from_commit, to_commit); classInfos = diffMain.run_diff(); }}// org/jacoco/core/internal/flow/ClassProbesAdapter.java
public final MethodVisitor visitMethod(final int access, final String name,
final String desc, final String signature,
final String[] exceptions) {
final MethodProbesVisitor methodProbes;
final MethodProbesVisitor mv = cv.visitMethod(access, name, desc,
signature, exceptions);
// Incremental calculation coverage
if(mv ! =null && DiffMain.is_contain_method(this.name,name,desc,CoverageBuilder.classInfos) ) {
methodProbes = mv;
} else {
// We need to visit the method in any case, otherwise probe ids
// are not reproducible
methodProbes = EMPTY_METHOD_PROBES_VISITOR;
}
return new MethodSanitizer(null, access, name, desc, signature,
exceptions) {
@Override
public void visitEnd(a) {
super.visitEnd();
LabelFlowAnalyzer.markLabels(this);
final MethodProbesAdapter probesAdapter = new MethodProbesAdapter(
methodProbes, ClassProbesAdapter.this);
if (trackFrames) {
final AnalyzerAdapter analyzer = new AnalyzerAdapter(
ClassProbesAdapter.this.name, access, name, desc,
probesAdapter);
probesAdapter.setAnalyzer(analyzer);
methodProbes.accept(this, analyzer);
} else {
methodProbes.accept(this, probesAdapter); }}}; }/ / org/jacoco/core/internal/diff2 DiffMain. The new custom Java methods
public static Boolean is_contain_method(String location, String current_method,String current_method_args,Map<String, List<MethodInfo>> diffs){
if (diffs == null) {// If DIFFs is null, full coverage is carried out
return true;
}
if (diffs.containsKey(location)){
List<MethodInfo> methods = diffs.get(location);
for (MethodInfo method:methods){
// Determine whether the method selects a method in the diff class
if (current_method.equals(method.getMethodName())){
returncheckArgs(current_method_args,method.getArgs()); }}}return false;
}
/** * Check whether the parameters are the same by parameter type and number * The return type is not verified yet, and can be optimized later *@param current_method_args_src
* @param reference_args
* @return* /
private static Boolean checkArgs(String current_method_args_src,List<ArgInfo> reference_args){
Type[] current_method_args = Type.getArgumentTypes(current_method_args_src);
// Check whether the number of parameters is empty
if (current_method_args.length ==0 && reference_args.size() ==0) {return true;
}
if (current_method_args.length == reference_args.size()){
// Check whether the parameter types are the same
List<Boolean> is_same_list = new ArrayList<>();
for (int i=0; i<current_method_args.length; i++){ Type current_method_arg = current_method_args[i]; String current_method_arg1 = current_method_arg.toString(); String current_method_arg_final; String reference_arg_type = reference_args.get(i).getType(); String reference_arg_type_final = reference_arg_type;// Ljava/lang/String; / Split the last name of the class
// Replace the JVM type with normal
if (type_map().containsKey(current_method_arg1)){
// If the parameter type is JVM short identifier
current_method_arg_final = type_map().get(current_method_arg1);
}else {
// marks whether the argument is an array
boolean is_array = current_method_arg1.contains("[");
String[] current_method_arg2 = current_method_arg1.split("/");
String current_method_arg3 = current_method_arg2[current_method_arg2.length - 1];
Pattern pattern = Pattern.compile(. "< + > |;"); // Remove Spaces to conform to newlines
Matcher matcher = pattern.matcher(current_method_arg3);
String current_method_arg4 = matcher.replaceAll("");
reference_arg_type_final = pattern.matcher(reference_arg_type).replaceAll("");
// Ignore the two-dimensional array for now
if (is_array) {
current_method_arg_final = current_method_arg4 + "[]";
}else {
current_method_arg_final = current_method_arg4;
}
}
is_same_list.add(current_method_arg_final.equals(reference_arg_type_final));
}
return is_same_list.stream().allMatch(f-> f);
}
return false;
}
Copy the codeThe finished product sample
At the method level, only incremental code coverage is reserved for display, and non-incremental background space is not included in the statistics.
insufficient
Based on git diff information obtained by Gitlab, if the diff content is too large and exceeds the Gitlab diff limit, the diff content will be empty. Plan to use the JGit local operation git repository to get diff information.
conclusion
- Incremental code coverage can be used as an admission criteria for development self-test to confirm that the proposed test criteria are met;
- Testers can improve their test cases based on incremental code coverage to further improve quality control;
- Use full code coverage as a reference for regression testing to improve regression use case coverage of the main flow logic.
Nanjing 300 Cloud Information Technology Co., LTD. (CHE 300) was founded on March 27, 2014. It is a mobile Internet enterprise rooted in Nanjing and currently located in Nanjing and Beijing. After 7 years of accumulation, the cumulative number of valuation has reached 5.2 billion times, and won the favor of many high-quality investment institutions at home and abroad, such as Sequoia Capital, SAIC Industry Fund. 300 Cloud is an outstanding domestic auto transaction and financial SaaS service provider with independent third party relying on artificial intelligence and standardization of auto transaction pricing and auto financial risk control as its core products.
Welcome to join 300 cloud, witness the booming development of the automobile industry together, look forward to walking with you hand in hand! Java development, Java internship, PHP internship, testing, testing, product manager, big data, algorithm internship, hot recruitment… Official website: www.sanbaiyun.com/, www.che300.com/ Resume: [email protected], please note from Nuggets 😁