I recently encountered a business requirement to count the capabilities provided by the business side. These capabilities are described in a total JSON configuration file that can be parsed locally and on the platform, such as:

{
  "components": [{"dependency": "Com. Codelang. Module: check: 1.0.0"."name": "zhangsan"."verifiedContainer": [
        "list"."home"]."verifiedProtocol": [
        "public"]."version": "1.0.0"}}]Copy the code

The easiest way to do this is to write a JSON file and have each line of business modify the JSON file. This is a lazy solution, but it has several disadvantages:

  • Json plain text files may lead to non-standard input by the business side. For example, the size of the JSON key is incorrectly written or the spelling of the word is incorrect. As a result, the platform and local authorities cannot parse the field
  • The business side does not know which keys are required, causing it to look at the document each time and which keys need to be entered
  • There is no way to know what function so many keys correspond to, and there is no way to write comments in JSON, so you have to check the document every time to see what the key expresses

What is the solution to these problems? For the business side, they just need to input the value required by the annotation, and the optional parameter is replaced by the default value. In addition, they can also annotate the prompt. Let’s look at the definition of annotation:

annotation class Component(
     // Mandatory: Module name
    val name: String,
     // Mandatory: Module version
    val version: String,
     // Mandatory: module dependency
    val dependency: String,
    // Optional: Verify the container
    val verifiedContainer: Array<String> = arrayOf(),
    // Optional: Verify the protocol
    val verifiedProtocol: Array<String> = arrayOf()
)
Copy the code

Then, the business side just needs to write a class that describes it with this annotation, for example:

@Component(
    name = "zhangsan",
    version = "1.0.0",
    dependency = "com.aa.bb",
    verifiedContainer = ["list"."homeContainer"],
    verifiedProtocol = ["public"])
class AComponent
Copy the code

Well, the specification business side is done with the input, so how do you translate this annotation into a JSON file? APT? This is too heavy, if the module has new functionality to change the annotation processor module, we just write a script.

We looked at the base department’s collection of privacy apis, using Javaparse to statically resolve sourceCode in the SDK, if methods were annotated by RequiresPermission.

Static parsing is a good idea, but for now Java is all you have. What if the business side is written in Kotlin? If there is Java file parsing, there must be Kotlin file parsing. After searching, we found three libraries:

  • Kotlin-parser: The research found it a little difficult to iterate over annotation parameters based on annotation method callbacks
  • Kastree: Traversal is simple. You can get the Node to traverse down
  • Kotlinx. ast: a large and comprehensive AST parsing library with many rules, but a bit heavy to use

For the brief introduction and demo testing, we decided to use kastree, a lightweight library, to implement this. In the description of the README, we can write a simple pseudo-code:

// Read the kt file contents
val code = File("xx/test.kt").readText()
// Generate a parser
val file = Parser.parseFile(code)
// Start parsing the syntax
Visitor.visit(file) { v, _ ->
    // v is a Node Node
    Log.i("node",v)
}
Copy the code

We can try to parse our annotation class. However, first we need to understand how to traverse a Node. We can print out what the structure of a Node looks like. You can check the test. TXT file of demo, the following code is slightly organized structure:

Structured(
  // The class name of the annotation
  name=App2Component, 
  mods=[
    AnnotationSet(
       target=null, 
       anns=[
           Annotation(
               // Annotation class Component
               names=[Component], 
               typeArgs=[], 
               args=[
                   // Annotate the parameter name
                   ValueArg(name=name, 
                            asterisk=false, 
                            expr=StringTmpl(
                                // Annotation parameter name corresponding value zhangsan
                                elems=[Regular(str=zhangsan)], 
                                raw=false)), 
                   // Annotate the version parameter
                   ValueArg(name=version, 
                            asterisk=false, 
                            expr=StringTmpl(
                                 // annotate the value 1.0.0 for version
                                elems=[Regular(str=1.0. 0)], 
                                raw=false)), 
                   // Annotation parameter dependency
                   ValueArg(name=dependency, 
                            asterisk=false, 
                            expr=StringTmpl(
                                 // Annotation parameter dependency corresponds to the value com.aa.bb
                                elems=[Regular(str=com.aa.bb)], 
                                raw=false))]])],...).Copy the code

The overall Node is similar to the JSON file format, each Node is a type, we only need to parse out the data we need step by step according to the Node type, for example:

// Check whether the node is Structured
if (v is Node.Decl.Structured) {
   // Fetch the annotated class name App2Component
   val className = v.name
    
   // The first element of the MOds array is strongly AnnotationSet
   val annotationSet = (v.mods[0] as Node.Modifier.AnnotationSet)
   // Get the Annotation node
   val anno = annotationSet.anns[0]
   // Fetch the annotation class name Component
   val annoName = anns.names[0]
   
   // Iterate over the parameter values of the annotations
   anno.args.forEach { node ->
     val expr = node.expr
     if (expr is Node.Expr.StringTmpl) {
        val elems = expr.elems[0]
        if (elems is Node.Expr.StringTmpl.Elem.Regular) {
            // Output the annotation parameter name and value
            println("key=" + node.name + " value=" + elems.str)
        } 
     }
   }
   ...
}
Copy the code

The overall parsing is very simple, the parameter names and values can be retrieved by traversing, which means that even if we add functionality points to the module later, we just need to touch our annotation class, and the script doesn’t need to be modified at all.

After we have parsed the contents, it will be easier to generate json files. We just need to create a JSONObject node for each KT file to be parsed and put the parsed information into it. If there are multiple files, create a JSONArray. Then add the JSONObject, create a File, convert the JSONArray to a string, and write.

Of course, there have been some problems, such as the initial integration of Kastree, following the README to write an example, run directly error, a bit of a retreat feeling:

Exception in thread "main" java.lang.IllegalStateException: LOGGING: Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.naming.ldap, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt] (no MessageCollector configured)
	at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.report(ClasspathRootsResolver.kt:312)
	at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.report$default(ClasspathRootsResolver.kt:310)
	at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.addModularRoots(ClasspathRootsResolver.kt:253)
	at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.computeRoots(ClasspathRootsResolver.kt:123)
	at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.convertClasspathRoots(ClasspathRootsResolver.kt:79)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment.<init>(KotlinCoreEnvironment.kt:279)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment.<init>(KotlinCoreEnvironment.kt:127)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.createForProduction(KotlinCoreEnvironment.kt:463)
	at kastree.ast.psi.Parser$proj$2.invoke(Parser.kt:16)
	at kastree.ast.psi.Parser$proj$2.invoke(Parser.kt:14)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at kastree.ast.psi.Parser.getProj(Parser.kt)
	at kastree.ast.psi.Parser.parsePsiFile(Parser.kt:30)
	at kastree.ast.psi.Parser.parseFile(Parser.kt:23)
	at KtParseKt.parseKotlinFile(KtParse.kt:44)
	at KtParseKt.main(KtParse.kt:27)
Copy the code

However, after looking at the log, I think it may be related to the JDK version, so I try to change jdK11 to JDK8 and run it perfectly

conclusion

Finally, we normalized the coding of the business side through annotations + scripts. There are a lot of tricks you can play with kt and Java file parsing, such as findBugs, Lint, etc.

Finally, post a link to the source code: github.com/MRwangqi/ko…