Lombok was used a lot in the last article, but do you know how it works? The annotation processor is a tool for handling compile-time annotations. We just generate some code ourselves, but unlike Lombok, Lombok adds classes to existing classes. How does Lombok modify the content of existing classes? Let’s take a closer look at Lombok’s principles.
Javac principle
Now that we are operating on the class at compile time, we need to understand what Javac does to the program in Java. Javac code compilation process is actually written in Java, we can view its source code for its simple analysis, how to download the source code, Debug source code here I will not analyze, recommend a good article to write. Javac source code debugging tutorial.
The compilation process is roughly divided into three phases
- Parse and populate symbol tables
- Annotation processing
- Analysis and bytecode generation
The interaction process of these three phases is shown in the figure below.
Parse and populate symbol tables
This step consists of two steps, including parsing and filling symbols. Parsing is divided into two steps: lexical analysis and syntax analysis.
Lexical analysis and grammatical analysis
Lexical analysis is to convert the character flow of the source code into a collection of tokens in Java. A single character is the smallest element in the process of program writing, while tokens are the smallest element in the process of compilation. Keywords, variable names, literals, and operators can all become tokens. For example, in Java int a = b+2, this code represents six tokens: int, a, =, b, +, and 2. The int keyword is made up of three characters, but it is only a Token and cannot be split.
Syntax analysis is the process of constructing Abstract object tree based on Token sequences. Abstract syntax tree is a tree representation method used to describe the syntax structure of code. Each node of the syntax tree represents a syntax structure in the program code. For example, packages, types, modifiers, operators, interfaces, return values, and even code comments are syntactic constructs.
The parse tree structure is represented by JCTree, and we can take a look at its subclasses.
We can build our own class and see how it looks as a tree structure during compilation.
1public class HelloJvm {
2
3 private String a;
4 private String b;
5
6 public static void main(String[] args) {
7 int c = 1+2;
8 System.out.println(c);
9 print();
10 }
11
12 private static void print(){
13
14 }
15}
Copy the codeAnd notice where I put the red line, you can see that these are all subclasses of JCTree. We can see that compile-time trees have JCCompilationUnit as their root node, and then as the building blocks of classes such as methods, private variables, and class classes, which are all part of the building blocks of the tree.
Fill symbol table
Populating a symbol table has little to do with our Lombok principles, as you can see here.
After completed the syntax analysis and lexical analysis, the next step is the process of filling the symbol table, the symbol table is composed of a group of symbols and symbolic address information form, it can be imagined a hash table in the form of the value of K – V (the symbol table is not necessarily the hash table implementation, can make the orderly symbol table, tree symbol table, stack structure symbol table, etc.). The information registered in the symbol table is used at different stages of compilation. In semantic analysis, the contents of the symbol table are used for semantic checking (such as checking whether the use of a name is consistent with the original description) and for generating intermediate code. In the object code generation stage, symbol table is the basis of address allocation for symbol names.
Annotation processor
After the first step of parsing and populating the symbol table, it’s time to focus on the annotation processor. Because this step is the key to Lombok’s implementation principle.
After JDK1.5, the Java language provides support for annotations, which come into play at run time, just like normal Java code. Jsr-269 was implemented in JDK1.6 and provides a standard API for a set of plug-in annotation handlers to process annotations at compile time. We can think of it as a set of compiler plug-ins that can read, modify, and add arbitrary elements to the abstract syntax tree.
If these plug-ins modify the syntax tree during annotation processing, the compiler will go back to parsing and populating the symbol table and reprocess the tree until all the plug-in annotation handlers are gone. Each loop becomes a Round.
With a standard API for compiler annotation processing, it is possible for our code to interfere with the behavior of the compiler. Since any element of the syntax tree, including code comments, can be accessed in the plug-in, plug-ins implemented through plug-in annotation processors have a lot of scope for functionality. With enough creativity, programmers can use plug-in annotation processors to do many things that would otherwise only be done in code.
Semantic analysis and bytecode generation
After parsing, the compiler gets an abstract syntax tree representation of the program code. The syntax tree can represent an abstraction of a well-structured source program, but there is no guarantee that the source program will be logical. The main task of semantic analysis is to check the context-related properties of structurally correct source programs, such as type checking.
For example, we have the following code
1int a = 1;
2boolean b = false;
3char c = 2;
Copy the codeNow we might have the following operation
1int d = b+c;
Copy the codeThe above code is structurally correct, but the semantics are wrong. So if you run it you’re going to get a compile failure, you’re not going to compile.
Implement a simple Lombok yourself
Now that we’ve seen how javac works, let’s write a simple widget to add code to an existing class. We’ll just generate the set method. Start by writing a custom annotation class.
1@Retention(RetentionPolicy.SOURCE) // Annotations are reserved only in the source code
2@Target(ElementType.TYPE) // Used to modify classes
3public @interface MySetter {
4}
Copy the codeThen write the annotation handler class for this annotation class
1@SupportedSourceVersion(SourceVersion.RELEASE_8)
2@SupportedAnnotationTypes("aboutjava.annotion.MySetter")
3public class MySetterProcessor extends AbstractProcessor {
4
5 private Messager messager;
6 private JavacTrees javacTrees;
7 private TreeMaker treeMaker;
8 private Names names;
9
10 / * *
11 * @Description: 1. Message is mainly used for logging at compile time
12* 2. JavacTrees provide abstract syntax trees to be processed
13* 3. TreeMaker encapsulates some methods for creating AST nodes
14* 4. Names provides methods for creating identifiers
15* /
16 @Override
17 public synchronized void init(ProcessingEnvironment processingEnv) {
18 super.init(processingEnv);
19 this.messager = processingEnv.getMessager();
20 this.javacTrees = JavacTrees.instance(processingEnv);
21 Context context = ((JavacProcessingEnvironment)processingEnv).getContext();
22 this.treeMaker = TreeMaker.instance(context);
23 this.names = Names.instance(context);
24 }
25
26 @Override
27 public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
28 return false;
29 }
30}
Copy the codeNotice here that we get some environment information at compile time in the init method. We extracted some key classes from the environment, described below.
JavacTrees: provides a tree of abstract syntax to be processedTreeMaker: encapsulates methods for manipulating the AST abstract syntax treeNames: provides methods to create identifiersMessager: mainly used for logging in the compiler
We then use the provided utility classes to modify the existing AST abstract syntax tree. The main change logic is in the process method, and if true is returned, the JavAC process starts again from parsing and populating the symbol table. The logic of the process method is mainly as follows
1@Override
2 public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
3 Set<? extends Element> elementsAnnotatedWith = roundEnv.getElementsAnnotatedWith(MySetter.class);
4 elementsAnnotatedWith.forEach(e->{
5 JCTree tree = javacTrees.getTree(e);
6 tree.accept(new TreeTranslator(){
7 @Override
8 public void visitClassDef(JCTree.JCClassDecl jcClassDecl) {
9 List<JCTree.JCVariableDecl> jcVariableDeclList = List.nil();
10 // Find all the variables in the abstract tree
11 for (JCTree jcTree : jcClassDecl.defs){
12 if (jcTree.getKind().equals(Tree.Kind.VARIABLE)){
13 JCTree.JCVariableDecl jcVariableDecl = (JCTree.JCVariableDecl) jcTree;
14 jcVariableDeclList = jcVariableDeclList.append(jcVariableDecl);
15 }
16 }
17 // Generate a method for a variable
18 jcVariableDeclList.forEach(jcVariableDecl -> {
19 messager.printMessage(Diagnostic.Kind.NOTE,jcVariableDecl.getName()+"has been processed");
20 jcClassDecl.defs = jcClassDecl.defs.prepend(makeSetterMethodDecl(jcVariableDecl));
21 });
22 super.visitClassDef(jcClassDecl);
23 }
24 });
25 });
26 return true;
27 }
Copy the codeIn fact, it looks difficult, the principle is relatively simple, mainly because we are not familiar with the API, so it looks difficult to understand, but the main idea is as follows
- find
@MySetterAnnotate the annotated class to get its syntax tree - Traverse its syntax tree to find its parameter node
- Create your own method node and add it to the syntax tree
Graphically, we have created a test class, TestMySetter, whose syntax tree we know roughly looks like the following figure.
So our goal is to make the syntax tree look like the one shown below. Since the final generation of bytecode is based on the syntax tree, we add nodes of methods to the syntax tree, so the bytecode of the corresponding method will be generated when the bytecode is generated.
The code for generating the method node is as follows
1private JCTree.JCMethodDecl makeSetterMethodDecl(JCTree.JCVariableDecl jcVariableDecl){
2
3 ListBuffer<JCTree.JCStatement> statements = new ListBuffer<>();
4 // Generate expressions such as this.a = a;
5 JCTree.JCExpressionStatement aThis = makeAssignment(treeMaker.Select(treeMaker.Ident(names.fromString("this")), jcVariableDecl.getName()), treeMaker.Ident(jcVariableDecl.getName()));
6 statements.append(aThis);
7 JCTree.JCBlock block = treeMaker.Block(0, statements.toList());
8
9 // Generate input parameters
10 JCTree.JCVariableDecl param = treeMaker.VarDef(treeMaker.Modifiers(Flags.PARAMETER), jcVariableDecl.getName(), jcVariableDecl.vartype, null);
11 List<JCTree.JCVariableDecl> parameters = List.of(param);
12
13 // Generate return objects
14 JCTree.JCExpression methodType = treeMaker.Type(new Type.JCVoidType());
15 return treeMaker.MethodDef(treeMaker.Modifiers(Flags.PUBLIC),getNewMethodName(jcVariableDecl.getName()),methodType,List.nil(),parameters,List.nil(),block,null);
16
17}
18
19private Name getNewMethodName(Name name){
20 String s = name.toString();
21 return names.fromString("set"+s.substring(0.1).toUpperCase()+s.substring(1,name.length()));
22}
23
24private JCTree.JCExpressionStatement makeAssignment(JCTree.JCExpression lhs, JCTree.JCExpression rhs) {
25 return treeMaker.Exec(
26 treeMaker.Assign(
27 lhs,
28 rhs
29 )
30 );
31}
Copy the codeFinally, we execute the following three commands
1javac -cp $JAVA_HOME/lib/tools.jar aboutjava/annotion/MySetter* -d
2javac -processor aboutjava.annotion.MySetterProcessor aboutjava/annotion//TestMySetter.java
3javap -p aboutjava/annotion/TestMySetter.class
Copy the codeYou can see the output below
1Compiled from "TestMySetter.java"
2public class aboutjava.annotion.TestMySetter {
3 private java.lang.String name;
4 public void setName(java.lang.String);
5 public aboutjava.annotion.TestMySetter();
6}
Copy the codeYou can see that the setName method we need is already generated in the bytecode.
The code address
conclusion
So far, you’ve covered Lombok’s basics, which are operations on abstract syntax trees. There are many other things you can do with compile time, such as checking code specifications. I’ve only written about the creation of a set method here, but if you’re interested you can write your own code to try Lombok’s get method creation.
Interested can pay attention to my new public number, search [program ape 100 treasure bag]. Or just scan the code below.
reference
- [Understanding the Java Virtual Machine]()
- Java annotation processor – Modifies the syntax tree at compile time
- Java-JSR-269- Plug-in annotation processor