Deeper understanding of the JVM (vii) -Execution engine (interpreter and JIT compiler)

Execution Engine Overview

• The execution engine is one of the core components of the Java Virtual Machine.

• Virtual machine is a relative to the concept of “physical machine”, the two machines have code execution ability, the difference is the physical machine execution engine is directly based on CPU, cache, instruction set, and the operating system level, and virtual machine execution engine is implemented by the software itself, so you can not is conditioned by the physical conditions custom instruction set and execution engine structure system, The ability to execute instruction set formats that are not directly supported by hardware.

• The MAIN task of the JVM is to load bytecode into the JVM, but bytecode cannot run directly on the operating system because bytecode instructions are not equivalent to native machine instructions, and it contains only bytecode instructions, symbol tables, and other auxiliary information that can be recognized by the JVM.

• If you want a Java program to run, the Execution Engine’s job is to interpret/compile bytecode instructions to local machine instructions on the corresponding platform. Simply put, the execution engine in the JVM acts as a translator for translating high-level languages into machine languages.

The JVM structure

Execute the engine process

1. Exactly what bytecode instructions the execution engine needs to execute during execution depends entirely on the PC register.
2. Whenever an instruction operation is completed, the PC register updates the address of the next instruction to be executed.
3. Of course, during the execution of the method, the execution engine may accurately locate the object instance information stored in the Java heap area through the object reference stored in the local variable table, and locate the type information of the target object through the metadata pointer in the object header.

In appearance, the execution engine inputs and outputs of all Java virtual machines are identical: input is a binary stream of bytecode, processing is the equivalent process of bytecode parsing execution, and output is the result of execution.

Java code compilation and execution process

• Front-end compilation: The yellow part is using the front-end compiler. Java uses Javac to compile the source code into.class bytecode files.

• The back-end compiler
  • Green is the interpreter
  • The blue is the compiler

Machine code, instruction, instruction set, assembly language, high level language.

High level languages go through assembly to machine code, and because they all go through assembly, the machine code described in this article defaults to assembly to machine code

Machine code

Instructions expressed in binary codes consisting of zeros and ones are called machine codes. From the beginning, people used it to write programs. This is machine language.

Although machine language can be understood and accepted by computers, it is too different from people’s language to be understood and remembered by people, and it is easy to make mistakes when programming.

Programs written in it are read and run directly by the CPU as soon as they are entered into the computer, so they run faster than programs written in other languages. Machine instructions are closely related to the CPU, so different types of CPU correspond to different machine instructions.

instruction

Since machine code was a binary sequence of zeros and ones, it was so unreadable that instructions were invented. Instruction is the machine code in a particular sequence of 0 s and 1 s, reduced to a corresponding instruction (generally for English abbreviations, such as mov, inc, etc.), readable slightly better due to the different hardware platforms, perform the same operation, the corresponding machine code may be different, so different hardware platform of the same kind of instruction (mov), the corresponding machine code may be different, too.

Instruction set

Different hardware platforms support different instructions. The instructions supported by each platform are therefore called the platform-specific instruction set. Such as common

• The x86 instruction set corresponds to the x86 architecture platform
• The ARM instruction set corresponds to the ARM architecture platform

Assembly language

Assembly language was invented because the instructions were still too hard to read.

In assembly language, the opcodes of machine instructions are replaced by Mnemonics, and the addresses of instructions or operands are replaced by address symbols or labels.

In different hardware platforms, assembly language corresponds to different sets of machine language instructions, which are converted into machine instructions by assembly process.

Since the computer only recognizes the instruction code, a program written in assembly language must also be translated into machine instruction code before the computer can recognize and execute it.

A high-level language

In order to make it easier for computer users to program, various high-level computer languages have come into being. High – level language is closer to human language than machine language and assembly language

When a computer executes a program written in a high-level language, the program still needs to be interpreted and compiled into machine code. A program that does this is called an interpreter or compiler.

Compiler classification

Front-end compiler

• Front end compiler: The process of turning.java files into.class files

• Sun’s Javac, the Incremental compiler (ECJ) in Eclipse JDT

The JIT compiler

• The virtual machine’s back-end run-time Compiler (JIT Compiler, Just In Time): the process of converting bytecode into assembly language

• C1, C2 compilers for HotSpot VM

AOT compiler:

• Static Ahead Of Time Compiler (AOT Compiler): The process Of compiling a. Java file directly into local machine code.

• GNU Compiler for the Java (GCJ), Excelsior JET

The bytecode

• Bytecode is mainly used to realize specific software running and software environment, which is independent of hardware environment and cross-platform.

• Bytecode is a binary code (file) in an intermediate state (intermediate code) that is more abstract than machine code and requires a literal interpreter to translate it into machine code

• Bytecode is implemented through compilers and virtual machines. A compiler compiles the source code into bytecode, and virtual machines on a particular platform translate the bytecode into instructions that can be executed directly.

A semi-compiled, semi-interpreted language

In the JDK1.0 era, it is more accurate to define the Java language as “interpret and execute.” Later, Java also developed compilers that could directly generate native code (assembly language). The JVM now typically executes Java code in a combination of interpreted execution and compiled execution (back-end compilation, not front-end compilation when class files are generated).

The Interpreter (Interpreter)

When the Java virtual machine starts, the bytecode is interpreted line by line according to predefined specifications, and the contents of each bytecode file are “translated” into the assembly language of the corresponding platform, which is executed by the local machine instructions generated by the assembly.

Interpreter classification

In the history of Java, there have been two sets of interpretation effectors: the old bytecode interpreter and the now-ubiquitous template interpreter.

• Bytecode interpreter: It is very inefficient to simulate bytecode execution through pure software code at execution time.

• Template interpreters: The performance of the interpreter is greatly improved by associating each bytecode with a template function that directly produces the local machine code when the bytecode executes.

• In HotSpot VM, the Interpreter mainly consists of the Interpreter module and Code module.

  • Interpreter module: Implements the core functions of the Interpreter
  • Code module: manages the local machine instructions generated by HotSpotVM at runtime

JIT Compiler (Just In Time Compiler)

This is where the virtual machine compiles the source code directly into the assembly language associated with the native machine platform, generating machine code through assembly.

Modern virtual machines use just-in-time compilation techniques to compile methods into local machine code before execution

The Hotspot JIT compiler generates assembly code, which is stored in the JIT cache of the method area

Native machine code does not equal machine code; different platform virtual machines translate locally identifiable instruction sets, or assembly languages. The instruction set consists of different architectures, such as the x86 instruction set

Compiler classification

There are two JIT compilers built into HotSpot VM, the Client Compiler and the Server Compiler, but for the most part we simply refer to them as C1 and C2. Developers can explicitly specify which just-in-time compiler the Java Virtual Machine will use at run time with the following command:

• -client: Specifies that the Java VIRTUAL machine runs in client mode and uses the C1 compiler
  • The C1 compiler makes bytecode optimizations simple and reliable, and takes less time. To achieve faster compilation speed.
• -server: Specifies that the Java virtual machine runs in server mode and uses the C2 compiler
  • C2 performs long optimizations, as well as radical optimizations. But optimized code execution is more efficient.
  • 64-bit JDKS can only be server compilers

Client compiler optimization strategy

C1: A simple and fast compiler that focuses on local optimizations and abandons many of the more time-consuming global optimizations.

• Method inlining: the referenced function code is compiled to the reference point, which reduces stack frame generation, parameter passing, and jumping
• De-virtualizing: Inlining the unique implementation class
• Redundancy elimination: Collapsing code that will not execute at runtime

The Server compiler

C2: a compiler specifically tailored for the server side and specially tuned for server side performance configuration. It is a global level, fully optimized advanced compiler based on escape analysis:

• Scalar replacement: Replace attribute values of aggregate objects with scalar values
• Stack allocation: For unescaped objects, the object is allocated on the stack rather than on the heap
• Synchronized: To clear a synchronization operation, usually referred to as synchronized

The status quo

Because interpreters are so simple to design and implement, there are many high-level languages besides The Java language that also execute based on them, such as Python, Perl, Ruby, and so on. Today, however, interpreter-based execution has become synonymous with inefficiency and is often derided by C/C++ programmers.

To solve this problem, the JVM platform supports a technique called just-in-time compilation. The purpose of just-in-time compilation is to prevent the function from being interpreted and executed. Instead, the entire body of the function is compiled into local machine code. Each time the function is executed, only the compiled local machine code is executed.

However, the implementation mode based on the interpreter still makes an indelible contribution to the development of intermediate languages.

HotSpot VM is one of the most popular high-performance virtual machines on the market today. It uses an interpreter and a just-in-time compiler. When the Java Virtual Machine is running, the interpreter and the just-in-time compiler can work together, learning from each other’s strengths and trying to choose the best way to balance the time of compiling native code against the time of directly interpreting the executed code.

Today, The performance of Java programs has changed to the point where they can compete with C/C++ programs.

Why do you need an interpreter

Some developers may wonder why it is necessary to use an interpreter to “slow” performance when the JIT compiler is already built into HotSpot VM. The JRockit VM, for example, does not contain an interpreter internally, and the bytecode is all compiled and executed by the just-in-time compiler.

First of all, when the program starts, the interpreter can be used immediately, saving compilation time and executing immediately. For the compiler to be effective, it takes some execution time to compile code into local code. However, when compiled into native code, execution is efficient.

So: Although the execution performance of the programs in JRockitVM can be very efficient, the programs will necessarily take longer to compile at startup. For server-side applications, startup time is not a priority, but for those that do, an interpreter and just-in-time compiler may be needed to strike a balance. In this mode, when the Java Virtual Machine starts, the interpreter can come into play first, rather than waiting for the just-in-time compiler to complete its full compilation, which saves a lot of unnecessary compilation time. Over time, compilers come into play, compiling more and more code into local code, achieving higher execution efficiency. At the same time, interpretative execution serves as the compiler’s “escape door” when radical compiler optimization is not feasible.

When the virtual machine starts, the interpreter can be used first, rather than waiting for the just-in-time compiler to complete, which saves a lot of unnecessary compilation time. And with the passage of the program running time, the just-in-time compiler gradually plays a role, according to the hot spot detection function, the valuable bytecode is compiled into assembly language, in exchange for higher program execution efficiency.

Start up and run first, and work immediately. Do not let the program wait (cold snacks are served first). After preheating time, the real-time compiler function can be played (big meal is served later).

JIT importance Example

How do I view JIT compile times

Jconsole viewVisualVM view

Hotspot code and detection mode

Of course, whether you need to start the JIT compiler to compile the bytecode directly to the native machine instructions for the platform depends on how often the code is called. The bytecode that needs to be compiled to native code, also known as “hot code,” is deeply optimized by the JIT compiler at run time to compile the frequently called “hot code” directly into the native machine instructions of the corresponding platform to improve the performance of Java programs.

A method that is called more than once, or a loop body that loops through its body more than once, is called “hot code” and can therefore be compiled into local machine instructions by the JIT compiler. Because this compilation occurs during method execution, it is also known as On StackReplacement, or OSR (On StackReplacement) compilation. (Not stack allocation)

How many times does a method have to be called, or how many times does the body of a loop have to execute before it reaches this level? There must be a clear threshold for the JIT compiler to compile this “hot code” into local machine instructions for execution. Here, the focus is on hotspot detection. HotSpot detection currently used by HotSpot VM is counter based HotSpot detection.

1. Sampling based hot spot detection

A virtual machine that uses this approach periodically checks the top of the stack for each thread, and if it finds a method that frequently appears on the top of the stack, the method is called a “hotspot method.” The advantage of this detection method is that it is simple and efficient to implement, and it can easily get the method call relationship (by unrolling the call stack). The disadvantage is that it is difficult to accurately confirm the popularity of a method, and it is easy to disrupt the hot spot detection due to thread blocking or other external factors.

2. Counter based hot spot detection

A virtual machine that takes this approach sets up a counter for each method (or even block of code) to count the number of times the method has been executed, and if the number of executions exceeds a certain threshold, it is considered a “hot method.” The implementation of this statistical method is more complex, requiring the establishment and maintenance of counters for each method, and can not directly obtain the method calling relationship, but its statistical results are more precise and rigorous.

• With Counter based HotSpot detection, HotSpot VM will establish 2 different types of Invocation Counter and BackEdge Counter for each method.
  • A method call counter measures the number of times a method has been called
  • The backside counter is used to count the number of loops executed by the body of the loop

Method call calculator

• The number of times a method is called. The default threshold is 1500 times in Client mode and 10000 times in Server mode. When this threshold is exceeded, JIT compilation is triggered.

• -xx :CompileThreshold: sets the threshold

• When a method is called, it is first checked for a JIT-compiled version of the method, and if so, the compiled native code is preferred for execution. If no compiled version exists, increments the call counter value of this method by 1, and then determines whether the sum of the method call counter and the backside counter values exceeds the threshold of the method call counter. If the threshold is exceeded, a code compilation request for the method is submitted to the just-in-time compiler.

Heat attenuation

If nothing is set, the method call counter counts not the absolute number of times a method is called, but a relative execution frequency, that is, the number of times a method is called over a period of time.

After a certain time limit, if the method has not been called enough times to be submitted to the just-in-time compiler, the method’s call Counter is reduced by half in a process called Counter Decay. This is called the Counter Half Life Time for this method.

Heat decay is done in passing while the virtual machine is garbage collected. Use the virtual machine parameter -xx: -usecounterdecay to turn off heat decay and let the method counter count the absolute number of method calls so that most methods are compiled as long as the system is running long enough.

Run the -xx :CounterHalfLifeTime parameter to set the half-aging period in seconds.

Back edge counter

Its function is to count the number of times the loop body code in a method is executed. The instruction that jumps after the control flow is encountered in the bytecode is called a “Back Edge.” Obviously, the purpose of the backedge counter statistics is to trigger OSR compilation.

Perform engine mode Settings

By default HotSpot VM runs with an interpreter and just-in-time compiler, which developers can change depending on the application scenario

• -xint: Executes the program entirely in interpreter mode.
• -xcomp: Executes programs entirely in just-in-time compiler mode. If there is a problem with just-in-time compilation, the interpreter steps in.
• -xmixed: Use interpreter + just-in-time compiler mixed mode to execute the program. (the default)

To view:

Interpreter and compiler performance tests

/** * Test interpreter mode and JIT compilation mode * -xint: 3974ms 3997 3973 * -xcomp: 541ms 535 533 * -xmixed: 610ms 598 601 * jdk8 64-bit 635ms 532 533 */ public class IntCompTest { public static void main(String[] args) { long start = System.currentTimeMillis(); testPrimeNumber(1000000); long end = System.currentTimeMillis(); System.out.println(" time spent: "+ (end-start)); } public static void testPrimeNumber(int count){ for (int i = 0; i < count; Label :for(int j = 2; int j = 2; j <= 100; j++){ for(int k = 2; k <= Math.sqrt(j); k++){ if(j % k == 0){ continue label; } } //System.out.println(j); }}}}Copy the code

The result of three runs for each type

        -Xint  : 3974ms 3997 3973
        -Xcomp : 541ms 535 533
        -Xmixed : 610ms 598 601
        
Copy the code

Conclusion: JIT compilation mode is significantly more efficient than interpreter mode. Mixed mode and JIT compilation mode are equivalent in this example because of the amount of code and running time involved.

See the JIT generated assembly code for reference

Print the generated assembly code from the Hotspot JIT compiler

Use HSDIS to view the JIT-generated assembly code

The VM series: Analysis JIT assembly code with an example

The AOT compiler

Java 9 introduces an experimental AOT Compiler tool, JAOTC. It uses the Graal compiler to convert the input Java class files into local machine code and store them in the generated dynamic shared library.

AOT compilation is the opposite of just-in-time compilation.

Just-in-time compilation refers to the process of converting bytecode to assembly language that can be run directly on hardware and deployed to a managed environment during the running of a program.

AOT compilation refers to the process of converting bytecode to local machine code before a program is run :.java ->.class->.so

1. advantages

• Java VIRTUAL machine loading has been precompiled into binary libraries and can be directly executed. You don’t have to wait for the just-in-time compiler to warm up, reducing the “slow first time” experience of Java applications.

2. disadvantages

• Broken Java “compile once, run everywhere”, must compile the distribution for each different hardware, OS.
• The dynamic nature of the Java linking process is reduced, and the loaded code must all be known at compile time.
• Still need to continue optimization, initially only support Linux X64 Java Base

Graal compiler

Since JDK10, HotSpot has added a new just-in-time compiler: the Graal compiler. It took only a few years to catch up with the C2 compiler. At present, with the state of “experiment” label, you need to use the switch parameters – XX: XX: + UnlockExperimentalVMOptions – + UseJVMCICompiler to activate, just can use.

conclusion

• The JIT compiles machine code with higher performance than the interpreter.
• The Server compiler takes longer to start than the Client compiler. When the system runs steadily, the Server compiler runs much faster than the Client compiler.
• -client: Specifies that the Java VIRTUAL machine runs in client mode and uses the C1 compiler
• -server: Specifies that the Java virtual machine runs in server mode and uses the C2 compiler (64-bit JDKS can only be server compiler)
• A method call counter measures the number of times a method has been called
• The backside counter is used to count the number of loops executed by the body of the loop
• The method call counter is 1500 times in Client mode and 10000 times in Server mode. When this threshold is exceeded, JIT compilation is triggered.
• -xx :CompileThreshold: sets the threshold of the method call counter
• -xx :CounterHalfLifeTime Specifies the half-aging period, in seconds.
• -xx: -usecounterdecay Turns off heat decay