Cheat on your JVM with JavaAgent

“This is my fourth day of participating in the First Challenge 2022. For more details: First Challenge 2022.”

If you are familiar with Spring, you should be familiar with AOP. Faceted programming allows you to weave the logic you want to execute before and after the target method. Java Agent technology, which is introduced to you today, is similar to AOP in thought.

Java Agent has been around since JDK1.5. It allows programmers to use Agent technology to build an application-independent Agent that can be used to monitor, run, and even replace programs on other JVMS.

If you are curious about what fairy technology can be applied in so many scenarios, today we will dig to see how the magic Java Agent works at the bottom, silently supporting so many excellent applications.

Going back to the analogy at the beginning of this article, we will use aop to compare the way, to get a general understanding of Java Agent:

Level of action: AOP runs at the method level within the application, while Agents can work at the virtual machine level
Components: AOP implementation requires the target method and the logic enhancement part of the method, while Java Agent to take effect requires two projects, one is the Agent Agent, the other is the main program needs to be propped
Execution context: AOP can run in front and back of the aspect, or around it, etc., whereas Java Agent execution has only two ways, which JDK1.5 providespreMainMode is executed before the main program runs, provided in jdk1.6agentMainExecuted after the main program runs

Let’s take a look at how to implement an Agent program in both modes.

Premain mode

The Premain mode allows an agent agent to be executed before the main program is executed. It is very simple to implement.

agent

Write a simple function to print a sentence before the main program executes and print the parameters passed to the agent:

public class MyPreMainAgent {
    public static void premain(String agentArgs, Instrumentation inst) {
        System.out.println("premain start");
        System.out.println("args:"+agentArgs); }}Copy the code

After writing the agent logic, we need to package it into jar files. Here we directly use maven plugin packaging method, before packaging some configuration.

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>3.1.0</version>
            <configuration>
                <archive>
                    <manifest>
                        <addClasspath>true</addClasspath>
                    </manifest>
                    <manifestEntries>
                        <Premain-Class>com.cn.agent.MyPreMainAgent</Premain-Class>                            
                        <Can-Redefine-Classes>true</Can-Redefine-Classes>
                        <Can-Retransform-Classes>true</Can-Retransform-Classes>
                        <Can-Set-Native-Method-Prefix>true</Can-Set-Native-Method-Prefix>
                    </manifestEntries>
                </archive>
            </configuration>
        </plugin>
    </plugins>
</build>
Copy the code

ManifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries: manifestEntries

Premain-Class: containspremainMethod class that needs to be configured as the full path of the class
Can-Redefine-Classes: in order totrue“Indicates that you can restartdefineclass
Can-Retransform-Classes: in order totrue“Indicates that you can restartconversionClass to implement bytecode substitution
Can-Set-Native-Method-Prefix: in order totrue“Indicates that the prefix of native method can be set

Premain-class is mandatory, and the other options are optional. By default, they are false, and it is usually recommended to add them. We will describe these functions later. After the configuration is complete, use the MVN command to package:

mvn clean package
Copy the code

Myagent-1.0.jar file is generated after packaging. We can unpack the jar file and take a look at the generated manifest.mf file:

You can see that the added attributes have been added to the file. Since the agent cannot run directly, it needs to be attached to other programs, so let’s create a new project to implement the main program.

The main program

In the project of the main program, all you need is an entry to the main method that can execute.

public class AgentTest {
    public static void main(String[] args) {
        System.out.println("main project start"); }}Copy the code

After the completion of the main program, it is necessary to consider how to connect the main program with the Agent project. To specify the agent to run, use the -javaAgent parameter. The command format is as follows:

java -javaagent:myAgent.jar -jar AgentTest.jar
Copy the code

In addition, there is no limit to the number of agents that can be specified. Each agent will be executed in the specified order. To run two agents at the same time, you can run the following command:

java -javaagent:myAgent1.jar -javaagent:myAgent2.jar  -jar AgentTest.jar
Copy the code

For example, if we run the program in IDEA, add startup parameters in VM options:

- javaagent: F: \ Workspace \ MyAgent \ target \ MyAgent - 1.0. Jar = Hydra - javaagent: F: \ Workspace \ MyAgent \ target \ MyAgent - 1.0. Jar = TrunksCopy the code

Execute the main method to view the output:

According to the print statement of the execution result, we can see that our agent agent was executed twice successively before executing the main program. The following diagram shows the order in which the execution agent is executed compared to the main program.

defects

In addition to providing convenience, the Premain mode also has some defects. For example, if an exception occurs during the operation of the Agent, the main program will fail to start. Let’s modify the agent code in the above example to manually throw an exception.

public static void premain(String agentArgs, Instrumentation inst) {
    System.out.println("premain start");
    System.out.println("args:"+agentArgs);
    throw new RuntimeException("error");
}
Copy the code

Run the main program again:

As you can see, the main program is not started after the Agent throws an exception. To address some of the drawbacks of premain, the agentmain pattern was introduced after jdk1.6.

Agentmain mode

The AgentMain pattern is an upgraded version of PreMain. It allows the agent’s target JVM to start first and then attach to the two JVMS, which we implement in three parts.

agent

The Agent part is the same as above to achieve simple printing function:

public class MyAgentMain {
    public static void agentmain(String agentArgs, Instrumentation instrumentation) {
        System.out.println("agent main start");
        System.out.println("args:"+agentArgs); }}Copy the code

Modify maven plugin configuration to specify agent-class:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.1.0</version>
    <configuration>
        <archive>
            <manifest>
                <addClasspath>true</addClasspath>
            </manifest>
            <manifestEntries>
                <Agent-Class>com.cn.agent.MyAgentMain</Agent-Class>
                <Can-Redefine-Classes>true</Can-Redefine-Classes>
                <Can-Retransform-Classes>true</Can-Retransform-Classes>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>
Copy the code

The main program

Here we directly start the main program and wait for the agent to be loaded. We block the main program using system. in to prevent the main process from ending prematurely.

public class AgentmainTest {
    public static void main(String[] args) throws IOException { System.in.read(); }}Copy the code

The attach mechanism

Unlike premain, we can no longer connect the agent to the main program by adding startup parameters. Instead, we need to use the VirtualMachine utility class provided by com.sun.tools.attach. Dependencies need to be introduced before use:

<dependency>
    <groupId>com.sun</groupId>
    <artifactId>tools</artifactId>
    <version>1.8</version>
    <scope>system</scope>
    <systemPath>${JAVA_HOME}\lib\tools.jar</systemPath>
</dependency>
Copy the code

VirtualMachine represents a Java VirtualMachine to be attached to, that is, the target vm to be monitored in the program. External processes can use the VirtualMachine instance to load the agent to the target vm. First take a look at its static attach method:

public static VirtualMachine attach(String var0);
Copy the code

The ATTACH method gets an object instance of the JVM, where the parameter passed in is the process id PID of the target VM. In other words, before using attach, we need to obtain the PID of the main program just started. Use the JPS command to check the thread PID:

11140
16372 RemoteMavenServer36
16392 AgentmainTest
20204 Jps
2460 Launcher
Copy the code

Get the runtime PID of the main program AgentmainTest is 16392 and apply it to the virtual machine connection.

public class AttachTest {
    public static void main(String[] args) {
        try {
            VirtualMachine  vm= VirtualMachine.attach("16392");
            vm.loadAgent("F: \ \ Workspace \ \ MyAgent \ \ target \ \ MyAgent - 1.0. The jar." "."param");
        } catch(Exception e) { e.printStackTrace(); }}}Copy the code

After obtaining the VirtualMachine instance, you can use the loadAgent method to inject the Agent proxy class. The first parameter of the method is the local path of the agent, and the second parameter is passed to the agent. Perform AttachTest, back to the main program AgentmainTest console, you can see executing code in the procession of the agent:

This completes the implementation of a simple agentMain pattern agent, which can be further illustrated by the following diagram.

application

At this point, we’ve looked briefly at how to implement both patterns, but as quality programmers, we can’t be content with simply printing statements using agents. Let’s look at how we can use Java Agents to do something practical.

In the above two modes, the agent part of the logic is implemented in the Premain and AgentMain methods respectively, and both methods have strict requirements on the parameters of the signature. The premain method can be defined in the following two ways:

public static void premain(String agentArgs)
public static void premain(String agentArgs, Instrumentation inst)
Copy the code

The AgentMain method allows you to define it in two ways:

public static void agentmain(String agentArgs)
public static void agentmain(String agentArgs, Instrumentation inst)
Copy the code

If there are two signature methods in the Agent, the method with Instrumentation parameters has higher priority and will be loaded by the JVM first. Its instance inST will be automatically injected by the JVM. Let’s see what can be implemented with Instrumentation.

Instrumentation

The Instrumentation interface is introduced in general. The methods allow Java to operate at run time, providing functions such as changing bytecode, adding JAR packages, replacing classes, and so on. Through these functions, Java has stronger dynamic control and interpretation capabilities. In the process of writing agent agent, Instrumentation in the following three methods are more important and commonly used, we will focus on.

addTransformer

The addTransformer method allows us to redefine the Class before the Class is loaded. First look at the method definition:

void addTransformer(ClassFileTransformer transformer);
Copy the code

ClassFileTransformer is an interface that has only a transform method that executes every class loaded by a transform once before the main method of the main program is executed. This can be called a transformer. We can implement this method to redefine the Class. Here’s an example of how to use it.

First, create a Fruit class in the main program:

public class Fruit {
    public void getFruit(a){
        System.out.println("banana"); }}Copy the code

Fruit2.class = ‘fruit2.class’; fruit2.class = ‘Fruit’;

public void getFruit(a){
    System.out.println("apple");
}
Copy the code

Create the main program, create a Fruit object in the main program and call its getFruit method:

public class TransformMain {
    public static void main(String[] args) {
        newFruit().getFruit(); }}Copy the code

The result will print the apple, and the implementation of the Premain agent section will begin.

In the premain method of the proxy, use the addTransformer method of Instrumentation to intercept class loading:

public class TransformAgent {
    public static void premain(String agentArgs, Instrumentation inst) {
        inst.addTransformer(newFruitTransformer()); }}Copy the code

The FruitTransformer class implements the ClassFileTransformer interface and transforms the class portion of the logic in the transform method:

public class FruitTransformer implements ClassFileTransformer {
    @Override
    public byte[] transform(ClassLoader loader, String className, Class<? > classBeingRedefined, ProtectionDomain protectionDomain,byte[] classfileBuffer){
        if(! className.equals("com/cn/hydra/test/Fruit"))
            return classfileBuffer;

        String fileName="F:\\Workspace\\agent-test\\target\\classes\\com\\cn\\hydra\\test\\Fruit2.class";
        return getClassBytes(fileName);
    }

    public static byte[] getClassBytes(String fileName){
        File file = new File(fileName);
        try(InputStream is = new FileInputStream(file);
            ByteArrayOutputStream bs = new ByteArrayOutputStream()){
            long length = file.length();
            byte[] bytes = new byte[(int) length];

            int n;
            while((n = is.read(bytes)) ! = -1) {
                bs.write(bytes, 0, n);
            }
            return bytes;
        }catch (Exception e) {
            e.printStackTrace();
            return null; }}}Copy the code

In the transform method, two main things are done:

becauseaddTransformerMethod does not specify the class to be converted, so passesclassNameCheck whether the currently loaded class is the target class that we want to intercept. For non-target classes, return the original byte arrayclassNameNeed to fully qualify the class name in.Replace with/
Read the class file that we copied earlier, read in the binary character stream, and replace the originalclassfileBufferByte array and return, completing the class definition replacement

After the agent part is packaged, add startup parameters in the main program:

- javaagent: F: \ Workspace \ MyAgent \ target \ transformAgent - 1.0. The jarCopy the code

Execute the main program again and print:

banana
Copy the code

This completes the class substitution before the main method is executed.

redefineClasses

We can intuitively understand its function from the name of the method, redefine the class, in plain English, implement the replacement of the specified class. The method is defined as follows:

void redefineClasses(ClassDefinition... definitions) throws  ClassNotFoundException, UnmodifiableClassException;
Copy the code

It takes a variable-length array of ClassDefinitions. Let’s look at the constructor of classDefinitions:

public ClassDefinition(Class<? > theClass,byte[] theClassFile) {... }Copy the code

The Class object specified in ClassDefinition and the modified bytecode array simply replace the original Class with the supplied Class file bytes. Also, when the redefineClasses method redefines, it passes in an array of ClassDefinitions, which it loads in order of the array to allow changes to be made if classes depend on each other.

Here’s an example of how it works, the premain proxy part:

public class RedefineAgent {
    public static void premain(String agentArgs, Instrumentation inst) 
            throws UnmodifiableClassException, ClassNotFoundException {
        String fileName="F:\\Workspace\\agent-test\\target\\classes\\com\\cn\\hydra\\test\\Fruit2.class";
        ClassDefinition def=new ClassDefinition(Fruit.class,
                FruitTransformer.getClassBytes(fileName));
        inst.redefineClasses(newClassDefinition[]{def}); }}Copy the code

The main program can directly reuse the above, print after execution:

banana
Copy the code

As you can see, the substitution of the specified class is implemented by replacing the original class with the bytes of the specified class file.

retransformClasses

RetransformClasses are applied to the AgentMain pattern to redefine the Class after it is loaded, that is, to trigger a reloading of the Class. Let’s first look at the definition of this method:

void retransformClasses(Class
       ... classes) throws UnmodifiableClassException;
Copy the code

The classes argument is the class array to convert, and the variable length argument indicates that it can convert class definitions in bulk, just like the redefineClasses method.

Here is an example of how to use the retransformClasses method:

public class RetransformAgent {
    public static void agentmain(String agentArgs, Instrumentation inst)
            throws UnmodifiableClassException {
        inst.addTransformer(new FruitTransformer(),true);
        inst.retransformClasses(Fruit.class);
        System.out.println("retransform success"); }}Copy the code

Take a look at the definition of the addTransformer method called here, which is slightly different:

void addTransformer(ClassFileTransformer transformer, boolean canRetransform);
Copy the code

If the canRetransform is true, it will allow the class to be redefined. This is equivalent to calling the transform method in ClassFileTransformer, which loads the bytes of the converted class as the new class definition.

The main program part of the code, we constantly execute print statements in an infinite loop to monitor if the class has changed:

public class RetransformMain {
    public static void main(String[] args) throws InterruptedException {
        while(true) {new Fruit().getFruit();
            TimeUnit.SECONDS.sleep(5); }}}Copy the code

Finally, use the Attach API to inject agent into the main application:

public class AttachRetransform {
    public static void main(String[] args) throws Exception {
        VirtualMachine vm = VirtualMachine.attach("6380");
        vm.loadAgent("F: \ \ Workspace \ \ MyAgent \ \ target \ \ retransformAgent - 1.0. The jar." "); }}Copy the code

Go back to the main program console and see the results:

You can see that the print statement changes after the agent is injected, indicating that the class definition has been changed and reloaded.

other

In addition to the main methods, there are some other methods in Instrumentation, here is just a simple list of the functions of common methods:

removeTransformer: Delete aClassFileTransformerType of converter
getAllLoadedClasses: Gets the currently loaded Class
getInitiatedClasses: Gets the specifiedClassLoaderTo load the Class
getObjectSize: Gets the amount of space an object occupies
appendToBootstrapClassLoaderSearch: Adds the JAR package to the startup class loader
appendToSystemClassLoaderSearch: Adds jar packages to the system classloader
isNativeMethodPrefixSupported: Determines whether native methods can be prefixed, that is, whether native methods can be intercepted
setNativeMethodPrefix: Sets the prefix of native methods

Javassist

In the above examples, we read the bytes directly from the class file to redefine or transform the class, but in a real world environment, where you might want to modify the bytecode of the class file on the fly, you can use JavAssist to modify the bytecode file more easily.

To put it simply, Javassist is a library for analyzing, editing, and creating Java bytecode, which provides apis that can be invoked directly to dynamically change or generate class structures in coded form. Compared to other bytecode frameworks, such as ASM, that require understanding of the underlying virtual machine instructions, JavAssist is really simple and fast.

Let’s look at a simple example of how Java Agent and Javassist can be used together. First, we introduce javassist’s dependency:

<dependency>
    <groupId>org.javassist</groupId>
    <artifactId>javassist</artifactId>
    <version>3.20.0 - GA</version>
</dependency>
Copy the code

So what we’re going to do is we’re going to calculate the execution time of the method by proxy. The premain proxy part is basically the same as before, adding a converter first:

public class Agent {
    public static void premain(String agentArgs, Instrumentation inst) {
        inst.addTransformer(new LogTransformer());
    }

    static class LogTransformer implements ClassFileTransformer {
        @Override
        public byte[] transform(ClassLoader loader, String className, Class<? > classBeingRedefined, ProtectionDomain protectionDomain,byte[] classfileBuffer) 
            throws IllegalClassFormatException {
            if(! className.equals("com/cn/hydra/test/Fruit"))
                return null;

            try {
                return calculate();
            } catch (Exception e) {
                e.printStackTrace();
                return null; }}}}Copy the code

In the Calculate method, using Javassist dynamically changes the method definition:

static byte[] calculate() throws Exception {
    ClassPool pool = ClassPool.getDefault();
    CtClass ctClass = pool.get("com.cn.hydra.test.Fruit");
    CtMethod ctMethod = ctClass.getDeclaredMethod("getFruit");
    CtMethod copyMethod = CtNewMethod.copy(ctMethod, ctClass, new ClassMap());
    ctMethod.setName("getFruit$agent");

    StringBuffer body = new StringBuffer("{\n")
            .append("long begin = System.nanoTime(); \n")
            .append("getFruit$agent($$); \n")
            .append("System.out.println(\"use \"+(System.nanoTime() - begin) +\" ns\"); \n")
            .append("}");
    copyMethod.setBody(body.toString());
    ctClass.addMethod(copyMethod);
    return ctClass.toBytecode();
}
Copy the code

In the above code, these functions are mainly implemented:

Gets a class with a fully qualified nameCtClass
Get methods by method nameCtMethod, and through theCtNewMethod.copyMethod copies a new method
The name of the method to modify the old methodgetFruit$agent
throughsetBodyMethod changes the contents of the copied method, makes logical enhancements in the new method, calls the old method, and finally adds the new method to the class

The main program still reuses the previous code, executes the view result, completes the execution time statistics function in the agent:

At this point we can look at it again through reflection:

for (Method method : Fruit.class.getDeclaredMethods()) {
    System.out.println(method.getName());
    method.invoke(new Fruit());
    System.out.println("-- -- -- -- -- -- --");
}
Copy the code

Looking at the results, you can see that a method has indeed been added to the class:

In addition, Javassist has many other functions, such as creating classes, setting superclasses, reading and writing bytecodes, etc. You can learn how to use javassist in specific scenarios.

conclusion

Although we may not use Java Agent directly in many scenarios in our daily work, they may be hidden in the corner of the business system, such as hot deployment, monitoring, performance analysis and so on, and have been quietly playing a huge role.

This paper starts with two modes of Java Agent, manually implements and briefly analyzes their workflow. Although they only use them to complete some simple functions here, it has to be said that it is the emergence of Java Agent that makes the operation of programs no longer follow rules and regulations, and also provides infinite possibilities for our code.

The last

If you feel helpful, you can click a “like” ah, thank you very much ~

Nongcanshang, an interesting, in-depth and direct public account that loves sharing, will talk to you about technology. Welcome to Hydra as a “like” friend

Premain mode

agent

The main program

defects

Agentmain mode

agent

The main program

The attach mechanism

application

Instrumentation

addTransformer

redefineClasses

retransformClasses

other

Javassist

conclusion

The last

Related Posts

Prometheus&Grafana performance monitoring

SpringBoot Aspect Oriented Programming (AOP)

1 billion order system sub – database sub – table design ideas!