Historical articles:

  1. OGNL syntax specification
  2. Vanishing stack
  3. Arthas Principles series 1: Implement a minimalist Watch command using the ATTACH mechanism of the JVM
  4. Arthas Principles Series 2: Overall architecture and project entry
  5. Arthas Principles Series 3: Server startup process
  6. Arthas Principles series 4: Bytecode instrumentation makes everything possible

preface

In previous articles, we can see the watch command on the original bytecode instrumentation, and to train of thought as we implemented a simplified version of the watch command, but the real watch’s ability to provide more than statistical method of running time, we are the most commonly used one of his functions and return values such as observation method in variable at runtime. All the commands need instrumented bytecode inherited EnhancerCommand, has a total of five commands, respectively is the monitor, stack, tt, watch, trace, this article will show you how the arthas for so many orders to design a unified process instrumentation.

The main process of instrumentation

In the previous article, we looked at the process method of the watch command:

@Override
public void process(final CommandProcess process) {
    // ctrl-C support
    process.interruptHandler(new CommandInterruptHandler(process));
    // q exit support
    process.stdinHandler(new QExitHandler(process));

    // start to enhance
    enhance(process);
}
Copy the code

You can see that the entry point to the entire instrumentation is the Enhance method, which is implemented in the parent EnhancerCommand class:

protected void enhance(CommandProcess process) {
    Session session = process.session();
    if(! session.tryLock()) { String msg ="someone else is enhancing classes, pls. wait.";
        process.appendResult(new EnhancerModel(null.false, msg));
        process.end(-1, msg);
        return;
    }
    EnhancerAffect effect = null;
    int lock = session.getLock();
    try {
        Instrumentation inst = session.getInstrumentation();
        // Get the AdviceListener specific to each command, which is injected into the target JVM during instrumentation
        AdviceListener listener = getAdviceListenerWithId(process);
        if (listener == null) {
            logger.error("advice listener is null");
            String msg = "advice listener is null, check arthas log";
            process.appendResult(new EnhancerModel(effect, false, msg));
            process.end(-1, msg);
            return;
        }
        boolean skipJDKTrace = false;
        if(listener instanceof AbstractTraceAdviceListener) {
            // If it is trace, determine whether you need to skip the methods provided by the JDK
            skipJDKTrace = ((AbstractTraceAdviceListener) listener).getCommand().isSkipJDKTrace();
        }

        Enhancer enhancer = new Enhancer(listener, listener instanceof InvokeTraceable, skipJDKTrace, getClassNameMatcher(), getMethodNameMatcher());
        // Register notification listeners
        process.register(listener, enhancer);
        // Start inserting
        effect = enhancer.enhance(inst);

        if(effect.getThrowable() ! =null) {
            String msg = "error happens when enhancing class: "+effect.getThrowable().getMessage();
            process.appendResult(new EnhancerModel(effect, false, msg));
            process.end(1, msg + ", check arthas log: " + LogUtil.loggingFile());
            return;
        }

        if (effect.cCnt() == 0 || effect.mCnt() == 0) {
            // no class effected
            // might be method code too large
            process.appendResult(new EnhancerModel(effect, false."No class or method is affected"));
            String msg = "No class or method is affected, try:\n"
                    + "1. sm CLASS_NAME METHOD_NAME to make sure the method you are tracing actually exists (it might be in your parent class).\n"
                    + "2. reset CLASS_NAME and try again, your method body might be too large.\n"
                    + "3. check arthas log: " + LogUtil.loggingFile() + "\n"
                    + "4. visit https://github.com/alibaba/arthas/issues/47 for more details.";
            process.end(-1, msg);
            return;
        }

        // As a compensation, if the unLock is called during Enhance, a compensatory waiver is given
        if (session.getLock() == lock) {
            if (process.isForeground()) {
                process.echoTips(Constants.Q_OR_CTRL_C_ABORT_MSG + "\n");
            }
        }

        process.appendResult(new EnhancerModel(effect, true));

        // Execute asynchronously, ending in AdviceListener
    } catch (Throwable e) {
        String msg = "error happens when enhancing class: "+e.getMessage();
        logger.error(msg, e);
        process.appendResult(new EnhancerModel(effect, false, msg));
        process.end(-1, msg);
    } finally {
        if (session.getLock() == lock) {
            // Unlock after enhanceprocess.session().unLock(); }}}Copy the code

This code is a bit long, is the most important thing in two places, one is the call getAdviceListenerWithId won a AdviceListener, this class implements the befor, arterReturning, methods of afterThrowing, Is injected into the target JVM by an instrumented function. The other is to create an Enhancer object and start instrumenting the classes and methods of the target JVM.

public synchronized EnhancerAffect enhance(final Instrumentation inst) throws UnmodifiableClassException {
    // Get the set of classes to be enhanced
    this.matchingClasses = GlobalOptions.isDisableSubClass
            ? SearchUtils.searchClass(inst, classNameMatcher)
            : SearchUtils.searchSubClass(inst, SearchUtils.searchClass(inst, classNameMatcher));

    // Filter out classes that cannot be enhanced
    filter(matchingClasses);

    logger.info("enhance matched classes: {}", matchingClasses);

    affect.setTransformer(this);

    try {
        ArthasBootstrap.getInstance().getTransformerManager().addTransformer(this, isTracing);

        // Batch enhancement
        if (GlobalOptions.isBatchReTransform) {
            final int size = matchingClasses.size();
            finalClass<? >[] classArray =newClass<? >[size]; arraycopy(matchingClasses.toArray(),0, classArray, 0, size);
            if (classArray.length > 0) {
                inst.retransformClasses(classArray);
                logger.info("Success to batch transform classes: "+ Arrays.toString(classArray)); }}else {
            // for each
            for(Class<? > clazz : matchingClasses) {try {
                    inst.retransformClasses(clazz);
                    logger.info("Success to transform class: " + clazz);
                } catch (Throwable t) {
                    logger.warn("retransform {} failed.", clazz, t);
                    if (t instanceof UnmodifiableClassException) {
                        throw (UnmodifiableClassException) t;
                    } else if (t instanceof RuntimeException) {
                        throw (RuntimeException) t;
                    } else {
                        throw new RuntimeException(t);
                    }
                }
            }
        }
    } catch (Throwable e) {
        logger.error("Enhancer error, matchingClasses: {}", matchingClasses, e);
        affect.setThrowable(e);
    }

    return affect;
}
Copy the code

The enhance method implementation will be familiar if you have seen the previous article implementing the Watch command. The most important thing in this method is to register this class as a transformation class and call the Instrumentation retransformClasses method to convert the class. With this mechanism, it’s the Transform method that does the real work.

@Override
public byte[] transform(finalClassLoader inClassLoader, String className, Class<? > classBeingRedefined, ProtectionDomain protectionDomain,byte[] classfileBuffer) throws IllegalClassFormatException {
    try {
        // Check if classLoader can load into SpyAPI, if not, abandon enhancement
        try {
            if(inClassLoader ! =null) { inClassLoader.loadClass(SpyAPI.class.getName()); }}catch (Throwable e) {
            logger.error("the classloader can not load SpyAPI, ignore it. classloader: {}, className: {}",
                    inClassLoader.getClass().getName(), className);
            return null;
        }

        // Filter again, why? It is possible that new classes will be created during the transform process
        // So you need to pass in the set of classes that need to be converted
        if(matchingClasses ! =null && !matchingClasses.contains(classBeingRedefined)) {
            return null;
        }

        //keep origin class reader for bytecode optimizations, avoiding JVM metaspace OOM.
        ClassNode classNode = new ClassNode(Opcodes.ASM9);
        ClassReader classReader = AsmUtils.toClassNode(classfileBuffer, classNode);
        // remove JSR https://github.com/alibaba/arthas/issues/1304
        classNode = AsmUtils.removeJSRInstructions(classNode);

        // Generate enhanced bytecode
        // Point 1: Build interceptors
        DefaultInterceptorClassParser defaultInterceptorClassParser = new DefaultInterceptorClassParser();

        final List<InterceptorProcessor> interceptorProcessors = new ArrayList<InterceptorProcessor>();

        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor1.class));
        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor2.class));
        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor3.class));

        if (this.isTracing) {
            if (this.skipJDKTrace == false) {
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor1.class));
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor2.class));
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor3.class));
            } else {
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor1.class));
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor2.class));
                interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor3.class));
            }
        }

        List<MethodNode> matchedMethods = new ArrayList<MethodNode>();
        for (MethodNode methodNode : classNode.methods) {
            if (!isIgnore(methodNode, methodNameMatcher)) {
                matchedMethods.add(methodNode);
            }
        }

        // Check to see if the SPY function has been inserted and do not repeat the process if it has
        // The GroupLocationFilter is not very important, its purpose is to prevent repeated interpolation
        GroupLocationFilter groupLocationFilter = new GroupLocationFilter();

        LocationFilter enterFilter = new InvokeContainLocationFilter(Type.getInternalName(SpyAPI.class), "atEnter",
                LocationType.ENTER);
        LocationFilter existFilter = new InvokeContainLocationFilter(Type.getInternalName(SpyAPI.class), "atExit",
                LocationType.EXIT);
        LocationFilter exceptionFilter = new InvokeContainLocationFilter(Type.getInternalName(SpyAPI.class),
                "atExceptionExit", LocationType.EXCEPTION_EXIT);

        groupLocationFilter.addFilter(enterFilter);
        groupLocationFilter.addFilter(existFilter);
        groupLocationFilter.addFilter(exceptionFilter);

        LocationFilter invokeBeforeFilter = new InvokeCheckLocationFilter(Type.getInternalName(SpyAPI.class),
                "atBeforeInvoke", LocationType.INVOKE);
        LocationFilter invokeAfterFilter = new InvokeCheckLocationFilter(Type.getInternalName(SpyAPI.class),
                "atInvokeException", LocationType.INVOKE_COMPLETED);
        LocationFilter invokeExceptionFilter = new InvokeCheckLocationFilter(Type.getInternalName(SpyAPI.class),
                "atInvokeException", LocationType.INVOKE_EXCEPTION_EXIT);
        groupLocationFilter.addFilter(invokeBeforeFilter);
        groupLocationFilter.addFilter(invokeAfterFilter);
        groupLocationFilter.addFilter(invokeExceptionFilter);

        for (MethodNode methodNode : matchedMethods) {
            // First check if there is atBeforeInvoke. If there is atBeforeInvoke, trace is already present
            if(AsmUtils.containsMethodInsnNode(methodNode, Type.getInternalName(SpyAPI.class), "atBeforeInvoke")) {
                for(AbstractInsnNode insnNode = methodNode.instructions.getFirst(); insnNode ! =null; insnNode = insnNode
                        .getNext()) {
                    if (insnNode instanceof MethodInsnNode) {
                        final MethodInsnNode methodInsnNode = (MethodInsnNode) insnNode;
                        if(this.skipJDKTrace) {
                            if(methodInsnNode.owner.startsWith("java/")) {
                                continue; }}// All boxes of primitive types are skipped
                        if(AsmOpUtils.isBoxType(Type.getObjectType(methodInsnNode.owner))) {
                            continue; } AdviceListenerManager.registerTraceAdviceListener(inClassLoader, className, methodInsnNode.owner, methodInsnNode.name, methodInsnNode.desc, listener); }}}else {
                MethodProcessor methodProcessor = new MethodProcessor(classNode, methodNode, groupLocationFilter);
                for (InterceptorProcessor interceptor : interceptorProcessors) {
                    try {
                        // Point 2: The real assembly work is done here
                        List<Location> locations = interceptor.process(methodProcessor);
                        for (Location location : locations) {
                            if (location instanceofMethodInsnNodeWare) { MethodInsnNodeWare methodInsnNodeWare = (MethodInsnNodeWare) location; MethodInsnNode methodInsnNode = methodInsnNodeWare.methodInsnNode(); AdviceListenerManager.registerTraceAdviceListener(inClassLoader, className, methodInsnNode.owner, methodInsnNode.name, methodInsnNode.desc, listener); }}}catch (Throwable e) {
                        logger.error("enhancer error, class: {}, method: {}, interceptor: {}", classNode.name, methodNode.name, interceptor.getClass().getName(), e); }}}// Enter /exist always inserts a listener
            AdviceListenerManager.registerAdviceListener(inClassLoader, className, methodNode.name, methodNode.desc,
                    listener);
            affect.addMethodAndCount(inClassLoader, className, methodNode.name, methodNode.desc);
        }

        / / https://github.com/alibaba/arthas/issues/1223, V1_5 major version is 49
        if (AsmUtils.getMajorVersion(classNode.version) < 49) {
            classNode.version = AsmUtils.setMajorVersion(classNode.version, 49);
        }

        byte[] enhanceClassByteArray = AsmUtils.toBytes(classNode, inClassLoader, classReader);

        // The enhancement succeeded, recording the class
        classBytesCache.put(classBeingRedefined, new Object());

        // dump the class
        dumpClassIfNecessary(className, enhanceClassByteArray, affect);

        // Success count
        affect.cCnt(1);

        return enhanceClassByteArray;
    } catch (Throwable t) {
        logger.warn("transform loader[{}]:class[{}] failed.", inClassLoader, className, t);
        affect.setThrowable(t);
    }

    return null;
}
Copy the code

The first step is to register an InterceptorProcessor list of interceptors. The interceptor’s function is to determine where the plug-in code can be injected. We will focus on this later. The second step is to register a list of filters, filter logic is relatively simple, is to avoid repeated instrumentation; The third step is to call each processor’s process for the specific instrumentation logic.

How is the InterceptorProcessor generated

 // Generate enhanced bytecode
DefaultInterceptorClassParser defaultInterceptorClassParser = new DefaultInterceptorClassParser();

final List<InterceptorProcessor> interceptorProcessors = new ArrayList<InterceptorProcessor>();
interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor1.class));
interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor2.class));
interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyInterceptor3.class));

if (this.isTracing) {
    if (this.skipJDKTrace == false) {
        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor1.class));
        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor2.class));
        interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceInterceptor3.class));
    } else{ interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor1.class)); interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor2.class)); interceptorProcessors.addAll(defaultInterceptorClassParser.parse(SpyTraceExcludeJDKInterceptor3.class)); }}Copy the code

This code snippet is important to understand how the Arthas interceptor works, First we see the com. Alibaba. Bytekit. Asm. Interceptor. Parser. DefaultInterceptorClassParser# parse logic:

@Override
public List<InterceptorProcessor> parse(Class
        clazz) {
    final List<InterceptorProcessor> result = new ArrayList<InterceptorProcessor>();

    MethodCallback methodCallback = new MethodCallback() {

        @Override
        public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException {
            for (Annotation onMethodAnnotation : method.getAnnotations()) {
                for (Annotation onAnnotation : onMethodAnnotation.annotationType().getAnnotations()) {
                    if (InterceptorParserHander.class.isAssignableFrom(onAnnotation.annotationType())) {

                        if(! Modifier.isStatic(method.getModifiers())) {throw new IllegalArgumentException("method must be static. method: "+ method); } InterceptorParserHander handler = (InterceptorParserHander) onAnnotation; InterceptorProcessorParser interceptorProcessorParser = InstanceUtils .newInstance(handler.parserHander()); InterceptorProcessor interceptorProcessor = interceptorProcessorParser.parse(method, onMethodAnnotation); result.add(interceptorProcessor); }}}}}; ReflectionUtils.doWithMethods(clazz, methodCallback);return result;
}
Copy the code

If the InterceptorParserHander is InterceptorParserHander, the parse method is called. If the InterceptorParserHander is InterceptorParserHander, the parse method is called. Then we will see to the com. Alibaba. Bytekit. Asm. Interceptor. Parser. DefaultInterceptorClassParser# parse several categories:

public class SpyInterceptors {

    public static class SpyInterceptor1 {

        @AtEnter(inline = true)
        public static void atEnter(@Binding.This Object target, @Binding.Class Class<? > clazz,@Binding.MethodInfo String methodInfo, @Binding.Args Object[] args) { SpyAPI.atEnter(clazz, methodInfo, target, args); }}}Copy the code

In fact, SpyInterceptors contain a large number of internal classes, which are generally divided into three categories: common instrumentation, common instrumentation for trace commands, and trace commands that filter JDK methods. In each class, there are methods that are instrumentated before a method call, methods that are instrumentated when a method returns. And three small classes for instrumenting method exceptions. For the sake of space, we show only one class, SpyInterceptor1.

The @atenter annotation has an inline parameter, which controls whether the inserted method needs to be inline with the target method. If you open the @atenter annotation, you will find that it is actually an InterceptorParserHander annotation:

@Documented
@Retention(RetentionPolicy.RUNTIME)
@java.lang.annotation.Target(ElementType.METHOD)
@InterceptorParserHander(parserHander = EnterInterceptorProcessorParser.class)
public @interface AtEnter {
    boolean inline(a) default true;

    Class<? extends Throwable> suppress() defaultNone.class; Class<? > suppressHandler()default Void.class;

    class EnterInterceptorProcessorParser implements InterceptorProcessorParser {

        @Override
        public InterceptorProcessor parse(Method method, Annotation annotationOnMethod) {

            // The purpose of the LocationMatcher is to avoid repeated instrumentation
            LocationMatcher locationMatcher = new EnterLocationMatcher();

            AtEnter atEnter = (AtEnter) annotationOnMethod;

            returnInterceptorParserUtils.createInterceptorProcessor(method, locationMatcher, atEnter.inline(), atEnter.suppress(), atEnter.suppressHandler()); }}}Copy the code

In front of the analysis we can see, the registered annotator work will be forwarded to the actually InterceptorParserHander parse method, do things in this method is simpler, the annotation the parameters in the forward to createInterceptorProcessor this method:

public static InterceptorProcessor createInterceptorProcessor(
        Method method,
        LocationMatcher locationMatcher,
        booleaninline, Class<? extends Throwable> suppress, Class<? > suppressHandler) {

    InterceptorProcessor interceptorProcessor = new InterceptorProcessor(method.getDeclaringClass().getClassLoader());

    //locationMatcher
    interceptorProcessor.setLocationMatcher(locationMatcher);

    //interceptorMethodConfig
    InterceptorMethodConfig interceptorMethodConfig = new InterceptorMethodConfig();
    interceptorProcessor.setInterceptorMethodConfig(interceptorMethodConfig);
    interceptorMethodConfig.setOwner(Type.getInternalName(method.getDeclaringClass()));
    interceptorMethodConfig.setMethodName(method.getName());
    interceptorMethodConfig.setMethodDesc(Type.getMethodDescriptor(method));

    //inline
    interceptorMethodConfig.setInline(inline);

    //bindings
    List<Binding> bindings = BindingParserUtils.parseBindings(method);
    interceptorMethodConfig.setBindings(bindings);

    //errorHandlerMethodConfig
    InterceptorMethodConfig errorHandlerMethodConfig = ExceptionHandlerUtils
            .errorHandlerMethodConfig(suppress, suppressHandler);
    if(errorHandlerMethodConfig ! =null) {
        interceptorProcessor.setExceptionHandlerConfig(errorHandlerMethodConfig);
    }

    return interceptorProcessor;
}
Copy the code

This method is also fairly clear:

  1. A filter is registered with the interceptor to prevent repeated instrumentation
  2. Create a new oneInterceptorMethodConfigClass that contains details about the method to be instrumented, as shown heremethodVariables arecom.taobao.arthas.core.advisor.SpyInterceptors.SpyInterceptor1#atEntermethods
  3. The next important step is the generation of interceptorsBindingObject, which we’ll talk more about later
  4. Generate exception handlers for interceptors, principles, and handlingBindingAnnotations are similar
public static List<Binding> parseBindings(Method method) {
    // select binding from parameter
    List<Binding> bindings = new ArrayList<Binding>();
    Annotation[][] parameterAnnotations = method.getParameterAnnotations();
    for (int parameterIndex = 0; parameterIndex < parameterAnnotations.length; ++parameterIndex) {
        Annotation[] annotationsOnParameter = parameterAnnotations[parameterIndex];
        for (int j = 0; j < annotationsOnParameter.length; ++j) {

            Annotation[] annotationsOnBinding = annotationsOnParameter[j].annotationType().getAnnotations();
            for (Annotation annotationOnBinding : annotationsOnBinding) {
                if(BindingParserHandler.class.isAssignableFrom(annotationOnBinding.annotationType())) { BindingParserHandler bindingParserHandler = (BindingParserHandler) annotationOnBinding; BindingParser bindingParser = InstanceUtils.newInstance(bindingParserHandler.parser()); Binding binding = bindingParser.parse(annotationsOnParameter[j]); bindings.add(binding); }}}}return bindings;
}
Copy the code

Binding annotations are handled in this method by taking the Parser attribute from the BindingParserHandler annotation and calling the parse method of that attribute to generate a Binding object

How do the Binding annotations work

Starting from this section, of a change in our thinking, thinking from the Angle of the bytecode, because our approach is the runtime cartridge into the target method, thus some runtime information of the target method also should from the runtime code form – byte code, below we analyze the Binding of some common annotations:

ArgsBindingThe working principle of

@Documented
@Retention(RetentionPolicy.RUNTIME)
@java.lang.annotation.Target(ElementType.PARAMETER)
@BindingParserHandler(parser = ArgsBindingParser.class)
public static @interface Args {
    
    boolean optional(a) default false;

}
public static class ArgsBindingParser implements BindingParser {
    @Override
    public Binding parse(Annotation annotation) {
        return newArgsBinding(); }}Copy the code

From the previous section, we can see that the last Binding object returned is ArgsBinding. Let’s look at its implementation:

public class ArgsBinding extends Binding {

    @Override
    public void pushOntoStack(InsnList instructions, BindingContext bindingContext) {
        AsmOpUtils.loadArgArray(instructions, bindingContext.getMethodProcessor().getMethodNode());
    }

    @Override
    public Type getType(BindingContext bindingContext) {
        returnAsmOpUtils.OBJECT_ARRAY_TYPE; }}Copy the code

We can see that both methods inherit from the Binding object, which means that all Binding objects have these two methods. The second method is easier to understand and identifies the type. The first method is to store the information we need in a local variable by manipulating the stack and the local variable table.

public static void loadArgArray(final InsnList instructions, MethodNode methodNode) {
    boolean isStatic = AsmUtils.isStatic(methodNode);
    Type[] argumentTypes = Type.getArgumentTypes(methodNode.desc);
    push(instructions, argumentTypes.length);
    newArray(instructions, OBJECT_TYPE);
    for (int i = 0; i < argumentTypes.length; i++) {
        // Copy the array reference from the previous step to the top of the stack
        dup(instructions);
        // Push the number I to the top of the stack
        push(instructions, i);
        // Load the ith entry in the local variable table to the top of the stack
        loadArg(isStatic, instructions, argumentTypes, i);
        / / syntactic sugar
        box(instructions, argumentTypes[i]);
        // Save the ith element of the arrayarrayStore(instructions, OBJECT_TYPE); }}Copy the code

This code is able to fully understand achievement is a byte code, we can see some familiar bytecode operator dup, for example, a push, and so on, the only difference is that every operator have a InsnList into, this into the parameter is a bytecode instruction of two-way linked list, because we want to instrumentation, so our bytecode must also exist in this list, Insert it somewhere in the original bytecode, and that’s it.

This code determines whether the instrumentality function is static and then simply new an array of type Object.

The JVM Opcode Reference defines the anewarray directive.

Description: allocate a new array of objects

Stack:

before after
size array ref
other other

Create a reference to the array in the local variable table, and then press the stack. This is exactly the same as the code before the for loop.

The implementation of loadArg may seem strange at first, but let’s take a look at its local variable table using a very simple program fragment

// Java program fragment
public String hello(String str, long num1, double num2);

// corresponds to the bytecode local change table
LocalVariableTable:
Start  Length  Slot  Name   Signature
    0     415     0  this   Lcom/example/Sample;
    0     415     1   str   Ljava/lang/String;
    0     415     2  num1   J
    0     415     4  num2   D
    94     321     6  num3   Ljava/lang/Long;
    313     102     7 result   Ljava/lang/String;
Copy the code
public static void loadArg(boolean staticAccess, final InsnList instructions, Type[] argumentTypes, int i) {
    final int index = getArgIndex(staticAccess, argumentTypes, i);
    final Type type = argumentTypes[i];
    // Load a variable with the offset index and length 'type.size()' in the local variable table onto the stack
    instructions.add(new VarInsnNode(type.getOpcode(Opcodes.ILOAD), index));
}

static int getArgIndex(boolean staticAccess, final Type[] argumentTypes, final int arg) {
    int index = staticAccess ? 0 : 1;
    for (int i = 0; i < arg; i++) {
        // Calculate the offset of the ith variable in the local variable table according to the variable type
        index += argumentTypes[i].getSize();
    }
    return index;
}
Copy the code

JVM Opcode Reference: Aastore instruction:

Description: store value in array[index]

Stack:

before after
value
index
arrayref

Arrayref [index]; arrayRef [index]; arrayRef [index]; So with that in mind, how does the loadArgArray method assign to an array

  1. Stack array length:

Stack at this point:

length
  1. callanewarrayThe stack directive creates a new array reference:

Stack at this point:

arrayref
  1. startThe for loop
  • Copy the top element of the stack:

Stack at this point:

arrayref
arrayref
  • Push loop variable I to the top of the stack:

Stack at this point:

i
arrayref
arrayref
  • Load the ith variable in the local variable table. How to load the ith variableloadArgThe fraction of this function

Analysis:

Stack at this point:

localvar
i
arrayref
arrayref
  • callaastoreInstruction,localvarenduresarrayref[i]And start the next cycle

Stack at this point:

arrayref

Arthas ByteKit is an in-depth explanation of the application of the diagnostic tool Arthas ByteKit (2) : Local variables and parameter binding

How ThisBinding works

With the previous analysis of method variables, it is easier to analyze other Binding annotations

@Documented
@Retention(RetentionPolicy.RUNTIME)
@java.lang.annotation.Target(ElementType.PARAMETER)
@BindingParserHandler(parser = ThisBindingParser.class)
public static @interface This {

}

public static class ThisBindingParser implements BindingParser {
    @Override
    public Binding parse(Annotation annotation) {
        return newThisBinding(); }}public class ThisBinding extends Binding {

    @Override
    public void pushOntoStack(InsnList instructions, BindingContext bindingContext) {
        bindingContext.getMethodProcessor().loadThis(instructions);
    }

    @Override
    public Type getType(BindingContext bindingContext) {
        returnType.getType(Object.class); }}public void loadThis(final InsnList instructions) {
    if (isConstructor()) {
        // load this.
        loadVar(instructions, 0);
    } else {
        if (isStatic()) {
            // load null.
            loadNull(instructions);
        } else {
            // load this.
            loadVar(instructions, 0); }}}Copy the code

The process of binding this is relatively simple. If it is a non-static method, this is the first slot of the local variable, and if it is static, this is null

Class and method information are relatively simple, and a simple LDC command can do it. Take class information for example:

public class ClassBinding extends Binding{

    @Override
    public void pushOntoStack(InsnList instructions, BindingContext bindingContext) {
        String owner = bindingContext.getMethodProcessor().getOwner();
        AsmOpUtils.ldc(instructions, Type.getObjectType(owner));
    }

    @Override
    public Type getType(BindingContext bindingContext) {
        returnType.getType(Class.class); }}Copy the code

So no further details

Summary of the Binding series annotations

The use of annotations to intercept and extract JVM runtime information is one of arthas’s highlights. This part of the code is also a separate framework byteKit. This section analyzes arthas’s principle of binding runtime information. Give the reader an idea of where the runtime information we output to the Arthas console came from.

How the InterceptorProcessor works

In the first section of the analysis, we can see that the interceptor’s process method is called for code that needs instrumentation:

public List<Location> process(MethodProcessor methodProcessor) throws Exception {
    List<Location> locations = locationMatcher.match(methodProcessor);

    List<Binding> interceptorBindings = interceptorMethodConfig.getBindings();

    for (Location location : locations) {
        // There are three small pieces of code, 1: save the value on the current stack, 2: insert callback, 3: restore the current stack
        InsnList toInsert = new InsnList();
        InsnList stackSaveInsnList = new InsnList();
        InsnList stackLoadInsnList = new InsnList();
        StackSaver stackSaver = null;
        if(location.isStackNeedSave()) {
            stackSaver = location.getStackSaver();
        }
        BindingContext bindingContext = new BindingContext(location, methodProcessor, stackSaver);

        if(stackSaver ! =null) {
            stackSaver.store(stackSaveInsnList, bindingContext);
            stackSaver.load(stackLoadInsnList, bindingContext);
        }


        Type methodType = Type.getMethodType(interceptorMethodConfig.getMethodDesc());
        Type[] argumentTypes = methodType.getArgumentTypes();
        // Check that the callback argument matches the binding number
        if(interceptorBindings.size() ! = argumentTypes.length) {throw new IllegalArgumentException("interceptorBindings size no equals with interceptorMethod args size.");
        }

        // Save the data on the current stack
        int fromStackBindingCount = 0;
        for (Binding binding : interceptorBindings) {
            if(binding.fromStack()) { fromStackBindingCount++; }}// Only one binding is allowed to hold data from the stack
        if(fromStackBindingCount > 1) {
            throw new IllegalArgumentException("interceptorBindings have more than one from stack Binding.");
        }
        // Assemble the arguments for the static function to call
        for(int i = 0 ; i < argumentTypes.length; ++i) {
            Binding binding = interceptorBindings.get(i);
            binding.pushOntoStack(toInsert, bindingContext);
            // Check the type of the callback argument to see if you want to box it.
            // Only if the type is different. If both of them are longs, you don't have to judge
            Type bindingType = binding.getType(bindingContext);
            if(! bindingType.equals(argumentTypes[i])) {if(AsmOpUtils.needBox(bindingType)) { AsmOpUtils.box(toInsert, binding.getType(bindingContext)); }}}// TODO checks that binding and the callback function have the same argument type. Callbacks can be of type Object or super. However, some obvious type problems, such as array going to int, are not allowed
        toInsert.add(new MethodInsnNode(Opcodes.INVOKESTATIC, interceptorMethodConfig.getOwner(), interceptorMethodConfig.getMethodName(),
                interceptorMethodConfig.getMethodDesc(), false));
        if(! methodType.getReturnType().equals(Type.VOID_TYPE)) {if (location.canChangeByReturn()) {
                // When the callback function returns a value, it needs to be updated to the previously saved stack
                // TODO should have type problem? You need to check if you want box
                Type returnType = methodType.getReturnType();
                Type stackSaverType = stackSaver.getType(bindingContext);
                if(! returnType.equals(stackSaverType)) { AsmOpUtils.unbox(toInsert, stackSaverType); } stackSaver.store(toInsert, bindingContext); }else {
                // If the return value of the callback function is not used, it needs to be removed from the stack
                int size = methodType.getReturnType().getSize();
                if (size == 1) {
                    AsmOpUtils.pop(toInsert);
                } else if (size == 2) {
                    AsmOpUtils.pop2(toInsert);
                }
            }
        }


        TryCatchBlock errorHandlerTryCatchBlock = null;
        // The generated code is surrounded by a try/catch
        if( exceptionHandlerConfig ! =null) {
            LabelNode gotoDest = new LabelNode();

            errorHandlerTryCatchBlock = new TryCatchBlock(methodProcessor.getMethodNode(), exceptionHandlerConfig.getSuppress());
            toInsert.insertBefore(toInsert.getFirst(), errorHandlerTryCatchBlock.getStartLabelNode());
            toInsert.add(new JumpInsnNode(Opcodes.GOTO, gotoDest));
            toInsert.add(errorHandlerTryCatchBlock.getEndLabelNode());
// How to store data on the stack? Or force the first argument to the callback to be an exception, followed by a binding.
            errorHandler(methodProcessor, toInsert);
            toInsert.add(gotoDest);
        }
        stackSaveInsnList.add(toInsert);
        stackSaveInsnList.add(stackLoadInsnList);
        if (location.isWhenComplete()) {
            methodProcessor.getMethodNode().instructions.insert(location.getInsnNode(), stackSaveInsnList);
        }else {
            methodProcessor.getMethodNode().instructions.insertBefore(location.getInsnNode(), stackSaveInsnList);
        }
        if( exceptionHandlerConfig ! =null) {
            errorHandlerTryCatchBlock.sort();
        }
        // inline callback
        if(interceptorMethodConfig.isInline()) { Class<? > forName = classLoader.loadClass(Type.getObjectType(interceptorMethodConfig.getOwner()).getClassName()); MethodNode toInlineMethodNode = AsmUtils.findMethod(AsmUtils.loadClass(forName).methods, interceptorMethodConfig.getMethodName(), interceptorMethodConfig.getMethodDesc()); methodProcessor.inline(interceptorMethodConfig.getOwner(), toInlineMethodNode); }if(exceptionHandlerConfig ! =null && exceptionHandlerConfig.isInline()) {
            Class<?> forName = classLoader.loadClass(Type.getObjectType(exceptionHandlerConfig.getOwner()).getClassName());
            MethodNode toInlineMethodNode = AsmUtils.findMethod(AsmUtils.loadClass(forName).methods, exceptionHandlerConfig.getMethodName(), exceptionHandlerConfig.getMethodDesc());

            methodProcessor.inline(exceptionHandlerConfig.getOwner(), toInlineMethodNode);
        }
    }
    
    return locations;
}
Copy the code

This section of code is quite long, and if we skip all the stackSaver code (see stackSaver later), the logic is clear:

  1. callBindingThe object’spushOntoStackMethod to complete all predefined bindings,pushOntoStackMethods are described in detail in the previous article
  2. Call the instrumentation method
  3. If an exception handler is registered, insert it into the code as well
  4. See if the annotation specifies whether to inline, and if so, inline both the instrumentation method and the exception handler

Com. Alibaba. Bytekit. Asm. MethodProcessor# inline implementation is longer, interested students may have a look, here we are but more in detail.

This process is relatively simple because it doesn’t consider StackSaver, but StackSaver’s principles are essential to a complete understanding of how the interceptor works.

StackSaverHow does it work

Not many subclasses of Location implement the getStackSaver method, so let’s use ExitLocation as an example:

public StackSaver getStackSaver(a) {
    StackSaver stackSaver = new StackSaver() {

        @Override
        public void store(InsnList instructions, BindingContext bindingContext) {
            Type returnType = bindingContext.getMethodProcessor().getReturnType();
            if(!returnType.equals(Type.VOID_TYPE)) {
                LocalVariableNode returnVariableNode = bindingContext.getMethodProcessor().initReturnVariableNode();
                AsmOpUtils.storeVar(instructions, returnType, returnVariableNode.index);
            }
        }

        @Override
        public void load(InsnList instructions, BindingContext bindingContext) {
            Type returnType = bindingContext.getMethodProcessor().getReturnType();
            if(!returnType.equals(Type.VOID_TYPE)) {
                LocalVariableNode returnVariableNode = bindingContext.getMethodProcessor().initReturnVariableNode();
                AsmOpUtils.loadVar(instructions, returnType, returnVariableNode.index);
            }
        }

        @Override
        public Type getType(BindingContext bindingContext) {
            returnbindingContext.getMethodProcessor().getReturnType(); }};return stackSaver;
}

// Add a variable to the local variable table
public LocalVariableNode initReturnVariableNode(a) {
    if (returnVariableNode == null) {
        returnVariableNode = this.addInterceptorLocalVariable(returnVariableName, returnType.getDescriptor());
    }
    return returnVariableNode;
}
Copy the code

When we call store, we add a variable to the local variable table, and then store the variable at the top of the stack into the new variable, and then load the value of the local variable onto the top of the stack after we call load. Back to the implementation of the interceptor, The stackSaver process in this implementation is highlighted and commented in detail to help you understand the process better.

// ...
If the subclass implements getStackSaver, it does two things
if(stackSaver ! =null) {
    // 1. Save the top element to the local variable table, which actually saves this instruction to the stackSaveInsnList
    stackSaver.store(stackSaveInsnList, bindingContext);
    // 2. Loading variables from the local variable table to the top of the stack is also a stackLoadInsnList. Note that these are two different lists
    stackSaver.load(stackLoadInsnList, bindingContext);
}
// ...
// In the case of ExistLocation, the return value of the original function can be replaced by the instrumenting function
// The JVM instruction is executed as if the instrumentality function had just been called
// The top of the stack is the return value of the insert function
// Return value of the original method for substitution purposes
if (location.canChangeByReturn()) {
    // When the callback function returns a value, it needs to be updated to the previously saved stack
    // TODO should have type problem? You need to check if you want box
    Type returnType = methodType.getReturnType();
    Type stackSaverType = stackSaver.getType(bindingContext);
    if(! returnType.equals(stackSaverType)) { AsmOpUtils.unbox(toInsert, stackSaverType); } stackSaver.store(toInsert, bindingContext); }else {
    // If the return value of the callback function is not used, it needs to be removed from the stack
    int size = methodType.getReturnType().getSize();
    if (size == 1) {
        AsmOpUtils.pop(toInsert);
    } else if (size == 2) { AsmOpUtils.pop2(toInsert); }}// ...
stackSaveInsnList.add(toInsert);
// The last instruction in toInsert is store, which clears the top element of the stack
// However, according to the JVM specification, the element at the top of the stack after a function call is completed must be the return value
// So the load directive is called to load the return value of the function onto the stack
stackSaveInsnList.add(stackLoadInsnList);
Copy the code

Exception handler instrumentation

// The generated code is surrounded by a try/catch
if( exceptionHandlerConfig ! =null) {
    LabelNode gotoDest = new LabelNode();
    errorHandlerTryCatchBlock = new TryCatchBlock(methodProcessor.getMethodNode(), exceptionHandlerConfig.getSuppress());
    toInsert.insertBefore(toInsert.getFirst(), errorHandlerTryCatchBlock.getStartLabelNode());
    toInsert.add(new JumpInsnNode(Opcodes.GOTO, gotoDest));
    toInsert.add(errorHandlerTryCatchBlock.getEndLabelNode());
    errorHandler(methodProcessor, toInsert);
    toInsert.add(gotoDest);
}
Copy the code

Exception handlers are relatively simple to insert. The idea is to wrap the code we just inserted around a try-catch, similar to the following pseudo-instruction:

startLabel
instrument instruction
goto dest
endLabel
errorHandler
dest
Copy the code

conclusion

The design of code instrumentation is arthas’s core logic and one of his core assets, so this section is a bit more complex and complex, so it’s important to switch gears and think about Java code in terms of bytecode. This article tries to use the simplest examples to help you understand the meaning of each interpolation statement. It may not cover all arthas instrumentation scenarios, but you can explore arthas further by combining the examples in this article with the arthas source code.

Next article in this article, on the basis of the bytecode instrumentation about arthas is how to realize the specific command, is expected to include watch, trace, tt, monitor, stack five command, stay tuned.

Welcome to pay attention to the author’s official account: