Understand Java reflection in depth

digression

Recently, the company set up an official account of the technology department to encourage everyone to share. Many students are struggling to find suitable topics to talk about. Generally speaking, there are probably two reasons: one is that I think it is too basic and low, and no one will pay attention to it; the other is that I am afraid of making mistakes when I talk about some great new technologies. However, each technology in its own application will have your own unique perspective, which may be the concern of others. I think it makes sense to share some of the topics that most of us are familiar with in coding, but most of us don’t know why. Today I’m going to share a tool that we use a lot in Java, reflection and dynamic proxy.

When we write code in the IDE, we put a dot, and the IDE automatically pops up the corresponding property and method name. When we debug, the IDE will display the values of local variables in the method at runtime and properties on the external instance. IOC and AOP in Spring, and an RPC framework, we deserialize, consumer proxy, and provider calls all use Java reflection. Some people say that using reflection is slow, so what is slow?

reflection

Reflection gives the Java language the ability to dynamically compile, which means that we don’t need to know the exact type of the object when coding, but at runtime we can get a Class object through class.forname () and an instance through newInstance.

Take a look at the diagram of the main classes in the java.lang.Reflect package, as well as the dynamic proxy utility classes.

• AnnotatedElement

As a top-level interface, this interface provides access to annotations. We can add annotations to methods, classes, properties, and constructors. The following fields, methods, and constructors implement this interface. In jdk8, the interface can be implemented through the default modification method

Annotation[] getAnnotations(); Default <T extends Annotation> T getDeclaredAnnotation(Class<T> annotationClass) { Objects.requireNonNull(annotationClass); // Loop over all directly-present annotations lookingfor a matching one
     for (Annotation annotation : getDeclaredAnnotations()) {
         if (annotationClass.equals(annotation.annotationType())) {
             // More robust to do a dynamic cast at runtime instead
             // of compile-time only.
             returnannotationClass.cast(annotation); }}return null;
 }
Copy the code

• GenericDeclaration

Generics are supported only on methods and constructors, so only Method, Constructor implements this interface

• Member

An abstraction of declarations of methods and properties inside an object, containing names, modifiers, and classes, including modifiers such as static final public private volatile, and so on, represented by an integer, with each type taking one bit in binary.

public Class<? > getDeclaringClass(); public String getName(); public int getModifiers(); Public static final int public = 0x00000001; public static final int PRIVATE = 0x00000002; public static final int PROTECTED = 0x00000004; public static final int STATIC = 0x00000008; public static final int FINAL = 0x00000010; public static final int SYNCHRONIZED = 0x00000020; public static final int VOLATILE = 0x00000040; public static final int TRANSIENT = 0x00000080; public static final int NATIVE = 0x00000100; public static final int INTERFACE = 0x00000200; public static final int ABSTRACT = 0x00000400; public static final int STRICT = 0x00000800; public static boolean isPublic(int mod) {return(mod & PUBLIC) ! = 0; }Copy the code

• AccessibleObject

  This is a class that provides permission management functions, such as whether a private method is allowed to be called externally in reflection to get the value of a private property. Method, constructor, and field all inherit from this class. Class object constructors are not allowed to be used externally.

private static void setAccessible0(AccessibleObject obj, boolean flag)
    throws SecurityException
{
    if (obj instanceof Constructor && flag == true) { Constructor<? > c = (Constructor<? >)obj;if (c.getDeclaringClass() == Class.class) {
            throw new SecurityException("Cannot make a java.lang.Class" +
                                        " constructor accessible");
        }
    }
    obj.override = flag;
}

boolean override;

public boolean isAccessible() {
    return override;
}
Copy the code

Get (the original object). Override is used to verify the access. If the access permission is not overridden, the access permission is verified

public Object get(Object obj)
    throws IllegalArgumentException, IllegalAccessException
{
    if(! override) {if(! Reflection.quickCheckMemberAccess(clazz, modifiers)) { Class<? >caller = Reflection.getCallerClass();
            checkAccess(caller, clazz, obj, modifiers); }}return getFieldAccessor(obj).get(obj);
}
Copy the code

Now let’s see how we can change the value of the property in the Field by reflection

From the above code, we can see that the JDK delegates reading and writing Field attributes to FieldAccessor. How do we get FieldAccessor

class UnsafeFieldAccessorFactory {
    UnsafeFieldAccessorFactory() { } static FieldAccessor newFieldAccessor(Field var0, boolean var1) { Class var2 = var0.getType(); boolean var3 = Modifier.isStatic(var0.getModifiers()); boolean var4 = Modifier.isFinal(var0.getModifiers()); boolean var5 = Modifier.isVolatile(var0.getModifiers()); boolean var6 = var4 || var5; boolean var7 = var4 && (var3 || ! var1);if (var3) {
            UnsafeFieldAccessorImpl.unsafe.ensureClassInitialized(var0.getDeclaringClass());

            return(FieldAccessor) ((! var6) ? ((var2 == Boolean.TYPE) ? new UnsafeStaticBooleanFieldAccessorImpl(var0) : ((var2 == Byte.TYPE) ? new UnsafeStaticByteFieldAccessorImpl(var0) : ((var2 == Short.TYPE) ? new UnsafeStaticShortFieldAccessorImpl(var0) : ((var2 == Character.TYPE) ? new UnsafeStaticCharacterFieldAccessorImpl(var0) : ((var2 == Integer.TYPE) ? new UnsafeStaticIntegerFieldAccessorImpl(var0) : ((var2 == Long.TYPE) ? new UnsafeStaticLongFieldAccessorImpl(var0) : ((var2 == Float.TYPE) ? new UnsafeStaticFloatFieldAccessorImpl(var0) : ((var2 == Double.TYPE) ? new UnsafeStaticDoubleFieldAccessorImpl(var0) : new UnsafeStaticObjectFieldAccessorImpl(var0))))))))) : ((var2 == Boolean.TYPE) ? new UnsafeQualifiedStaticBooleanFieldAccessorImpl(var0, var7) : ((var2 == Byte.TYPE) ? new UnsafeQualifiedStaticByteFieldAccessorImpl(var0, var7) : ((var2 == Short.TYPE) ? new UnsafeQualifiedStaticShortFieldAccessorImpl(var0, var7) : ((var2 == Character.TYPE) ? new UnsafeQualifiedStaticCharacterFieldAccessorImpl(var0, var7) : ((var2 == Integer.TYPE) ? new UnsafeQualifiedStaticIntegerFieldAccessorImpl(var0, var7) : ((var2 == Long.TYPE) ? new UnsafeQualifiedStaticLongFieldAccessorImpl(var0, var7) : ((var2 == Float.TYPE) ? new UnsafeQualifiedStaticFloatFieldAccessorImpl(var0, var7) : ((var2 == Double.TYPE) ? new UnsafeQualifiedStaticDoubleFieldAccessorImpl(var0, var7) : new UnsafeQualifiedStaticObjectFieldAccessorImpl(var0, var7)))))))))); }else {
            return(FieldAccessor) ((! var6) ? ((var2 == Boolean.TYPE) ? new UnsafeBooleanFieldAccessorImpl(var0) : ((var2 == Byte.TYPE) ? new UnsafeByteFieldAccessorImpl(var0) : ((var2 == Short.TYPE) ? new UnsafeShortFieldAccessorImpl(var0) : ((var2 == Character.TYPE) ? new UnsafeCharacterFieldAccessorImpl(var0) : ((var2 == Integer.TYPE) ? new UnsafeIntegerFieldAccessorImpl(var0) : ((var2 == Long.TYPE) ? new UnsafeLongFieldAccessorImpl(var0) : ((var2 == Float.TYPE) ? new UnsafeFloatFieldAccessorImpl(var0) : ((var2 == Double.TYPE) ? new UnsafeDoubleFieldAccessorImpl(var0) : new UnsafeObjectFieldAccessorImpl(var0))))))))) : ((var2 == Boolean.TYPE) ? new UnsafeQualifiedBooleanFieldAccessorImpl(var0, var7) : ((var2 == Byte.TYPE) ? new UnsafeQualifiedByteFieldAccessorImpl(var0, var7) : ((var2 == Short.TYPE) ? new UnsafeQualifiedShortFieldAccessorImpl(var0, var7) : ((var2 == Character.TYPE) ? new UnsafeQualifiedCharacterFieldAccessorImpl(var0, var7) : ((var2 == Integer.TYPE) ? new UnsafeQualifiedIntegerFieldAccessorImpl(var0, var7) : ((var2 == Long.TYPE) ? new UnsafeQualifiedLongFieldAccessorImpl(var0, var7) : ((var2 == Float.TYPE) ? new UnsafeQualifiedFloatFieldAccessorImpl(var0, var7) : ((var2 == Double.TYPE) ? new UnsafeQualifiedDoubleFieldAccessorImpl(var0, var7) : new UnsafeQualifiedObjectFieldAccessorImpl(var0, var7)))))))))); }}}Copy the code

As you can see from the above code, the factory mode is used to fetch fields based on their type and whether they are static or not. Why is there such a division?

First, the JDK is UNSAFE in the class attribute of an object in the heap memory directly read and write, to read and write the first thing you need to determine the position of the property is located, which is relative to the object offset of the starting position and static attributes for each object instance is not a class, so the static properties of offset needs to be alone.

In fact, from this offset we can roughly infer the relative location of each attribute in the heap memory of an instance and how much space each attribute occupies. With this location information, we also need the type of this field so that the executor knows how many bytes of data to read and how to parse it. Eight basic types (char vs Charector) and array and plain reference types are currently provided.

To ensure that each object occupies a multiple of eight bytes, the Java virtual machine sometimes fills certain fields to avoid having two volatile fields on the same cache line.

The following is part of the UnSafe class

public final class Unsafe {
    private static final Unsafe theUnsafe;
    private Unsafe() {
    }

    @CallerSensitive
    public static Unsafe getUnsafe() {
        Class var0 = Reflection.getCallerClass();
        if(! VM.isSystemDomainLoader(var0.getClassLoader())) { throw new SecurityException("Unsafe");
        } else {
            return theUnsafe;
        }
    }

    public native int getInt(Object var1, long var2);
    public native void putInt(Object var1, long var2, int var4);
    public native Object getObject(Object var1, long var2);
    public native void putObject(Object var1, long var2, Object var4);
    public native boolean getBoolean(Object var1, long var2);
    public native void putBoolean(Object var1, long var2, boolean var4);
    public native byte getByte(Object var1, long var2);

    public native long objectFieldOffset(Field var1);
@Deprecated
public int fieldOffset(Field var1) {
    returnModifier.isStatic(var1.getModifiers())? (int)this.staticFieldOffset(var1):(int)this.objectFieldOffset(var1); }Copy the code

Then we’ll look at calling methods through reflection

Also, the JDK uses MethodAccessor to make method calls, The Java VIRTUAL machine provides two modes to support method invocation: NativeMethodAccessorImpl and ASM bytecode directly generating a class that calls the target method inside the Invoke method. The source code is not available in the JDK because it is dynamically generated. But the JDK offer DelegatingMethodAccessorImpl delegate pattern to facilitate in the process of running can dynamically switch the bytecode mode and native mode, we can see MethodAccessor generated code.

class NativeMethodAccessorImpl extends MethodAccessorImpl {
    private final Method method;
    private DelegatingMethodAccessorImpl parent;
    private int numInvocations;

    NativeMethodAccessorImpl(Method var1) {
        this.method = var1;
    }

    public Object invoke(Object var1, Object[] var2) throws IllegalArgumentException, InvocationTargetException {
        if(++this.numInvocations > ReflectionFactory.inflationThreshold() && ! ReflectUtil.isVMAnonymousClass(this.method.getDeclaringClass())) { MethodAccessorImpl var3 = (MethodAccessorImpl)(new MethodAccessorGenerator()).generateMethod(this.method.getDeclaringClass(), this.method.getName(), this.method.getParameterTypes(), this.method.getReturnType(), this.method.getExceptionTypes(), this.method.getModifiers()); this.parent.setDelegate(var3); }return invoke0(this.method, var1, var2);
    }

    void setParent(DelegatingMethodAccessorImpl var1) {
        this.parent = var1;
    }

    private static native Object invoke0(Method var0, Object var1, Object[] var2);
}
Copy the code

Can see within the JDK by numInvocations judgment. If the number of reflection calls over ReflectionFactory inflationThreshold () is implemented using bytecode. Native invocation is much slower than bytecode. Dynamic implementation is 20 times faster than local implementation, because dynamic implementation does not need to go through JAVA,C++ and JAVA conversion. The JDK provides a default threshold of 15, since many ReflectAsm implementations need to be executed only once, and dynamic generation of bytecode takes a long time. When a reflection is called less than 15 times, it is implemented locally, and when it is called more than 15 times, it is implemented dynamically, and this threshold can be configured in JDK startup parameters.

Why is reflection slow

After the above optimization, the efficiency of reflection is not slow. In some cases, reflection can achieve almost the same efficiency as direct invocation, but there is still a performance overhead in the case of first execution or no cache, mainly in the following aspects

1. Class.forName(); Local methods will be called, and all methods and fields used will be loaded at this point. Although cached, local methods will have to be converted from JAVA to C+= in JAVA.
2. Class.getmethod (), iterates through all the common methods of the class, and all the methods of the parent class if there is no match, and getMethods() returns a copy of the result. Therefore, this operation consumes both CPU and heap memory and should be avoided or cached in hot code.
3. The Invoke argument is an Object array, and Object arrays do not support Java base types, and automatic boxing is time-consuming.

Use of reflection

• spring doc

Spring’s process for loading beans basically uses reflection

1. Get an instance of the class through the constructor getInstance (static variable initialization, attribute assignment, constructor);
2. If the BeanNameAware interface is implemented, a reflection injection bean is used to assign values to properties;
3. If the BeanFactoryAware interface is implemented, set beanFactory.
4. If ApplicationContextAware is implemented, set ApplicationContext;
5. Call BeanPostProcesser’s pre-initialization method;
6. If InitializingBean is implemented, call the AfterPropertySet method;
7. Call @postconstruct directly to the custom init-method() method;
8. Call the post-initialization method of BeanPostProcesser.

• serialization

For fastjson, see ObjectDeserializer for JavaBeanDeserializer and ASMJavaBeanDeserializer.

A dynamic proxy

The JDK provides a utility class to dynamically generate a proxy, allowing additional processing when a method is executed.

Proxy.newProxyInstance(ClassLoader loader,Class<? > [] interfaces, InvocationHandler h) class HWInvocationHandler implements InvocationHandler {/ / the target Object private Object target; public HWInvocationHandler(Object target){ this.target = target; } public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { System.out.println("------ Insert pre-notification code -------------"); Object rs = method.invoke(target,args); System.out.println("------ Insert post-processing code -------------");
            returnrs; }}Copy the code

Let’s analyze the implementation of this method. The first generated proxy object needs to implement all the interfaces declared in the parameter. The implementation of the interface should be delegated to the InvocationHandler, which can determine whether enhancements need to be made based on the Method declaration. Therefore, the generated proxy class must be able to get the InvocationHandler. If we do not know the specific type of the proxy class, we can pass the InvocationHandler from the constructor to the instance of the proxy class through reflection. So in general, there are two steps to generating proxy objects

1. Gets the class object of the proxy class
2. The constructor is obtained through the class object, an instance of the proxy class is generated through reflection, and the InvocationHandler is passed to the person

@CallerSensitive public static Object newProxyInstance(ClassLoader loader, Class<? >[] interfaces, InvocationHandler h) throws IllegalArgumentException { Objects.requireNonNull(h); final Class<? >[] intfs = interfaces.clone(); /* * Look up or generate the designated proxy class. > cl = getProxyClass0(loader, intfs); // Invoke the designated Invocation handler. // Final constructor <? > cons = cl.getConstructor(constructorParams); final InvocationHandler ih = h;if(! Modifier.isPublic(cl.getModifiers())) { AccessController.doPrivileged(new PrivilegedAction<Void>() { public Voidrun() {
                    cons.setAccessible(true);
                    returnnull; }}); } // Get an instance of the proxy class and pass invocationHandler to the personreturncons.newInstance(new Object[]{h}); } catch (IllegalAccessException|InstantiationException e) { ... }}Copy the code

Let’s see how getProxyClass0 gets the class object of the proxy class. Here, IDK uses WeakCache to cache the generated class object, because it is still time-consuming to generate the class through bytecode generation. At the same time, in order to solve the memory shortage caused by too many class objects generated by dynamic proxy, WeakReference is used to cache the generated proxy object class. When GC occurs, if the class object has no other strong reference, it will be directly recycled. The class that generates the proxy class is implemented in the generateProxyClass method of ProxyGenerator, which returns an array of byte[] and is loaded into the virtual machine via a local method, so generating the object is still quite time-consuming.

/ / generated bytecode byte array [] proxyClassFile = ProxyGenerator. GenerateProxyClass (proxyName, interfaces, accessFlags); Try {// load into the virtual machinereturn defineClass0(loader, proxyName,
                        proxyClassFile, 0, proxyClassFile.length);
} catch (ClassFormatError e) {
    /*
     * A ClassFormatError here means that (barring bugs in the
     * proxy class generation code) there was some other
     * invalid aspect of the arguments supplied to the proxy
     * class creation (such as virtual machine limitations
     * exceeded).
     */
    throw new IllegalArgumentException(e.toString());
}

private byte[] generateClassFile() {
    this.addProxyMethod(hashCodeMethod, Object.class);
    this.addProxyMethod(equalsMethod, Object.class);
    this.addProxyMethod(toStringMethod, Object.class);
    Class[] var1 = this.interfaces;
    int var2 = var1.length;

    int var3;
    Class var4;
    for(var3 = 0; var3 < var2; ++var3) {
        var4 = var1[var3];
        Method[] var5 = var4.getMethods();
        int var6 = var5.length;

        for(int var7 = 0; var7 < var6; ++var7) { Method var8 = var5[var7]; this.addProxyMethod(var8, var4); } } this.methods.add(this.generateConstructor()); . } / / generate a private constructor with invocationhandler parameters ProxyGenerator. The MethodInfo generateConstructor () throws IOException { ProxyGenerator.MethodInfo var1 = new ProxyGenerator.MethodInfo("<init>"."(Ljava/lang/reflect/InvocationHandler;) V", 1);
    DataOutputStream var2 = new DataOutputStream(var1.code);
    this.code_aload(0, var2);
    this.code_aload(1, var2);
    var2.writeByte(183);
    var2.writeShort(this.cp.getMethodRef("java/lang/reflect/Proxy"."<init>"."(Ljava/lang/reflect/InvocationHandler;) V"));
    var2.writeByte(177);
    var1.maxStack = 10;
    var1.maxLocals = 2;
    var1.declaredExceptions = new short[0];
    return var1;
}
Copy the code

The above process can be summarized as follows

1. Add hashCode,equals,toString methods;
2. Add unimplemented methods declared in all interfaces.
3. Add a method parameter as; The structure of the Java/lang/reflect/InvocationHandler method
4. Other statically initialized data.

Application of dynamic proxy

1. spring-aop

   Spring AOP is implemented based on JDK dynamic proxies by default

   In A class, there are two methods A and B, and method B has an annotation to enhance the object. Why does A call this.B have no object effect?

   Because Spring-AOP defaults to a dynamic proxy implementation based on the JDK, The final execution is through the generated proxy object, and the proxy object executes A method and B method is actually called in the InvocationHandler enhanced methods, where B method is enhanced by the InvocationHandler before and after the method to add things open and commit code, and the actual execution of the code is through Methodb. invoke(primitive object) And the implementation of method A contains this Methoda. invoke(methodA) invoke(methodA); this.b () invoke(methodA); this.b () invoke(methodA); this.b () invoke(methodA) Subclasses, which call this.B methods with transactional effects.

2. rpc consumer

   Those who have experience in RMI development may be familiar with why two stub files are generated in client and server respectively when RMI service is exported. The client file actually generates a proxy class using dynamic proxy This proxy class, implements to foreign service all interfaces, the realization of each method is actually the interface information, method statement, parameters and return values of information through the network to the server, the server after receipt of a request by finding corresponding implementation and use reflection method. Invoke the calling, and then return the results to the client

   In fact, the implementation of other RPC frameworks is roughly similar to this, except that the proxy class of the client may not only transmit the method declaration to the service provider over the network, but also perform some service routing, load balancing, and transmit some additional attachment data to the provider

Author’s brief introduction

Xiaoqiang, development engineer of capital terminal and background of Tongban Street, joined Tongban Street in June 2015. At present, I am responsible for the development of tongban Street capital end clearing settlement.