Dubbo series -Dubbo SPI mechanism

The following articles are for referenceBook.douban.com/subject/344…

1. Overview of loading mechanism

1.1 Java SPI

SPI stands for Service Provider Interface, which provides a Service Interface. But it is not the same concept as service discovery in distribution. The Java SPI uses the policy pattern, where an interface can have multiple implementations, but the implementation is not directly determined in the program. Instead, it is controlled by external configuration, using the following steps:

1. Define an interface and corresponding methods
2. Write an implementation class for this interface
3. Create a file named after the interface’s full path in the meta-INF /services directory
4. The content of the file is the full path name of the concrete implementation class
5. Use java.util.Serviceloader in your code to load the concrete implementation classes

1.2 Improvement of extension point loading mechanism

The Dubbo SPI has made some improvements and optimizations. The STANDARD SPI of the JDK instantiates all implementations of the extension point at once, which is slow to initialize if there is an extension implementation, and wasteful of resources if not used. Dubbo SPI simply loads the classes in the configuration file and caches them in memory, not all at once. The STANDARD SPI in the JDK swallows error messages when initializing errors, making error checking expensive. Dubbo SPI throws a real exception and prints a log when an extension fails to load. When an extension point fails to load passively, it does not affect the use of other extension points or the entire framework. Dubbo SPI implements IoC and AOP mechanisms of its own, and one extension point can inject other extension methods directly through setter methods. In addition, Dubbo supports wrapper extension classes, and it is recommended to put generic abstract logic in the wrapper class. For implementing the AOP features of extension points.

1.3 Configuration specifications of extension points

Dubbo SPI is similar to Java SPI. You need to place the SPI configuration file under meta-INF/Dubbo /. The file name is the full path name of the interface name. The content of the configuration file is key= the full path name of the extension point implementation class. If there are multiple implementations, use a newline character to split them. Where key is the parameter passed in the Dubbo SPI annotation. When Dubbo starts, meta-INF /services/, meat-INF/Dubbo /, meta-INF/Dubbo /internal/ will be scanned by default

1.4 Classification and caching of extension points

The Dubbo API can be divided into Class cache and instance cache. These two types of caches can be divided into ordinary extension classes, Wrapper classes and Adaptive classes according to the types of extension classes. Class cache: When Dubbo SPI retrieves extended classes, it first reads them from the cache. If the Class is not present in the cache, the configuration file is loaded, and the Class is cached in memory based on the configuration. Instantiation cache: After the Dubbo SPI instantiates a Class, it caches the instantiated object. Each time a Class is fetched, it is read from the cache first. If it cannot be read from the cache, it is reloaded and cached. Common extension Classes: The most basic extension class implementation configured in the SPI configuration file. There is no concrete implementation of the Wrapper extension class: Wrapper, which abstracts the general logic and requires a concrete implementation of the extension interface to be passed in the constructor. Dubbo’s automatic packaging feature. Adaptive extension classes: An interface may have multiple implementation classes, and which implementation class to use can be determined dynamically by passing in some parameters in the URL when running code. Adaptive properties that belong to extension points. Other caches: such as extension loader cache, extension name cache, etc.

1.5 Features of extension points

The extension class contains four features: auto-wrap, auto-load, adaptive, and auto-activate.

1.5.1 Automatic Packaging

When the ExtensionLoader loads an extension class, if it finds that the extension class contains other extension points as arguments to the constructor, it considers the extension class to be a Wrapper class.

public class ProtocolFilterWrapper implements Protocol {
    private final Protocol protocol;

    public ProtocolFilterWrapper(Protocol protocol) {
        if (protocol == null) {
            throw new IllegalArgumentException("protocol == null");
        }
        this.protocol = protocol; }}Copy the code

The ProtocolFilterWrapper class inherits the Protocol interface, but the ProtocolFilterWrapper constructor injects a Protocol parameter. The ProtocolFilterWrapper class is treated as a Wrapper class.

1.5.2 Automatic Loading

If an extension class is a member property of another extension class and has setter methods, the framework will automatically inject the corresponding extension point instance. When ExtensionLoader performs extension point initialization, it automatically injects the corresponding implementation class through setter methods. If there are multiple implementation classes, then adaptability is used.

1.5.3 Adaptability

In Dubbo SPI, using the @Adaptive annotation, you can dynamically determine which specific implementation class to use using parameters in the URL.

@SPI("netty")
public interface Transporter {
    
    @Adaptive({Constants.SERVER_KEY, Constants.TRANSPORTER_KEY})
    RemotingServer bind(URL url, ChannelHandler handler) throws RemotingException;

    @Adaptive({Constants.CLIENT_KEY, Constants.TRANSPORTER_KEY})
    Client connect(URL url, ChannelHandler handler) throws RemotingException;
}
Copy the code

@Adaptive can pass in multiple parameters. When calling the corresponding method, it will dynamically extract the value of the key parameter from the “URL”. If it can match an extended implementation class, the corresponding implementation class will be used directly; if not, the search will continue through the value of the second key parameter. If none is matched, an exception is thrown. This dynamic method of finding implementation classes is flexible, but only one implementation class can be activated. If multiple implementation classes need to be activated at the same time, or according to different conditions, multiple implementation classes need to be activated at the same time, then automatic activation needs to be used.

1.5.4 Automatic Activation

Using the @activate annotation, you can mark the extension point as enabled by default. The annotation can also pass in different parameters to set the extension point to be activated automatically under different conditions.

2. Extension point annotations

2.1 @ SPI

Start by defining a basic interface com.impassive.SpiService

/ * *@author impassivey */
public interface SpiService {

  / * *@return spi */
  String spi(a);
}
Copy the code

Then implement two derived classes: com. Impassive. Service. SpiServiceImpl and com. The impassive. Service. JavaSpiServiceImpl.

/** @author impassivey */ public class SpiServiceImpl implements SpiService { public SpiServiceImpl() { System.out.println("Spi Service"); } @Override public String spi() { System.out.println(SpiServiceImpl.class); return "spi"; } } import com.impassive.SpiService; /** @author impassivey */ public class JavaSpiServiceImpl implements SpiService { public JavaSpiServiceImpl() { System.out.println("Java Spi Service"); } @Override public String spi() { System.out.println(JavaSpiServiceImpl.class); return "Java Spi"; }}Copy the code

Create a configuration file in the meta-INF /services/ path with the same name as the full path name of the interface com.impassive.SpiService and the content is the full path name of the implementation class.

com.impassive.service.SpiServiceImpl
com.impassive.service.JavaSpiServiceImpl
Copy the code

Write a test method (SPI implementation for Java) :

import java.util.ServiceLoader;
import org.junit.Test;

public class SpiTest {

  @Test
  public void testJavaSpi(a) {
    ServiceLoader<SpiService> spiServices = ServiceLoader.load(SpiService.class);
    for(SpiService spiService : spiServices) { spiService.spi(); }}}Copy the code

The final output result is as follows:

Spi Service
class com.impassive.service.SpiServiceImpl
Java Spi Service
class com.impassive.service.JavaSpiServiceImpl

Process finished with exit code 0
Copy the code

As you can see, Java’s SPI instantiates all implementation classes. If one of the implementation classes is not needed at this point, space will be wasted and initialization will be slow. Dubbo SPI implementation, need to add @SPI annotation on the interface, table name interface is an SPI interface, modified interface as follows:

import org.apache.dubbo.common.extension.SPI;

/ * *@author impassivey */
@SPI
public interface SpiService {

  / * *@return spi */
  String spi(a);
}
Copy the code

Note that to use this interface, you need to import the Dubbo-Commen package under Apache. Then add the configuration information in the file named with the interface’s full path name under meta-INF /dubbo. It belongs to a key-value form, where the left-hand side of the equals sign is key and the right-hand side is value.

dubbo-spi=com.impassive.service.SpiServiceImpl
java-spi=com.impassive.service.JavaSpiServiceImpl
Copy the code

Writing test methods:

@Test
public void testDubboSpi(a) {
  ExtensionLoader<SpiService> spiService = ExtensionLoader.getExtensionLoader(SpiService.class);
  SpiService extension = spiService.getExtension("dubbo-spi");
  extension.spi();
}
Copy the code

The following output is displayed:

Spi Service
class com.impassive.service.SpiServiceImpl

Process finished with exit code 0
Copy the code

Based on the return value, JavaSpiServiceImpl is not loaded. So Dubbo implements on-demand loading of extension points, which reduces wasted space and performance. The @spi annotation supports arguments for setting a default implementation subclass. If a default subclass of the setting is not found, an exception is thrown. If it is not set, the default implementation is NULL.

2.2 @ the Adaptive

The @Adaptive annotation can be annotated on classes, interfaces, enumerated classes, and methods. If the annotation is on a method, it is a method-level annotation, and the implementation class can be obtained dynamically through parameters. The method level annotation automatically generates and compels a dynamic Adaptive class on the first getExtension, thus achieving the effect of dynamically implementing the class. At org.apache.dubbo.com mon. The extension. ExtensionLoader# createAdaptiveExtensionClass method generates a Adaptive class, Concrete is implemented by calling the URL org.apache.dubbo.com. Mon URL# getParameter (Java. Lang. String, Java, lang, String) method to obtain the corresponding an extension point. If fetching fails, use a default implementation. For example org. Apache. Dubbo. Remoting. Transporter# connect using the default implementation is netty. If the annotation is on the implementation class, the implementation class is the default extension point. If an interface has more than one implementation class, and more than one implementation class annotates @Adaptive, an exception will be thrown. The annotation can also be passed in an array, which is compared by order of the array, and if the first key is not found, the second key is matched. If none of the keys is matched, the hump rule is used. If none is matched, an exception is thrown. Hump rule: If the value of @adaptive is empty, the humped interface name is converted to a name separated by “.” based on the interface name. Such as the full path name com. Impassiev. ImpassiveService will convert impassive. Service, see concrete implementation methods: Org.apache.dubbo.com mon. The extension. AdaptiveClassCodeGenerator# getMethodAdaptiveValue. ExtensionLoader caches two objects: cachedAdaptiveClass, which caches the Class type of the Adaptive specific implementation Class; CachedAdaptiveInstance, a specific instance of the cache Class, See the specific code org.apache.dubbo.common.extension.ExtensionLoader#getAdaptiveExtensionorg.apache.dubbo.com mon. The extension. ExtensionLo Ader# getAdaptiveExtension.

2.3 @ Activate

@Activate can be annotated on classes, interfaces, enumerated classes, and methods, mainly in scenarios where multiple extension points are implemented and need to be activated depending on different conditions. See the following sections for specific usage methods.

3. Working principle of ExtensionLoader

ExtensionLoader is the main logical class for the entire extension mechanism. This class implements configuration loading, extended class caching, adaptive object generation, and more.

3.1 Workflow

The logical entry of ExtensionLoader can be divided into getExtension, getAdaptiveExtension and getActivateExtension, which are to get the ordinary extension class, get the adaptive extension class, and get the automatically activated extension class respectively. The overall logic starts with calling these three methods, each of which may have a different overloaded method, adjusting for the different incoming methods.

3.2 Implementation principles of getExtension

If the name of the extension point is true, the default extension point will be obtained. If the name of the extension point is true, the default extension point will be obtained. 4. The extension point is loaded from the extension point Class cache. If the extension point cannot be loaded, the extension point configuration file is loaded. META-INF/dubbo/internal/ META-INF/services/ (compatible with JavaSPI) Java’s SPI mechanism is used to load these three directories. 6. After loading the configuration file, a class loader is obtained to load the extended classes in the configuration file. Class loader access mechanism is: first, to get the current thread context class loading, if failed, you get ExtensionLoader class loading, if failed, you get the application class loader 7, reads the configuration file of each line, split, according to the position of the equal sign on the left to the name of the extension point, the right to extend an instance of the class. Extension point names can be multiple, separated by commas (,), and then cached. However, only one class can be cached. When a class is cached, it is never cached again. 8, if there is a repeated the name of the extension points, 9 will throw an exception, and finally the class information cached 10, use the cached information of class, instantiate an object, and cache the instantiated object 11, extension points internal attributes, if any, using the set method into 12, if extension point is a wrapper class object, 13. If the class has an inherited Lifecycle interface, call the init method of that interface to initialize it

3.3 Implementation principle of getAdaptiveExtension

1. It will fetch the resource from the adaptive class cache first. If it does not get the resource, it will load the resource. If there is no @Adaptive annotation, an exception will be thrown. 2. Instantiate an object using the loaded class. 3

3.4 Implementation principle of getActivateExtension

According to the case:

ExtensionLoader<SpiService> spiService = ExtensionLoader.getExtensionLoader(SpiService.class);
URL url = new URL("test"."name".1123);
URL we = url.addParameter("we"."java1");
List<SpiService> extension = spiService.getActivateExtension(we, "we");
System.out.println(extension);
Copy the code

/ * *@author impassivey */
@Activate(value = "we")
public class JavaSpiServiceImpl implements SpiService {

  public JavaSpiServiceImpl(a) {
    System.out.println("Java Spi Service");
  }

  @Override
  public String spi(a) {
    System.out.println(JavaSpiServiceImpl.class);
    return "Java Spi";
  }

  @Override
  public void setName(String name) {}@Override
  public void test(URL url) {}}Copy the code

In the above example, JavaSpiServiimpl is deactivated according to key->we, because its configuration has a value of key. It can also be written as key:value, but when the value is key:value, The key:value of the parameters configured in the URL must be in the same form. Otherwise, the activation is invalid. Since the above method is a Activate method, the configuration file must also be configured with an extension point named java1, otherwise an exception will be thrown indicating that the extension point named java1 cannot be found. The activation process is as follows: 1. Obtain the corresponding value based on the key value. If value has a value of “-default”, these extension points are not loaded. If the name of the extension point is default, all extension points will be loaded. If the name of the extension point starts with “-“, it will not be activated

3.5 Implementation principle of ExtensionFactory

ExtensionFactory factory class diagram:

It can be seen from the class diagram that there are three main implementations of ExtensionFactory: SpringExtensionFactory, SpiExtensionFactory and AdaptiveExtensionFactory. SpringExtensionFactory stores the Spring context to connect Dubbo to the Spring container during service references and service exposures. In Spring, all the contexts are iterated and then looked up by name and bean type. If the interface is annotated with SPI annotations, SpringExtensionFactory will not do the lookup and will use SpiExtensionFactory to do the lookup. AdaptiveExtensionFactory is the default implementation class of ExtensionFactory. At initialization, all extensionFactories are found and instantiated, and then when an instance is obtained. If it is an instance in Spring, the lookup instantiation is done using SpringExtensionFactory, or SpiExtensionFactory if the interface is annotated with SPI annotations. ExtensionFactory is activated when the ExtensionLoader is initialized, and an ExtensionFactory is automatically acquired. The following shows the flow of the entire SPI mechanism

4, extension point dynamic compilation implementation

All that is generated in the adaptive Class is a string that needs to be compiled to become a Class.

4.1 Overall Structure

There are three types of code compilers in Duubo, namely the JDK compiler, Javassist compiler, and Adaptive compilers.

The SPI annotation on Compiler has a default value, Javassist, that is, JavassistCompier, which acts as the default Compiler for compiling the code. The implementation principle of AdaptiveCompiler is the same as that of AdaptiveExtensionFactory. It is used as the choice between two compilers. If the user has specified a corresponding compiler, the specified compiler is used; otherwise, the default compiler is used. AbstractCompiler implements the compiler method by default. The implementation logic mainly consists of obtaining the package name and class name, stitching together the full path, and then using the path to determine whether the class has been loaded. If so, return; otherwise, call the doCompier method to load. The doCompiler method is implemented by subclasses.

4.2 Javassist Dynamic code compilation

Initialize Javassist ClassBuilder 2. Set ClassName 3. Retrieve import, extends, and implements information using re, call Javassist API, and set parameters 4. 5. Specify ClassLoader to compile and generate Class information

4.3 JDK dynamic code compilation

It is mainly divided into three steps: 1. Initialize a JavaFileObject; The string code is wrapped as a file object. 2. Set the JavaFileManage object value. 3. Compile using JavaCompiler

5 Key class entry notes detailed explanation

ExtensionLoader