1. What is SPI

SPI is short for Service Provider Interface. Service Provider interface in Chinese. The SPI should be compatible with the API standards of that service provided by a provider that meets that service standard. Applications should generally be written based on the API, unless the SPI contains functionality not provided in the API and must be used.

2. SPI mechanism in Java

2.1 define

A new feature introduced in JDK6, ServiceLoader, is designed to load a series of service providers. In addition, the ServiceLoader can load the specified service Provider through the service Provider configuration file. When the service provider provides an implementation of the service interface, we only need to create a file named after the service interface in the META-INF/services/ directory of the JAR package. This file contains the concrete implementation class that implements the service interface. When an external application assembs the module, it can find the implementation class name in the meta-INF /services/ jar file and load the instantiation to complete the module injection.

2.2 the Demo sample

Interfaces and implementation classes

package spi;

public interface Animal {

    void hello();

}
Copy the code
package spi; public class Dog implements Animal { @Override public void hello() { System.out.println("This is Dog"); }}Copy the code
package spi; public class Cat implements Animal { @Override public void hello() { System.out.println("This is Cat"); }}Copy the code

The test class

package spi; import java.util.Iterator; import java.util.ServiceLoader; public class TestSPI { public static void main(String[] args) { ServiceLoader<Animal> serviceLoader = ServiceLoader.load(Animal.class); Iterator<Animal> iterator = serviceLoader.iterator(); while (iterator.hasNext()){ Animal animal = iterator.next(); animal.hello(); }}}Copy the code

Create a file spi.Animal in the resources directory meta-INF. services directory

spi.Cat
spi.Dog
Copy the code

Start the test class, and finally output

This is Cat
This is Dog
Copy the code

2.3 Source Code Analysis

Overall process:

  1. Find the file spi.Animal from meta-inf. services according to the interface name Animal
  2. Parse the file to get the fully qualified names of all the implementation classes
  3. Looping loads the implementation class
  4. Creates an instance of the implementation class by reflection based on the name
  5. Into the cache

2.3.1 ServiceLoader. The load () method

Public static <S> ServiceLoader<S> load(Class<S> service) {// Get the current thread Class loader cl = Thread.currentThread().getContextClassLoader(); return ServiceLoader.load(service, cl); }Copy the code

2.3.2 ServiceLoader constructor

The constructor is finally called

Private ServiceLoader(Class<S> SVC, ClassLoader cl) {private ServiceLoader(Class<S> SVC, ClassLoader CL) {private ServiceLoader(Class<S> SVC, ClassLoader CL) {private ServiceLoader(Class<S> SVC, ClassLoader CL) { "Service interface cannot be null"); (cl == null)? ClassLoader.getSystemClassLoader() : cl; acc = (System.getSecurityManager() ! = null) ? AccessController.getContext() : null; // Reload (); }Copy the code

2.3.3 reload () method

public void reload() { providers.clear(); LazyIterator lookupIterator = new LazyIterator(service, loader); }Copy the code

2.3.4 serviceLoader. The iterator () method

Public Iterator<S> Iterator () {return new Iterator<S>() {// Get cache Iterator< map. Entry<String,S>> knownProviders = providers.entrySet().iterator(); Public Boolean hasNext() {public Boolean hasNext() {if (knownproviders.hasNext ()) return true; Return lookupiterator.hasnext (); return lookupiterator.hasnext (); If (knownproviders.hasNext ()) return knownproviders.next ().getValue(); if (knownproviders.hasnext ()) return knownproviders.next ().getValue(); Return lookupiterator.next (); return lookupiterator.next (); } public void remove() { throw new UnsupportedOperationException(); }}; }Copy the code

2.3.5 hasNext () method

public boolean hasNext() { if (acc == null) { return hasNextService(); } else { PrivilegedAction<Boolean> action = new PrivilegedAction<Boolean>() { public Boolean run() { return hasNextService(); }}; return AccessController.doPrivileged(action, acc); }}Copy the code

2.3.6 hasNextService () method

private boolean hasNextService() { if (nextName ! = null) { return true; } if (configs == null) {try {// Obtain the name from meta-inf.services String fullName = PREFIX + service.getName(); if (loader == null) configs = ClassLoader.getSystemResources(fullName); else configs = loader.getResources(fullName); } catch (IOException x) { fail(service, "Error locating configuration files", x); } } while ((pending == null) || ! pending.hasNext()) { if (! configs.hasMoreElements()) { return false; Parse (service, configs.nextelement ()); } // assign nextName = pending.next(); return true; }Copy the code

2.3.7 lookupIterator. Next () method

Public S next() {if (acc == null) {return nextService(); } else { PrivilegedAction<S> action = new PrivilegedAction<S>() { public S run() { return nextService(); }}; return AccessController.doPrivileged(action, acc); }}Copy the code

2.3.8 nextService () method

private S nextService() { if (! hasNextService()) throw new NoSuchElementException(); String cn = nextName; nextName = null; Class<? > c = null; Try {// reflection get Class c = class.forname (cn, false, loader); } catch (ClassNotFoundException x) { fail(service, "Provider " + cn + " not found"); } if (! service.isAssignableFrom(c)) { fail(service, "Provider " + cn + " not a subtype"); } try {// instantiate the implementation class S p = service.cast(c.newinstance ()); Providers. Put (cn, p); return p; } catch (Throwable x) { fail(service, "Provider " + cn + " could not be instantiated", x); } throw new Error(); // This cannot happen }Copy the code

So far all source code analysis is finished.

2.4 Database Drivers

DriverManager is a JDBC tool class that manages and registers different database drivers. From the point of view of its original design, it bears some similarities to the scenes we discussed earlier. There may be different database-driven implementations for a database. When we use a specific driver implementation, we don’t want to modify the existing code, and we want a simple configuration to achieve this effect.

After loading the mysql Driver with class.forname (” com.mysql.jdbc.driver “), static code is executed to register the Driver with DriverManager for subsequent use.

    static {
        try {
            java.sql.DriverManager.registerDriver(new Driver());
        } catch (SQLException E) {
            throw new RuntimeException("Can't register driver!");
        }
    }
Copy the code

View the static code block for DriverManager

    static {
        loadInitialDrivers();
        println("JDBC DriverManager initialized");
    }
Copy the code

The loadInitialDrivers() method has this line

ServiceLoader<Driver> loadedDrivers = ServiceLoader.load(Driver.class);
                Iterator<Driver> driversIterator = loadedDrivers.iterator();
Copy the code

You can see that DriverManager uses spi in its initialization, using the ServiceLoader to load the Driver written to the configuration file.

3. SPI mechanism in Dubbo

3.1 Why does Dubbo not use SPI written in Java

Having seen the Java SPI example above, we need to ask ourselves a question: What are the shortcomings of Java’s implementation?

Java’s SPI mechanism, which fetches everything in a file, is too rigid to support specified fetches. If we only want to fetch an implementation class, the SPI in Java does not work.

The easiest way to do this is to define content like this

dog = spi.Dog
cat = spi.Cat
Copy the code

It then loads the content into a Map and gets it by name. So what does Dubbo do? Let’s dig into the source code.

3.2 the Demo

Let’s start with a simple Demo of Dubbo SPI

Create a project with the following dependencies:

<? The XML version = "1.0" encoding = "utf-8"? > < project XMLNS = "http://maven.apache.org/POM/4.0.0" XMLNS: xsi = "http://www.w3.org/2001/XMLSchema-instance" Xsi: schemaLocation = "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" > The < modelVersion > 4.0.0 < / modelVersion > < groupId > org. Example < / groupId > < artifactId > DubboDemo < / artifactId > 1.0 the SNAPSHOT < version > < / version > < dependencies > < the dependency > < groupId > org. Apache. Dubbo < / groupId > <artifactId>dubbo</artifactId> <version>2.7.5</version> </dependency> </dependencies> </dependencies> </project>Copy the code

Create an interface and implementation class

package com.dubbo.spi;

import org.apache.dubbo.common.extension.SPI;

@SPI
public interface Animal {

    void hello();

}
Copy the code
package com.dubbo.spi; public class Cat implements Animal { public void hello() { System.out.println("This is Cat"); }}Copy the code
package com.dubbo.spi; public class Dog implements Animal { public void hello() { System.out.println("This is Dog"); }}Copy the code

In the resources directory, create a meta-INF directory. In this directory, create a dubbo directory and create a com.dubo.spi. Animal file

The contents of the document are as follows:

dog = com.dubbo.spi.Dog
cat = com.dubbo.spi.Cat
Copy the code

The test class

package com.dubbo.spi; import org.apache.dubbo.common.extension.ExtensionLoader; public class TestDubboSPI { public static void main(String[] args) { ExtensionLoader<Animal> extensionLoader = ExtensionLoader.getExtensionLoader(Animal.class); Animal dog = extensionLoader.getExtension("dog"); dog.hello(); }}Copy the code

The test results

This is Dog
Copy the code

4. Source code analysis

4.1 Definitions of directories in Dubbo

Let’s take a look at Dubbo’s convention for configuration file directories. Unlike the Java SPI, Dubbo has three categories of directories.

  • Meta-inf /services/ directory: The SPI configuration files in this directory are intended to be compatible with the Java SPI.
  • Meta-inf /dubbo/ directory: This directory stores user-defined SPI configuration files.
  • Meta-inf /dubbo/internal/ : This directory stores SPI configuration files for internal use in Dubbo.

4.2 Source Code Entry

Look at the Demo above

ExtensionLoader<Animal> extensionLoader =
                ExtensionLoader.getExtensionLoader(Animal.class);
        Animal dog = extensionLoader.getExtension("dog");
Copy the code

Following the Java SPI source code conventions, these two lines generate an ExtensionLoader object based on the interface name, and then instantiate the implementation class based on the implementation class name.

4.3 ExtensionLoader. GetExtensionLoader () method

Animal -> ExtensionLoader<Animal> private static final ConcurrentMap Class<? >, ExtensionLoader<? >> EXTENSION_LOADERS = new ConcurrentHashMap<>(); Public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {// omit some validation logic // getExtensionLoader <T> object from the cache ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type); // If the cache is empty, create ExtensionLoader<T> PutIfAbsent (type, new ExtensionLoader<T>(type)); if (loader == null) {extension_loaders. putIfAbsent(type, new ExtensionLoader<T>(type)); // Get ExtensionLoader<T> loader = (ExtensionLoader<T>) extension_loaders.get (type); } // return ExtensionLoader<T> return loader; }Copy the code

4.4 ExtensionLoader. GetExtension () method

Since this is the first time an instance is created, the createExtension logic is followed

Public T getExtension(String name) {// Check name if (stringutils.isEmpty (name)) {throw new IllegalArgumentException("Extension name == null"); } // Return the default instance if ("true".equals(name)) {return getDefaultExtension(); } final Holder<Object> Holder = getOrCreateHolder(name); Object instance = holder.get(); Synchronized (holder) {instance = holder.get(); synchronized (holder) {instance = holder.get(); If (instance == null) {// Create instance instance = createExtension(name); holder.set(instance); } } } return (T) instance; }Copy the code

4.5 createExtension () method

Private T createExtension(String name) { > clazz = getExtensionClasses().get(name); if (clazz == null) { throw findException(name); } try {// get instance from cache T instance = (T) EXTENSION_INSTANCES. Get (clazz); PutIfAbsent (clazz, clazz.newinstance ()); if (instance == null) {// Create EXTENSION_INSTANCES. instance = (T) EXTENSION_INSTANCES.get(clazz); } injectExtension(instance); Set<Class<? >> wrapperClasses = cachedWrapperClasses; if (CollectionUtils.isNotEmpty(wrapperClasses)) { for (Class<? > wrapperClass : wrapperClasses) { instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance)); } } initExtension(instance); return instance; } catch (Throwable t) { throw new IllegalStateException("Extension instance (name: " + name + ", class: " + type + ") couldn't be instantiated: " + t.getMessage(), t); }}Copy the code

The current process is as follows:

So how does the getExtensionClasses() method generate classes?

4.5.1 getExtensionClasses () method

The same routine is fetched from the cache first, but not from loadExtensionClasses()

private Map<String, Class<? >> getExtensionClasses() { Map<String, Class<? >> classes = cachedClasses.get(); if (classes == null) { synchronized (cachedClasses) { classes = cachedClasses.get(); if (classes == null) { classes = loadExtensionClasses(); cachedClasses.set(classes); } } } return classes; }Copy the code

4.5.2 loadExtensionClasses () method

Here we load the files from the three directories we mentioned above.

private Map<String, Class<? >> loadExtensionClasses() { cacheDefaultExtensionName(); Map<String, Class<? >> extensionClasses = new HashMap<>(); // internal extension load from ExtensionLoader's ClassLoader first loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName(), true); loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"), true); loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName()); loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName().replace("org.apache", "com.alibaba")); loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName()); loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName().replace("org.apache", "com.alibaba")); return extensionClasses; }Copy the code

4.5.3 loadDirectory () method

private void loadDirectory(Map<String, Class<? >> extensionClasses, String dir, String type, boolean extensionLoaderClassLoaderFirst) { String fileName = dir + type; try { Enumeration<java.net.URL> urls = null; ClassLoader classLoader = findClassLoader(); // try to load from ExtensionLoader's ClassLoader first if (extensionLoaderClassLoaderFirst) { ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader(); if (ClassLoader.getSystemClassLoader() ! = extensionLoaderClassLoader) { urls = extensionLoaderClassLoader.getResources(fileName); } } if(urls == null || ! urls.hasMoreElements()) { if (classLoader ! = null) { urls = classLoader.getResources(fileName); } else { urls = ClassLoader.getSystemResources(fileName); } } if (urls ! = null) { while (urls.hasMoreElements()) { java.net.URL resourceURL = urls.nextElement(); // Load logic loadResource(extensionClasses, classLoader, resourceURL); } } } catch (Throwable t) { logger.error("Exception occurred when loading extension class (interface: " + type + ", description file: " + fileName + ").", t); }}Copy the code

4.5.4 loadResource () method

This method will eventually call the loadClass method.

4.5.6 loadClass () method

Let’s get rid of some unnecessary logic here

private void loadClass(Map<String, Class<? >> extensionClasses, java.net.URL resourceURL, Class<?> clazz, String name) throws NoSuchMethodException {/ / if there is the Adaptive annotation is deposited in the corresponding cache if (clazz. IsAnnotationPresent (Adaptive. Class)) { cacheAdaptiveClass(clazz); } else if (isWrapperClass(clazz)) {cacheWrapperClass(clazz); } else {// clazz.getconstructor (); if (StringUtils.isEmpty(name)) { name = findAnnotationName(clazz); if (name.length() == 0) { throw new IllegalStateException("No such extension name for the class " + clazz.getName() + " in the config " + resourceURL); } } String[] names = NAME_SEPARATOR.split(name); CacheActivateClass (clazz, names[0]); if (arrayUtils.isnotempty (names)) {// cacheActivateClass(clazz, names[0]); For (String n: names) {// Put in a cache cacheName(clazz, n); saveInExtensionClass(extensionClasses, clazz, n); }}}}Copy the code

The improvement flow chart is as follows:

So what are Adaptive annotations?

4.6 the Adaptive annotations

The problem with this example is that all calls are fixed at startup. Suppose there is a requirement that the interface class is Pay and there are two implementation classes, one AliPay and one WeChatPay. We need to dynamically decide which implementation class to access by passing in parameters. The Adaptive annotation can do this instead of the previous SPI implementation.

This annotation is an adaptive extension annotation that can modify classes and methods, but when you modify a class, you don’t generate a proxy class, because that class is a proxy class, and when you modify a method, you generate a proxy class.

4.6.1 use

Here’s an example of an annotation on a method like this:

package com.dubbo.spi.auto; import org.apache.dubbo.common.extension.Adaptive; import org.apache.dubbo.common.extension.SPI; import org.apache.dubbo.common.URL; @adaptive ({"payType"}) void Pay (URL URL); }Copy the code
package com.dubbo.spi.auto; import org.apache.dubbo.common.URL; public class AliPay implements Pay { public void pay(URL url) { System.out.println("User AliPay"); }}Copy the code
package com.dubbo.spi.auto; import org.apache.dubbo.common.URL; public class WeChatPay implements Pay { public void pay(URL url) { System.out.println("Use WeChat"); }}Copy the code

Add this file com.dubo.spi.auto.Pay to meta-inf.dubbo:

wechat = com.dubbo.spi.auto.WeChatPay
ali = com.dubbo.spi.auto.AliPay
Copy the code

The test class

package com.dubbo.spi.auto;

import org.apache.dubbo.common.URL;
import org.apache.dubbo.common.extension.ExtensionLoader;

public class TestAutoSPI {

    public static void main(String[] args) {
        ExtensionLoader<Pay> extensionLoader = ExtensionLoader.getExtensionLoader(Pay.class);
        Pay pay = extensionLoader.getAdaptiveExtension();
        pay.pay(URL.valueOf("http://localhost:8080/"));
        pay.pay(URL.valueOf("http://localhost:8080?payType=wechat"));
    }

}
Copy the code

The final output

User AliPay
Use WeChat
Copy the code

Success!!!!!!!!!!

4.6.2 Source code analysis

4.6.2.1 Annotations on classes

Annotation on a class called logic simpler createAdaptiveExtension () – > createAdaptiveExtension () – > getAdaptiveExtensionClass () – > GetExtensionClasses () is the same logic here.

4.6.2.2 Annotations on methods

Comments on the way there is a little different, it is finally not call getExtensionClasses (), but call createAdaptiveExtensionClass () to generate the proxy class

private Class<? > createAdaptiveExtensionClass() { String code = new AdaptiveClassCodeGenerator(type, cachedDefaultName).generate(); ClassLoader classLoader = findClassLoader(); org.apache.dubbo.common.compiler.Compiler compiler = ExtensionLoader.getExtensionLoader(org.apache.dubbo.common.compiler.Compiler.class).getAdaptiveExtension(); return compiler.compile(code, classLoader); }Copy the code

In the example above, you end up with a proxy class that gets the name of the implementation class to be invoked based on the request parameters, the URL, and then calls getExtension.

package com.dubbo.spi.auto;

import org.apache.dubbo.common.extension.ExtensionLoader;

public class Pay$Adaptive implements com.dubbo.spi.auto.Pay {
	public void pay(org.apache.dubbo.common.URL arg0)  {
		if (arg0 == null) throw new IllegalArgumentException("url == null");
		org.apache.dubbo.common.URL url = arg0;
		String extName = url.getParameter("payType", "ali");
			if(extName == null) 
				throw new IllegalStateException("Failed to get extension (com.dubbo.spi.auto.Pay) name from url (" + url.toString() + ") use keys([payType])");
					com.dubbo.spi.auto.Pay extension = (com.dubbo.spi.auto.Pay)ExtensionLoader.getExtensionLoader(
						com.dubbo.spi.auto.Pay.class).getExtension(extName);
						extension.pay(arg0);
		}
}
Copy the code

4.7 WrapperClass – AOP

When looking at the source code, notice that there is a function to determine whether the class is a Wrapper. This function is an AOP function in Dubbo. Here is a simple example of how to use it.

4.7.1 use

AnimalWrapper class AnimalWrapper class AnimalWrapper class

package com.dubbo.spi.wrapper; import com.dubbo.spi.Animal; public class AnimalWrapper implements Animal { private final Animal animal; public AnimalWrapper(Animal animal) { this.animal = animal; } public void hello() { System.out.println("before"); animal.hello(); System.out.println("after"); }}Copy the code

Add a new line to the com.dubo.spi. Animal file:

wrapper = com.dubbo.spi.wrapper.AnimalWrapper
Copy the code

Start the class:

package com.dubbo.spi.wrapper; import com.dubbo.spi.Animal; import org.apache.dubbo.common.extension.ExtensionLoader; public class TestWrapper { public static void main(String[] args) { ExtensionLoader<Animal> extensionLoader = ExtensionLoader.getExtensionLoader(Animal.class); Animal wrapper = extensionLoader.getExtension("cat"); wrapper.hello(); }}Copy the code

The output is:

before
This is Dog
after
Copy the code

Given this result, it’s worth guessing how Dubbo did it.

  1. Get the AnimalWrapper class from the file
  2. When creating an instance, pass the Cat class to the AnimalWrapper constructor, so that the Animal instance in AnimalWrapper is Cat, and then create the AnimalWrapper instance
  3. The animalWrapper.hello () method is finally called

4.7.2 Source code analysis

4.7.2.1 isWrapperClass () method

This method is used to determine whether the class is a Wrapper class. If the constructor of a class takes a given type, it is a Wrapper class, and in this case, whether the constructor of an AnimalWrapper class takes an Animal interface.

    private boolean isWrapperClass(Class<?> clazz) {
        try {
            clazz.getConstructor(type);
            return true;
        } catch (NoSuchMethodException e) {
            return false;
        }
    }
Copy the code

4.7.2.2 cacheWrapperClass () method

We will take the Wrapper class from the cache and re-create it if it is not in the cache. In this case, we will add the AnimalWrapper class to the cache.

private void cacheWrapperClass(Class<? > clazz) { if (cachedWrapperClasses == null) { cachedWrapperClasses = new ConcurrentHashSet<>(); } cachedWrapperClasses.add(clazz); }Copy the code

4.7.2.3 Creating an Instance

These lines of code will eventually create and return an instance of the AnimalWrapper class, and the Wrapper class logic will be analyzed.

Set<Class<? >> wrapperClasses = cachedWrapperClasses; if (CollectionUtils.isNotEmpty(wrapperClasses)) { for (Class<? > wrapperClass : wrapperClasses) { instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance)); }}Copy the code

4.8 injectExtension – the IOC

All I need is this last piece of code.

private T injectExtension(T instance) { if (objectFactory == null) { return instance; } for (Method Method: instance.getClass().getmethods ()) {if (! isSetter(method)) { continue; } /** * Check {@link DisableInject} to see if we need auto injection for this property */ if (method.getAnnotation(DisableInject.class) ! = null) { continue; } Class<? > pt = method.getParameterTypes()[0]; if (ReflectUtils.isPrimitives(pt)) { continue; } try {// find Setter method String property = getSetterProperty(method); Object object = objectFactory.getExtension(pt, property); if (object ! // invoke method. Invoke (instance, object); } } catch (Exception e) { logger.error("Failed to inject via method " + method.getName() + " of interface " + type.getName() + ": " + e.getMessage(), e); } } } catch (Exception e) { logger.error(e.getMessage(), e); } return instance; }Copy the code

At this point all code analysis is complete.

The resources

  1. Java SPI thought comb
  2. Dubbo Series -Dubbo SPI mechanism – Ao Bing