Resources: OBing _Dubbo’s SPI
What is the SPI
SPI: Service Provider Interface. Literally, it’s hard to understand what this thing does.
For example, there are a lot of different databases out there, and the underlying protocols for each database are different. You can’t develop different API access for each database. Therefore, you need to have an interface that defines some canonical protocol for accessing the database, and all the databases are developed according to this interface. This interface is java.sql.Driver. Since database vendors develop corresponding implementation classes according to the interface, but there is a problem, when really used, which implementation class to use, where to find the implementation class?
The answer is that it is a convention to write the configuration of the implementation class somewhere, and to look it up when you need to use it.
Java SPI
The Java SPI convention is to create a file named the service interface in the META-INF/services/ directory of the classpath with the fully qualified name of the implementation class.
Here’s how MySQL does it
Java SPI sample
We define an interface, then create two implementation classes, and create a file with the fully qualified name of the service interface in the meta-INF /services directory on the classpath.
public interface IHelloSPI {
void say(a);
}
@Slf4j
public class HelloSPIImpl1 implements IHelloSPI {
@Override
public void say(a) {
log.info("-- >> I am implementation class 1"); }}@Slf4j
public class HelloSPIImpl2 implements IHelloSPI {
@Override
public void say(a) {
log.info("-- >> I am implementation class 2"); }}Copy the codecom.drh.springboot_hello.spi.java.HelloSPIImpl1
com.drh.springboot_hello.spi.java.HelloSPIImpl2
Copy the codepublic class JavaSPITest {
public static void main(String[] args) {
// The load method creates a ServiceLoader and LazyIterator object
ServiceLoader<IHelloSPI> serviceLoader = ServiceLoader.load(IHelloSPI.class);
// Get the LazyIterator object
Iterator<IHelloSPI> iterator = serviceLoader.iterator();
// The hasNext method parses the contents of the interface file, iterating through each line to get the fully qualified name of the implementation class
while (iterator.hasNext()) {
// Next method, based on the fully qualified name, creates the class object, and then creates the instance object by reflectioniterator.next().say(); }}}Copy the codeThe output
Java SPI source code analysis
As you can see in the code above, ServiceLoader#load is an entry point to the Java SPI. Let’s trace the code to see what’s going on.
As you can see from the above call, first find the class loader for the current thread binding, if not, use the system loader. Then clear the cache and create a ServiceLoader and LazyIterator.
LazyIterator is a subclass of Iterator. Therefore, it can be inferred that the serviceLoader#iterator is actually getting the LazyIterator instance object.
Let’s move on to the lazyator #hasNext method
private static final String PREFIX = "META-INF/services/";
private boolean hasNextService(a) {
if(nextName ! =null) {
return true;
}
if (configs == null) {
try {
// Find the file location
String fullName = PREFIX + service.getName();
// Load the file
if (loader == null)
configs = ClassLoader.getSystemResources(fullName);
else
configs = loader.getResources(fullName);
} catch (IOException x) {
fail(service, "Error locating configuration files", x); }}// Walk through the file line by line
while ((pending == null) | |! pending.hasNext()) {if(! configs.hasMoreElements()) {return false;
}
// Parse the content
pending = parse(service, configs.nextElement());
}
// Assign to nextName, which will be used by the nextService method later
nextName = pending.next();
return true;
}
Copy the codeAs you can see, the LazyIterator#hasNextService method finds the interface file according to the convention path, and then parses the contents of the file line by line.
Next, let’s look at the Lazyator # Next method
private S nextService(a) {
if(! hasNextService())throw new NoSuchElementException();
// Get the line data from the hasNextService parse file
String cn = nextName;
nextName = null; Class<? > c =null;
try {
// Implement the class object of the class
c = Class.forName(cn, false, loader);
} catch (ClassNotFoundException x) {
fail(service,
"Provider " + cn + " not found");
}
if(! service.isAssignableFrom(c)) { fail(service,"Provider " + cn + " not a subtype");
}
try {
// Create an implementation instance object and force it into an interface reference
S p = service.cast(c.newInstance());
// Save the instance object to the cache
providers.put(cn, p);
return p;
} catch (Throwable x) {
fail(service,
"Provider " + cn + " could not be instantiated",
x);
}
throw new Error(); // This cannot happen
}
Copy the codeLazyIterator#nextService gets the implementation class object from the fully qualified name, creates the instance object through reflection, puts it in the cache, and returns the result.
Summary: Java SPI relies on the ServiceLoader implementation. When the Load method is called, a ServiceLoader object is created with its internal property LazyIterator object. Next, the ServiceLoader retrieves the iterator, the LazyIterator.
Call the iterator #hasNext to the meta-INF /services directory to find a file named with the fully qualified name of the service interface, and walk through the line to get the fully qualified name of the implementation class. Finally, we call the next method and invoke methods like class.forname and newInstance based on the fully qualified name of the Class to create the instance object and store it in the cache.
Duboo SPI
Because the Java SPI instantiates an object for all implementation classes, it can be a waste of resources if the object is not needed in the system. To enable on-demand loading, Dubbo implements an SPI of its own, which loads the corresponding concrete implementation class with a specified name.
Let’s take a look at dubbo’s convention for configuration file directories, which fall into three categories:
META-INF/services/: This directory is used for Java SPI compatibilityMETA-INF/dubbo/: This directory stores user-defined configuration filesMETA-INF/dubbo/internal/: This directory houses SPI configuration files for internal use in Dubbo
Dubbo SPI sample
Specify a service interface and use the @spi annotation to indicate an SPI using Dubbo. Then, in the meta-INF /dubbo/ directory, create a file named after the service interface, with key-value content representing the relationship between the parameter and the implementation class.
@SPI
public interface IHelloDubboSPI {
void say(a);
}
@Slf4j
public class HelloDubboSPIImpl implements IHelloDubboSPI {
@Override
public void say(a) {
log.info("-- >> Dubbo SPI example...); }}Copy the code
impl1=spi.dubbo.HelloDubboSPIImpl
Copy the code@SpringBootTest
public class DubboSPITest {
@Test
public void testDubboSPI(a) {
ExtensionLoader<IHelloDubboSPI> extensionLoader = ExtensionLoader.getExtensionLoader(IHelloDubboSPI.class);
IHelloDubboSPI impl1 = extensionLoader.getExtension("impl1"); impl1.say(); }}Copy the code
Dubbo source code analysis
As you can see from the above test code, the general flow is to find the corresponding ExtensionLoader through the interface class object, and then get the concrete implementation class object with the specified name through the getExtension method.
Take a look at the ExtensionLoader#getExtensionLoader method
public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
// Ignore a bunch of parameter validations
ExtensionLoader<T> loader = (ExtensionLoader)EXTENSION_LOADERS.get(type);
if (loader == null) {
New ExtensionLoader(type) -> this method uses adaptive extension
EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader(type));
loader = (ExtensionLoader)EXTENSION_LOADERS.get(type);
}
return loader;
}
private ExtensionLoader(Class
type) {
this.type = type;
this.objectFactory = type == ExtensionFactory.class ?
null : (ExtensionFactory)getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension();
}
Copy the codeThe getExtensionLoader method is simple. It first looks into the cache to see if there is already an ExtensionLoader of the specified type. If not, it creates one and puts it in the cache.
Next, let’s look at getting the concrete implementation class object, ExtensionLoader#getExtension
public T getExtension(String name) {
// Omit some code
// Get the specified implementation object from the cache
Holder<Object> holder = (Holder)this.cachedInstances.get(name);
if (holder == null) {
this.cachedInstances.putIfAbsent(name, new Holder());
holder = (Holder)this.cachedInstances.get(name);
}
Object instance = holder.get();
if (instance == null) { // Double check
synchronized(holder) {
instance = holder.get();
if (instance == null) {
// Key!! Creating instance Objects
instance = this.createExtension(name); holder.set(instance); }}}return instance;
}
Copy the codeAs you can see from the code, the focus is this.createExtension(name), moving on
private T createExtension(String name) {
// Get the class object of the implementation classClass<? > clazz = (Class)this.getExtensionClasses().get(name);
if (clazz == null) {
throw this.findException(name);
} else {
try {
// Check whether the cache already has instance objects
T instance = EXTENSION_INSTANCES.get(clazz);
if (instance == null) {
// If not, call newInstance to create and cache it
EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
instance = EXTENSION_INSTANCES.get(clazz);
}
// setdependency injection
this.injectExtension(instance); Set<Class<? >> wrapperClasses =this.cachedWrapperClasses;
Class wrapperClass;
// If there is a wrapper class, just wrap it
if(wrapperClasses ! =null && !wrapperClasses.isEmpty()) {
for(Iterator var5 = wrapperClasses.iterator(); var5.hasNext();
instance = this.injectExtension(wrapperClass.getConstructor(this.type).newInstance(instance))) { wrapperClass = (Class)var5.next(); }}return instance;
} catch (Throwable var7) {
throw new IllegalStateException("xxx"); }}}Copy the codeThe method logic is clear: find the implementation class, see if there are any instance objects in the cache, create them by reflection, and perform set dependency injection.
At this point, let’s look at how to find the implementation class.
Find the implementation class
Let’s move on to ExtensionLoader#getExtensionClasses
privateMap<String, Class<? >> getExtensionClasses() { Map<String, Class<? >> classes = (Map)this.cachedClasses.get();
if (classes == null) {
synchronized(this.cachedClasses) {
classes = (Map)this.cachedClasses.get();
if (classes == null) {
// Key!! Look it up from the cache first, if not, call the loadExtensionClasses method
classes = this.loadExtensionClasses();
this.cachedClasses.set(classes); }}}return classes;
}
privateMap<String, Class<? >> loadExtensionClasses() {// Dubbo's SPI annotation reads its value
SPI defaultAnnotation = (SPI)this.type.getAnnotation(SPI.class);
if(defaultAnnotation ! =null) {
String value = defaultAnnotation.value();
// Check the name of the SPI interface definition
if ((value = value.trim()).length() > 0) {
String[] names = NAME_SEPARATOR.split(value);
if (names.length > 1) {
throw new IllegalStateException(
"more than 1 default extension name on extension " + this.type.getName() + ":" +
Arrays.toString(names));
}
if (names.length == 1) {
// SPI's value is the default implementation class for the service interface
this.cachedDefaultName = names[0]; }}}// Find the three directories of the conventionMap<String, Class<? >> extensionClasses =new HashMap();
this.loadDirectory(extensionClasses, "META-INF/dubbo/internal/");
this.loadDirectory(extensionClasses, "META-INF/dubbo/");
this.loadDirectory(extensionClasses, "META-INF/services/");
return extensionClasses;
}
Copy the codeLoadDirectory scans the three directories of the convention, finds the service interface file, parses and reads the contents of the file, loads these implementation objects, and caches them through loadClass.
Before executing the loadClass method, the implementation Class has been loaded, class.forname (XXX), loadClass method only according to the annotation above the implementation Class to do different caching operations, respectively, there are three kinds of Adaptive, WrapperClass and ordinary Class.
loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name);
private void loadClass(Map
> extensionClasses, java.net.URL resourceURL, Class
clazz, String name)
,> {
// Omit some code
clazz.getConstructor();
String[] names = NAME_SEPARATOR.split(name);
if(names ! =null && names.length > 0) {
Activate activate = clazz.getAnnotation(Activate.class);
if(activate ! =null) {
cachedActivates.put(names[0], activate);
}
for (String n : names) {
if(! cachedNames.containsKey(clazz)) { cachedNames.put(clazz, n); } Class<? > c = extensionClasses.get(n);if (c == null) { extensionClasses.put(n, clazz); }}}}Copy the codeconclusion
This is the end of Dubbo SPI’s analysis. For the analysis of Adaptive and WrapperClass, you can read aobing’s article, which cannot be digested for the time being. Finally, let’s conclude
To get an instance object with the specified name, create the ExtensionLoader corresponding to the interface, then call the getExtension method to find the specified subclass object based on the name in the method argument. First, go to the meta-INF /service/meta-INF /dubbo/internal directory, find the file named after the service interface, parse the file contents, and load all the implementation classes in the file. And use HashMap to store the mapping between name and implementation class objects, key->name, value->class objects. Finally, according to the getExtension method parameter, find the corresponding class object from the map, create the instance object through reflection and return it.
Conclusion:
Dubbo SPI is implemented in much the same way as Java SPI in that it is a convention directory with a service interface as the file name. The difference is that Dubbo SPI specifies the relationship between the name and the implementation class.
Note: Loading a class and instantiating an object are two different things. Java SPI loads implementation classes and instantiates objects, while Dubbo SPI loads classes and instantiates on demand.