SPI, also known as the Service Provider Interface, is a Service discovery mechanism. When the program runs to invoke the interface, the corresponding implementation class is loaded based on the configuration file or default rule information. So instead of specifying which implementation of the interface to use directly in the program, it is assembled externally. In order to understand the design and implementation of Dubbo, it is necessary to understand the loading mechanism of Dubbo SPI. There are a lot of functions implemented in Dubbo that are decoupled based on Dubbo SPI, which also makes Dubbo so scalable.
Java SPI
See how it works by performing an operation on a Java SPI.
- Create an AnimalService interface and a category method
- Create an implementation class Cat
- Create a meta-INF /services directory and create a file in that directory with the fully qualified name of AnimalService as the file name
- Add the fully qualified name of the implementation class Cat to the file
Animal interface
public interface AnimalService {
void category(a);
}
Copy the codeThe Cat implementation class
public class Cat implements AnimalService {
@Override
public void category(a) {
System.out.println("cat: Meow ~"); }}Copy the codeIn the meta-inf/services directory. Top ytao. Demo. Spi. AnimalService file:
top.ytao.demo.spi.Cat
Copy the codeLoad the implementation of SPI:
public class JavaSPITest {
@Test
public void javaSPI(a) throws Exception {
ServiceLoader<AnimalService> serviceLoader = ServiceLoader.load(AnimalService.class);
// Iterate through all the implementation classes of the AnimalService configured in the configuration file
for(AnimalService animalService : serviceLoader) { animalService.category(); }}}Copy the codeExecution Result:
At this point, a Java SPI implementation is complete, and the serviceloader.load gets the configured interface implementation classes that load all interfaces, and you can then iterate to find the required implementations.
Dubbo SPI
Dubbo SPI is more powerful than Java SPI, and is a set of SPI mechanisms implemented by itself. Major improvements and optimizations:
- Instead of Java SPI loading all implementations at once, Dubbo SPI loads on demand, loading only the implementation classes you need to use. Also with cache support.
- More detailed information about the extension load failure.
- Added support for extending IOC and AOP.
Dubbo SPI sample
The Dubbo SPI configuration file is placed under meta-INF/Dubbo, and the implementation class is configured in k-V mode, where key is the parameter passed in the instantiation object and value is the fully qualified name of the extension point implementation class. For example, the contents of the Cat configuration file:
cat = top.ytao.demo.spi.Cat
Copy the codeThe Java SPI directory is also compatible during Dubbo SPI loading.
Add @spi annotation to the interface. @spi can specify a key value.
public class DubboSPITest {
@Test
public void dubboSPI(a){
ExtensionLoader<AnimalService> extensionLoader = ExtensionLoader.getExtensionLoader(AnimalService.class);
// Get the extension class implementation
AnimalService cat = extensionLoader.getExtension("cat");
System.out.println("Dubbo SPI"); cat.category(); }}Copy the codeThe result is as follows:
Get the ExtensionLoader instance
GetExtensionLoader instance by getExtensionLoader method.
public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
if (type == null) {
throw new IllegalArgumentException("Extension type == null");
}
// Check that type must be interface
if(! type.isInterface()) {throw new IllegalArgumentException("Extension type (" + type + ") is not an interface!");
}
// Check if the interface has SPI annotations
if(! withExtensionAnnotation(type)) {throw new IllegalArgumentException("Extension type (" + type +
") is not an extension, because it is NOT annotated with @" + SPI.class.getSimpleName() + "!");
}
// Get ExtensionLoader instance from cache
ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
if (loader == null) {
// Load the ExtensionLoader instance into the cache
EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
}
return loader;
}
Copy the codeThe above process of obtaining the extended class loader is mainly to check whether the type passed is valid and whether there is an interface of the current type from the extended class loader cache. If not, add the current interface to the cache. ConcurrentMap
, ExtensionLoader
> EXTENSION_LOADERS is a cache of extension classloaders that cache interfaces as keys and extension classloaders as values.
Gets the extension class object
The method ExtensionLoader#getExtension is used to cache and create the extension object:
public T getExtension(String name) {
if (StringUtils.isEmpty(name)) {
throw new IllegalArgumentException("Extension name == null");
}
// If true is passed in, the default extension class object operation is obtained
if ("true".equals(name)) {
return getDefaultExtension();
}
// Get the extension object. The value property in the Holder holds the extension object instance
final Holder<Object> holder = getOrCreateHolder(name);
Object instance = holder.get();
// Use double-checked locks
if (instance == null) {
synchronized (holder) {
instance = holder.get();
if (instance == null) {
// Create an extension objectinstance = createExtension(name); holder.set(instance); }}}return (T) instance;
}
Copy the code(ConcurrentMap
> cachedInstances); Set to extended object cache. If it is a newly created instance of an extension object, then holder.get() must be null. If the extension object is empty, the extension object is created through double-checked locks.
Creating an extension object
Procedure for creating an extension object:
private T createExtension(String name) {
// From all extension classes, get the extension class corresponding to the current extension nameClass<? > clazz = getExtensionClasses().get(name);if (clazz == null) {
throw findException(name);
}
try {
// Get the extension instance from the cache and set the extension instance cache
T instance = (T) EXTENSION_INSTANCES.get(clazz);
if (instance == null) {
EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
instance = (T) EXTENSION_INSTANCES.get(clazz);
}
// Inject dependencies into the current instance
injectExtension(instance);
// Get the wrapper extension class cacheSet<Class<? >> wrapperClasses = cachedWrapperClasses;if (CollectionUtils.isNotEmpty(wrapperClasses)) {
for(Class<? > wrapperClass : wrapperClasses) {// Create a wrapper extension class instance and inject dependencies into itinstance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance)); }}// Initialize the extension object
initExtension(instance);
return instance;
} catch (Throwable t) {
throw new IllegalStateException("Extension instance (name: " + name + ", class: " +
type + ") couldn't be instantiated: "+ t.getMessage(), t); }}Copy the codeWhen creating the extension above, there is a Wrapper class that uses the decorator pattern. This class has no concrete implementation but abstracts the common logic. The process of creating this is to get the mapping of the current extension from all the extension classes and to inject dependencies into the current extension object.
Get all extension classes:
privateMap<String, Class<? >> getExtensionClasses() {// Get the normal extended class cacheMap<String, Class<? >> classes = cachedClasses.get();// If not in cache, load by double-checking lock
if (classes == null) {
synchronized (cachedClasses) {
classes = cachedClasses.get();
if (classes == null) {
// Load all extension classesclasses = loadExtensionClasses(); cachedClasses.set(classes); }}}return classes;
}
Copy the codeCheck whether the normal extension class cache is empty, if not reload, and actually load the extension class in loadExtensionClasses:
private static final String SERVICES_DIRECTORY = "META-INF/services/";
private static final String DUBBO_DIRECTORY = "META-INF/dubbo/";
private static final String DUBBO_INTERNAL_DIRECTORY = DUBBO_DIRECTORY + "internal/";
privateMap<String, Class<? >> loadExtensionClasses() {// Get the default extension on @spicacheDefaultExtensionName(); Map<String, Class<? >> extensionClasses =new HashMap<>();
// First load Dubbo extension class, via Boolean control
loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName(), true);
// Because Dubbo moved to Apache, the package name has been changed, and the previous Alibaba will be replaced with Apache
loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"), true);
loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName());
loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"));
loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName());
loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"));
return extensionClasses;
}
Copy the codeAbove gets the @spi extension and specifies the file to load. Dubbo /internal (Dubbo /internal); Dubbo /internal (Dubbo /internal); Dubbo /internal (Dubbo /internal); Load the directory configuration file in loadDirectory.
private void loadDirectory(Map<String, Class<? >> extensionClasses, String dir, String type,boolean extensionLoaderClassLoaderFirst) {
/ / get in the project file path, such as: meta-inf/dubbo/top ytao. Demo. Spi. AnimalService
String fileName = dir + type;
try {
Enumeration<java.net.URL> urls = null;
ClassLoader classLoader = findClassLoader();
// Load the internal extension class
if (extensionLoaderClassLoaderFirst) {
ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader();
if (ClassLoader.getSystemClassLoader() != extensionLoaderClassLoader) {
urls = extensionLoaderClassLoader.getResources(fileName);
}
}
// Load the current fileName file
if(urls == null| |! urls.hasMoreElements()) {if(classLoader ! =null) {
urls = classLoader.getResources(fileName);
} else{ urls = ClassLoader.getSystemResources(fileName); }}if(urls ! =null) {
// Iterate over the contents of the file with the same name
while (urls.hasMoreElements()) {
java.net.URL resourceURL = urls.nextElement();
// Load the file contentsloadResource(extensionClasses, classLoader, resourceURL); }}}catch (Throwable t) {
logger.error("Exception occurred when loading extension class (interface: " +
type + ", description file: " + fileName + ").", t); }}Copy the codeLoad all files with the same name after getting the file name, and then iterate over each file, loading the contents one by one.
private void loadResource(Map
> extensionClasses, ClassLoader classLoader, java.net.URL resourceURL)
,> {
try {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(resourceURL.openStream(), StandardCharsets.UTF_8))) {
String line;
// The entire line reads the contents of the file
while((line = reader.readLine()) ! =null) {
// Get the index of the position of the first "#" in the current row
final int ci = line.indexOf(The '#');
// If there is a "#" in the current line, remove the content after "#"
if (ci >= 0) {
line = line.substring(0, ci);
}
line = line.trim();
if (line.length() > 0) {
try {
String name = null;
// Get the index of the current row "="
int i = line.indexOf('=');
// If the current line has "=", copy the values around "=" separately to name and line
if (i > 0) {
name = line.substring(0, i).trim();
line = line.substring(i + 1).trim();
}
if (line.length() > 0) {
// Load the extension class
loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name); }}catch (Throwable t) {
IllegalStateException e = new IllegalStateException("Failed to load extension class (interface: " + type + ", class line: " + line + ") in " + resourceURL + ", cause: " + t.getMessage(), t);
exceptions.put(line, e);
}
}
}
}
} catch (Throwable t) {
logger.error("Exception occurred when loading extension class (interface: " +
type + ", class file: " + resourceURL + ") in "+ resourceURL, t); }}Copy the codeThis code loads and parses the contents of the file, and then loads the extension class via loadClass.
private void loadClass(Map
> extensionClasses, java.net.URL resourceURL, Class
clazz, String name)
,> throws NoSuchMethodException {
// Check whether the current implementation class implements the Type interface
if(! type.isAssignableFrom(clazz)) {throw new IllegalStateException("Error occurred when loading extension class (interface: " +
type + ", class line: " + clazz.getName() + "), class "
+ clazz.getName() + " is not subtype of interface.");
}
// Whether the current implementation class has Adaptive annotations
if (clazz.isAnnotationPresent(Adaptive.class)) {
cacheAdaptiveClass(clazz);
// Whether the current class wraps the extension class as Wrapper
} else if (isWrapperClass(clazz)) {
cacheWrapperClass(clazz);
} else {
// Tries to see if the current class has a parameterless constructor
clazz.getConstructor();
if (StringUtils.isEmpty(name)) {
// If name is empty, get the value of clazz's @extension annotation, if the annotation value is also absent, use the lowercase class name
name = findAnnotationName(clazz);
if (name.length() == 0) {
throw new IllegalStateException("No such extension name for the class " + clazz.getName() + " in the config " + resourceURL);
}
}
String[] names = NAME_SEPARATOR.split(name);
if (ArrayUtils.isNotEmpty(names)) {
// Cache extension and the cache of @activate
cacheActivateClass(clazz, names[0]);
for (String n : names) {
// Cache the cache of extension classes and extensions
cacheName(clazz, n);
// save extensionClasses and extensions to extensionClasses extension -> extension class relational mappingsaveInExtensionClass(extensionClasses, clazz, n); }}}}Copy the codeAt this point, getExtensionClasses() loads the extension class method analysis and injectExtension() is injected into the analysis.
private T injectExtension(T instance) {
//
if (objectFactory == null) {
return instance;
}
try {
for (Method method : instance.getClass().getMethods()) {
// Iterate through all methods of the current extension class, if the current method is not a setter method,
// If the method name does not start with 'set' and the parameter is not a single one, the method access level is not public, the execution will not proceed
if(! isSetter(method)) {continue;
}
// Does the current method add annotations that do not inject dependencies
if(method.getAnnotation(DisableInject.class) ! =null) {
continue; } Class<? > pt = method.getParameterTypes()[0];
// Check whether the current parameter belongs to eight basic types or void
if (ReflectUtils.isPrimitives(pt)) {
continue;
}
try {
// Get the property name through the property setter method
String property = getSetterProperty(method);
// Get the dependent object
Object object = objectFactory.getExtension(pt, property);
if(object ! =null) {
// Set dependenciesmethod.invoke(instance, object); }}catch (Exception e) {
logger.error("Failed to inject via method " + method.getName()
+ " of interface " + type.getName() + ":"+ e.getMessage(), e); }}}catch (Exception e) {
logger.error(e.getMessage(), e);
}
return instance;
}
Copy the codeThe dependency is set by iterating through all the methods of the extension class to find the corresponding dependency and using reflection to call the settter method. The objectFactory object is shown as follows:
The corresponding dependency is found in SpiExtensionFactory or SpringExtensionFactory, and both factories are maintained in AdaptiveExtensionFactory.
@Adaptive
public class AdaptiveExtensionFactory implements ExtensionFactory {
private final List<ExtensionFactory> factories;
public AdaptiveExtensionFactory(a) {
/ /...
}
@Override
public <T> T getExtension(Class<T> type, String name) {
// Match the mapping to type->name by traversing
for (ExtensionFactory factory : factories) {
T extension = factory.getExtension(type, name);
if(extension ! =null) {
returnextension; }}return null; }}Copy the codeThe above is the simple loading process analysis of the Dubbo SPI extension class.
Adaptive loading mechanism
The extension for Dubbo makes it more flexible for an interface to be loaded without hard coding, instead loading through use, Dubbo’s other loading mechanism – adaptive loading. Adaptive loading mechanism uses @adaptive annotation:
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface Adaptive {
String[] value() default {};
}
Copy the codeThe Adaptive value is an array that can be configured with multiple keys. At initialization, all keys are iterated to match, or if none is matched to the value of @spi. When Adaptive annotations are annotated on a class, simple pairs should be implemented. If the annotation is annotated on the interface method, the code is dynamically generated based on the parameters to get the implementation of the extension point. Annotation handling on a class is relatively easy to understand, while annotation loading on a method is relatively readable. Get the extension implementation by calling ExtensionLoader#getAdaptiveExtension.
public T getAdaptiveExtension(a) {
// Get the instantiated object cache
Object instance = cachedAdaptiveInstance.get();
if (instance == null) {
if(createAdaptiveInstanceError ! =null) {
throw new IllegalStateException("Failed to create adaptive instance: " +
createAdaptiveInstanceError.toString(),
createAdaptiveInstanceError);
}
// Create an adaptive extension after double-checking the lock
synchronized (cachedAdaptiveInstance) {
instance = cachedAdaptiveInstance.get();
if (instance == null) {
try {
// Create an adaptive extension
instance = createAdaptiveExtension();
cachedAdaptiveInstance.set(instance);
} catch (Throwable t) {
createAdaptiveInstanceError = t;
throw new IllegalStateException("Failed to create adaptive instance: "+ t.toString(), t); }}}}return (T) instance;
}
private T createAdaptiveExtension(a) {
try {
// After getting the adaptive extension, inject the dependency
return injectExtension((T) getAdaptiveExtensionClass().newInstance());
} catch (Exception e) {
throw new IllegalStateException("Can't create adaptive extension " + type + ", cause: "+ e.getMessage(), e); }}Copy the codeThe above code completes whether the extension class object exists in the cache. If not, it is set in the instantiated adaptive extension object by creating an adaptive extension and injecting the instance into the dependency. The getAdaptiveExtensionClass is the core process.
privateClass<? > getAdaptiveExtensionClass() {// Load all extension classes
getExtensionClasses();
// cachedAdaptiveClass must not be empty if the @adaptive annotation exists after all extended classes are loaded
if(cachedAdaptiveClass ! =null) {
return cachedAdaptiveClass;
}
// Create an adaptive extension class
return cachedAdaptiveClass = createAdaptiveExtensionClass();
}
privateClass<? > createAdaptiveExtensionClass() {// Generate adaptive extension code
String code = new AdaptiveClassCodeGenerator(type, cachedDefaultName).generate();
// Get the extension class loader
ClassLoader classLoader = findClassLoader();
// Get the implementation class of the compiler type
org.apache.dubbo.common.compiler.Compiler compiler = ExtensionLoader.getExtensionLoader(org.apache.dubbo.common.compiler.Compiler.class).getAdaptiveExtension();
// Compile the code to return the object
return compiler.compile(code, classLoader);
}
Copy the codeThe main work done here is to load all extension classes, which represent the implementation classes of all extension interface classes. During the loading process, classes annotated with @Adaptive will be saved into cachedAdaptiveClass. Get the extension class instantiation object by automatically generating adaptive extension code and compiling it.
Dynamically generate code in the generate method:
public String generate(a) {
// Check whether there are Adaptive annotations on the methods of the current extension interface
if(! hasAdaptiveMethod()) {throw new IllegalStateException("No adaptive method exist on extension " + type.getName() + ", refuse to create the adaptive class!");
}
// Generate code
StringBuilder code = new StringBuilder();
// Generate the package name of the class
code.append(generatePackageInfo());
// Generate a dependent class for the class
code.append(generateImports());
// Generate class declaration information
code.append(generateClassDeclaration());
// Generate method
Method[] methods = type.getMethods();
for (Method method : methods) {
code.append(generateMethod(method));
}
code.append("}");
if (logger.isDebugEnabled()) {
logger.debug(code.toString());
}
return code.toString();
}
Copy the codeThe above is the method to generate class information. The generation design principle is to replace the template that has been set and generate classes. The specific information is not a lot of code, but it is relatively easy to read. The adaptive loading mechanism has been briefly analyzed. At first glance, it is very complex, but it is relatively easy to understand if you understand the overall structure and process, and then go into detail.
conclusion
From the point of view of Dubbo design, its good scalability, a more important point is to benefit from Dubbo SPI loading mechanism. In the study of its design concept, the expansibility of coding thinking also has certain inspiration.
Personal blog: Ytao.top
Pay attention to the public number [Ytao], more original good articles
