Spring Dynamic data source

What is a dynamic data source? What does it solve??

In real-world development, it is common to use multiple data sources in the same project. For example, a project with read/write separation has master and read data sources.

Dynamic data sources automatically control which data source a certain piece of data manipulation logic goes to through Spring configuration. For an example of read/write separation, the project references two data sources, master and slave. Spring configures or extends the ability to use slave data sources automatically if a query method is invoked in an interface.

This effect can generally be achieved by:

  • Use the @mapperscan annotation to specify that all methods in a package go to a fixed data source (this is more rigid, resulting in redundant code, but can also be used as a temporary solution);

  • Using annotations + AOP + AbstractRoutingDataSource to specify in the form of a method of database operations is that the data source.

  • Through the Sharding-JDBC component (external dependencies need to be introduced, which is recommended if the project references the component itself)

    <hr>
Copy the code

Key core class[Obtain information]

Here introduces through annotations + AOP + AbstractRoutingDataSource linkage to realize the dynamic data source.

Is the starting point of all AbstractRoutingDataSource this class that implements the DataSource interface

public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBean {

    // .... 省略 ... 
    
    @Nullable
	private Map<Object, Object> targetDataSources;
	
	@Nullable
	private Map<Object, DataSource> resolvedDataSources;


	public void setTargetDataSources(Map<Object, Object> targetDataSources) {
		this.targetDataSources = targetDataSources;
	}
	
	public void setDefaultTargetDataSource(Object defaultTargetDataSource) {
		this.defaultTargetDataSource = defaultTargetDataSource;
	}
	
	@Override
	public void afterPropertiesSet(a) {
	    
	    // Initialize targetDataSources, resolvedDataSources
		if (this.targetDataSources == null) {
			throw new IllegalArgumentException("Property 'targetDataSources' is required");
		}
		this.resolvedDataSources = new HashMap<>(this.targetDataSources.size());
		this.targetDataSources.forEach((key, value) -> {
			Object lookupKey = resolveSpecifiedLookupKey(key);
			DataSource dataSource = resolveSpecifiedDataSource(value);
			this.resolvedDataSources.put(lookupKey, dataSource);
		});// Join Java development chat
		if (this.defaultTargetDataSource ! =null) {
			this.resolvedDefaultDataSource = resolveSpecifiedDataSource(this.defaultTargetDataSource); }}@Override
	public Connection getConnection(a) throws SQLException {
		return determineTargetDataSource().getConnection();
	}

	@Override
	public Connection getConnection(String username, String password) throws SQLException {
		return determineTargetDataSource().getConnection(username, password);
	}


	/**
	 * Retrieve the current target DataSource. Determines the
	 * {@link #determineCurrentLookupKey() current lookup key}, performs
	 * a lookup in the {@link #setTargetDataSources targetDataSources} map,
	 * falls back to the specified
	 * {@link #setDefaultTargetDataSource default target DataSource} if necessary.
	 * @see #determineCurrentLookupKey()
	 */
	protected DataSource determineTargetDataSource(a) {
		Assert.notNull(this.resolvedDataSources, "DataSource router not initialized");
	
	    // @1 start
		Object lookupKey = determineCurrentLookupKey();
		DataSource dataSource = this.resolvedDataSources.get(lookupKey);
		// @1 end
		
		if (dataSource == null && (this.lenientFallback || lookupKey == null)) {
			dataSource = this.resolvedDefaultDataSource;
		}
		if (dataSource == null) {
			throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]");
		}
		return dataSource;
	}

	/** * returns a key that is used to retrieve the data source object from the resolvedDataSources data source. See */
	@Nullable
	protected abstract Object determineCurrentLookupKey(a);

}
Copy the code

Can see a extensible abstraction method in AbstractRoutingDataSource determineCurrentLookupKey (), using this method can realize the dynamic data source.

Write a simple dynamic data source component from zero

From the last part we know can achieve AbstractRoutingDataSource determineCurrentLookupKey () method of the dynamic setting a key, We then set our DataSource Map in the configuration class using the setTargetDataSources() method.

Annotation, constant definition


/** * @author axin * @summary Dynamic data source annotation definition */
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface MyDS {
    String value(a) default "default";
}

/** * @author axin * @summary Dynamic data source constant */
publicInterface DSConst {String Default ="default"; String = main library"master"; String = from the library"slave"; A String of statistical ="stat";
}

Copy the code
/** * @author axin * @summary dynamic data source ThreadLocal tool */
public class DynamicDataSourceHolder {
     // Join Java development chat
    // Save the DataSource specified by the current thread
    private static final ThreadLocal<String> THREAD_DATA_SOURCE = new ThreadLocal<>();

    public static String getDataSource(a) {
        return THREAD_DATA_SOURCE.get();
    }

    public static void setDataSource(String dataSource) {
        THREAD_DATA_SOURCE.set(dataSource);
    }

    public static void removeDataSource(a) { THREAD_DATA_SOURCE.remove(); }}Copy the code

A custom AbstractRoutingDataSource class

/** * @author axin * @summary dynamic data source */
public class DynamicDataSource extends AbstractRoutingDataSource {

    /** * Get the target data source key * @return */ from the data source
    @Override
    protected Object determineCurrentLookupKey(a) {
        // Get the key from ThreadLocal
        String dataSourceKey = DynamicDataSourceHolder.getDataSource();
        if (StringUtils.isEmpty(dataSourceKey)) {
            returnDSConst. The default; }returndataSourceKey; }}Copy the code

AOP implementations

/** * @author axin * @summary Data source switch AOP */
@Slf4j
@Aspect
@Service
public class DynamicDataSourceAOP {

    public DynamicDataSourceAOP(a) {
        log.info("/ * -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - * /");
        log.info("/ * -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - * /");
        log.info("/*---------- dynamic data source initialization... -- -- -- -- -- -- -- -- -- - * /");
        log.info("/ * -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - * /");
        log.info("/ * -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - * /");
    }

    /** ** cut point */
    @Pointcut(value = "@annotation(xxx.xxx.MyDS)")
    private void method(a){}

    /** * switch to the specified data source * @param point */ before executing the method
    @Before("method()")
    public void before(JoinPoint point) {
        MethodSignature methodSignature = (MethodSignature) point.getSignature();
        // Get the proxied method object
        Method targetMethod = methodSignature.getMethod();
        // Get the annotation information for the proxied method
        CultureDS cultureDS = AnnotationUtils.findAnnotation(targetMethod, CultureDS.class);

        // Dynamic data source annotations at the outermost layer of the method chain have the highest priority
        // Join Java development chat
        String key = DynamicDataSourceHolder.getDataSource();

        if(! StringUtils.isEmpty(key)) {log.warn("Warning: overwrite scenario appears on dynamic data source annotation call chain, please make sure there is no problem");
            return;
        }

        if(cultureDS ! = null ) {// Set the database flagDynamicDataSourceHolder.setDataSource(MyDS.value()); }}/** * release data source */
    @AfterReturning("method()")
    public void doAfter(a) { DynamicDataSourceHolder.removeDataSource(); }}Copy the code

DataSourceConfig configuration

Configure dynamic data sources into SqlSession with the following code

/** * sqlsession; There is no use mybatis annotation * @ Configuration @ EnableTransactionManagement @ EnableAspectJAutoProxy class DataSourceConfig { /** read-write SQL Session */
    public static final String BEANNAME_SQLSESSION_COMMON = "sqlsessionCommon";
    /** Specifies the name of the transaction manager. If there are multiple transaction managers, specify beanName */
    public static final String BEANNAME_TRANSACTION_MANAGER = "transactionManager";

    /** primary DataSource, must be configured. Spring starts to initialize data (whether it is needed or not)
    @Bean
    @Primary
    @ConfigurationProperties(prefix = "datasource.common")
    public DataSource datasourceCommon(a) {
        // Data source configuration can be changed to another implementation
        return DataSourceBuilder.create().build();
    }

    /** * dynamic data source * @returnr */
    @Bean
    public DynamicDataSource dynamicDataSource(a) {
        DynamicDataSource dynamicDataSource = newDynamicDataSource(); LinkedHashMap<Object, Object> hashMap = Maps.newLinkedHashMap(); Hashmap.put (dsconconst. Default, datasourceCommon()); Hashmap.put (dsconst. main library, datasourceCommon()); Hashmap.put (dsconconst. Slave library, datasourceReadOnly()); Hashmap.put (dsconconst. Statistics, datasourceStat());// Initialize the data source Map
        dynamicDataSource.setTargetDataSources(hashMap);
        dynamicDataSource.setDefaultTargetDataSource(datasourceCommon());
        return dynamicDataSource;
    }

    /** * Configure the transaction manager */
    @Primary
    @Bean(name = BEANNAME_TRANSACTION_MANAGER)
    public DataSourceTransactionManager createDataSourceTransactionManager2(a) {
        DataSource dataSource = this.dynamicDataSource();
        DataSourceTransactionManager manager = new DataSourceTransactionManager(dataSource);
        return manager;
    }

    /** * Configure read and write SQLSession */
    @Primary
    @Bean(name = BEANNAME_SQLSESSION_COMMON)
    public SqlSession readWriteSqlSession(a) throws Exception {
        SqlSessionFactoryBean factory = new SqlSessionFactoryBean();
      	// Join Java development chat
        // Set the dynamic data source
        factory.setDataSource(this.dynamicDataSource());
        PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
        factory.setConfigLocation(resolver.getResource("mybatis/mybatis-config.xml"));
        factory.setMapperLocations(resolver.getResources("mybatis/mappers/**/*.xml"));
        return newSqlSessionTemplate(factory.getObject()); }}Copy the code

conclusion

To sum up, you implement a simple Spring dynamic data source feature using AOP+ annotations that simply add the @myds annotation to the target method. Many open source components add an extension to the existing functionality, such as routing policies, and so on. [Obtain information]

By the way, sharding-JDBC implementation, update write class SQL automatically go to the master library, query class automatically go to the library, if the new project has no historical debt, is able to use this scheme. If you are working on a legacy project, then if you are using sharding-JDBC read-write separation, you will have to tease out SQL calls in your existing code logic to avoid the business impact of data inconsistencies caused by master-slave latency.

Inconsistent read data due to master/slave delay means that the master/slave synchronization has a certain delay time. The delay value is always in milliseconds regardless of the network situation. At this time, if sharding-JDBC is used for read and write separation processing, real-time data insertion and query judgment, judgment exceptions will occur. 【 References 】

In the end, I wish you all success as soon as possible, get satisfactory offer, fast promotion and salary increase, and walk on the peak of life.

If you can please give me a three support me yo 🧐🧐🧐[Obtain information]