Introduction to the
In the previous few articles, we have run the core functions of ShardingSphere based on the source code. At the beginning of this article, we begin to explore the source code to see how ShardingSphere works
An overview of
Before we begin, let’s consider the mystery of this quest:
- 1. How does ShardingSphere load the multiple database sources we configured?
- 2. How does ShardingSphere write statements to different data sources?
Explore with questions. You may not get answers, but at least you have a direction
Based on our JDBC exploration article, click on the relevant breakpoint and start our exploration journey
- ShardingSphere JDBC sub-database sub-table read-write separation data encryption
The source code to explore
1. Find the ShardingSphere JDBC entry
Put breakpoints on init and process in the code below and see if you can find any clues through the program call stack
public final class ExampleExecuteTemplate {
public static void run(final ExampleService exampleService) throws SQLException {
try {
exampleService.initEnvironment();
exampleService.processSuccess();
} finally{ exampleService.cleanEnvironment(); }}}Copy the codeExampleService. InitEnvironment () come in later, the call stack is empty, what also have no, it looks like we’re the first to explore if the load multiple configuration data for the purpose of fell through
However, I continued to Debug, but it was all Ibatis related, and there was no code related to ShardingSphere. I comforted myself that it was normal
Continue to explore: exampleService processSuccess () to the following statements related to insert, query and other related operations, the insertData () on the breakpoint
public class OrderServiceImpl implements ExampleService {
@Override
@Transactional
public void processSuccess(a) throws SQLException {
System.out.println("-------------- Process Success Begin ---------------");
List<Long> orderIds = insertData();
printData();
deleteData(orderIds);
printData();
System.out.println("-------------- Process Success Finish --------------"); }}Copy the codeBased on previous attempts that didn’t work, I had to tread carefully and follow function calls further down the line
Mapperproxy. class, execute the following method to proceed:
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
try {
if (Object.class.equals(method.getDeclaringClass())) {
return method.invoke(this, args);
}
if (this.isDefaultMethod(method)) {
return this.invokeDefaultMethod(proxy, method, args); }}catch (Throwable var5) {
throw ExceptionUtil.unwrapThrowable(var5);
}
MapperMethod mapperMethod = this.cachedMapperMethod(method);
// Enter from here
return mapperMethod.execute(this.sqlSession, args);
}
Copy the codeLet’s go to the following function
public Object execute(SqlSession sqlSession, Object[] args) {
Object result;
Object param;
switch(this.command.getType()) {
case INSERT:
param = this.method.convertArgsToSqlCommandParam(args);
// Enter from here and trace the insert method
result = this.rowCountResult(sqlSession.insert(this.command.getName(), param));
break;
case UPDATE:
......
}
Copy the codeSqlSession = SqlSession; SqlSession = SqlSession; SqlSession = SqlSession;
Database name is logic_DB, think of Teacher Qin said ShardingSphere UI can also use JDBC, but its name is logic_DB
There’s some speculation at this point, but we need to keep tracking and confirm
Sqlsessiontemplate. class, which reobtains a sqlSession, but is still logic_db
private class SqlSessionInterceptor implements InvocationHandler {
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
SqlSession sqlSession = SqlSessionUtils.getSqlSession(SqlSessionTemplate.this.sqlSessionFactory, SqlSessionTemplate.this.executorType, SqlSessionTemplate.this.exceptionTranslator);
Object unwrapped;
try {
// Continue here
Object result = method.invoke(sqlSession, args);
if(! SqlSessionUtils.isSqlSessionTransactional(sqlSession, SqlSessionTemplate.this.sqlSessionFactory)) {
sqlSession.commit(true);
}
unwrapped = result;
} catch (Throwable var11) {
......
} finally{... }returnunwrapped; }}Copy the codeAnd met a lot of update, we’re going down, finally we come to: PreparedStatementHandler. Class
public class PreparedStatementHandler extends BaseStatementHandler {
public int update(Statement statement) throws SQLException {
PreparedStatement ps = (PreparedStatement)statement;
// Execution is critical here
ps.execute();
int rows = ps.getUpdateCount();
Object parameterObject = this.boundSql.getParameterObject();
KeyGenerator keyGenerator = this.mappedStatement.getKeyGenerator();
keyGenerator.processAfter(this.executor, this.mappedStatement, ps, parameterObject);
returnrows; }}Copy the codeIn the above: ps. The execute (), we to go on, surprise came, bang, came to ShardingSphere own writing class: ShardingSpherePreparedStatement. Java
Here are some thoughts on ShardingSphere JDBC:
ShardingSphere JDBC feels in fact there is a middleware in essence, embedded, is also between the program and the database, in the local exploration, it is a logic_DB, the most downstream of Mybatis, all database access truncation processing
I have such a preliminary experience, and there may be more discoveries with the further research. Here we find the entrance of ShardingSphere
2. Initially explore the ShardingSphere JDBC processing path
We came to the entrance, find: on ShardingSpherePreparedStatement. Java
@Override
public boolean execute(a) throws SQLException {
try {
clearPrevious();
// Here are the actual database sources and statements
executionContext = createExecutionContext();
if (metaDataContexts.getMetaData(connection.getSchemaName()).getRuleMetaData().getRules().stream().anyMatch(each -> each instanceof RawExecutionRule)) {
// TODO process getStatement
Collection<ExecuteResult> executeResults = rawExecutor.execute(createRawExecutionGroupContext(), executionContext.getLogicSQL(), new RawSQLExecutorCallback());
return executeResults.iterator().next() instanceof QueryResult;
}
if (executionContext.getRouteContext().isFederated()) {
List<QueryResult> queryResults = executeFederatedQuery();
return! queryResults.isEmpty(); }// Continue tracing
ExecutionGroupContext<JDBCExecutionUnit> executionGroupContext = createExecutionGroupContext();
cacheStatements(executionGroupContext.getInputGroups());
return driverJDBCExecutor.execute(executionGroupContext,
executionContext.getLogicSQL(), executionContext.getRouteContext().getRouteUnits(), createExecuteCallback());
} finally{ clearBatch(); }}Copy the codeExecutionContext = createExecutionContext() has the original statement, or we timed the actual data source and SQL statement, as shown below. Let’s continue:
To: AbstractExecutionPrepareEngine. Java, actually have been able to get in front of the display the execution of the statement, it is less clear what is the purpose of this step
public final ExecutionGroupContext<T> prepare(final RouteContext routeContext, final Collection<ExecutionUnit> executionUnits) throws SQLException {
Collection<ExecutionGroup<T>> result = new LinkedList<>();
for (Entry<String, List<SQLUnit>> entry : aggregateSQLUnitGroups(executionUnits).entrySet()) {
String dataSourceName = entry.getKey();
List<SQLUnit> sqlUnits = entry.getValue();
List<List<SQLUnit>> sqlUnitGroups = group(sqlUnits);
ConnectionMode connectionMode = maxConnectionsSizePerQuery < sqlUnits.size() ? ConnectionMode.CONNECTION_STRICTLY : ConnectionMode.MEMORY_STRICTLY;
result.addAll(group(dataSourceName, sqlUnitGroups, connectionMode));
}
return decorate(routeContext, result);
}
Copy the codeBack to ShardingSpherePreparedStatement. Java, tracking driverJDBCExecutor. Execute, came to: driverJDBCExecutor. Java
public boolean execute(final ExecutionGroupContext<JDBCExecutionUnit> executionGroupContext, final LogicSQL logicSQL,
final Collection<RouteUnit> routeUnits, final JDBCExecutorCallback<Boolean> callback) throws SQLException {
try {
ExecuteProcessEngine.initialize(logicSQL, executionGroupContext, metaDataContexts.getProps());
List<Boolean> results = jdbcLockEngine.execute(executionGroupContext, logicSQL.getSqlStatementContext(), routeUnits, callback);
boolean result = null! = results && ! results.isEmpty() &&null! = results.get(0) && results.get(0);
ExecuteProcessEngine.finish(executionGroupContext.getExecutionID());
return result;
} finally{ ExecuteProcessEngine.clean(); }}Copy the codeThe interesting thing is whether jdbcLockEngine or logic_db keeps track of the execute function, all the way down to executorengae.java
public <I, O> List<O> execute(final ExecutionGroupContext<I> executionGroupContext,
final ExecutorCallback<I, O> firstCallback, final ExecutorCallback<I, O> callback, final boolean serial) throws SQLException {
if (executionGroupContext.getInputGroups().isEmpty()) {
return Collections.emptyList();
}
return serial ? serialExecute(executionGroupContext.getInputGroups().iterator(), firstCallback, callback)
: parallelExecute(executionGroupContext.getInputGroups().iterator(), firstCallback, callback);
}
private <I, O> List<O> serialExecute(final Iterator<ExecutionGroup<I>> executionGroups, final ExecutorCallback<I, O> firstCallback, final ExecutorCallback<I, O> callback) throws SQLException {
ExecutionGroup<I> firstInputs = executionGroups.next();
List<O> result = new LinkedList<>(syncExecute(firstInputs, null == firstCallback ? callback : firstCallback));
while (executionGroups.hasNext()) {
// Continue tracing
result.addAll(syncExecute(executionGroups.next(), callback));
}
return result;
}
Copy the codeTracking all the way down to: JDBCExecutorCallback. Java
public final Collection<T> execute(final Collection<JDBCExecutionUnit> executionUnits, final boolean isTrunkThread, final Map<String, Object> dataMap) throws SQLException {
// TODO It is better to judge whether need sane result before execute, can avoid exception thrown
Collection<T> result = new LinkedList<>();
for (JDBCExecutionUnit each : executionUnits) {
T executeResult = execute(each, isTrunkThread, dataMap);
if (null != executeResult) {
result.add(executeResult);
}
}
return result;
}
private T execute(final JDBCExecutionUnit jdbcExecutionUnit, final boolean isTrunkThread, final Map<String, Object> dataMap) throws SQLException {
SQLExecutorExceptionHandler.setExceptionThrown(isExceptionThrown);
// Here we get our real database source
DataSourceMetaData dataSourceMetaData = getDataSourceMetaData(jdbcExecutionUnit.getStorageResource().getConnection().getMetaData());
SQLExecutionHook sqlExecutionHook = new SPISQLExecutionHook();
try {
SQLUnit sqlUnit = jdbcExecutionUnit.getExecutionUnit().getSqlUnit();
sqlExecutionHook.start(jdbcExecutionUnit.getExecutionUnit().getDataSourceName(), sqlUnit.getSql(), sqlUnit.getParameters(), dataSourceMetaData, isTrunkThread, dataMap);
// Continue tracing the execution statement
T result = executeSQL(sqlUnit.getSql(), jdbcExecutionUnit.getStorageResource(), jdbcExecutionUnit.getConnectionMode());
sqlExecutionHook.finishSuccess();
finishReport(dataMap, jdbcExecutionUnit);
return result;
} catch (finalSQLException ex) { ...... }}Copy the codeIn the code above, find the code related to directly fetching the real database source
We continue to monitor on the relevant code: ShardingSpherePreparedStatement. Java, come
private JDBCExecutorCallback<Boolean> createExecuteCallback(a) {
boolean isExceptionThrown = SQLExecutorExceptionHandler.isExceptionThrown();
return new JDBCExecutorCallback<Boolean>(metaDataContexts.getMetaData(connection.getSchemaName()).getResource().getDatabaseType(), sqlStatement, isExceptionThrown) {
@Override
protected Boolean executeSQL(final String sql, final Statement statement, final ConnectionMode connectionMode) throws SQLException {
return ((PreparedStatement) statement).execute();
}
@Override
protected Optional<Boolean> getSaneResult(final SQLStatement sqlStatement) {
returnOptional.empty(); }}; }Copy the codeIf you use native JDBC to write your own database, you should compare the data
conclusion
Although the relevant database source parsing and how to hit the corresponding part of the database has not been explored, but we have located its general location, can continue to study later
ShardingSphere JDBC in this exploration, feel is an embedded Proxy, its database name fixed logic_DB, truncated Mybatis and the real database link, the current middleman, roughly as follows:
