ShardingSphere SQLToken generation exploration

Introduction to the

In the previous article, we explored the parsing process of logical SQL into real SQL in ShardingSphere, where SQLToken plays a key role. In this article, we will explore the generation of SQLToken

Source code analysis

Based on the above:

• Analysis and generation of ShardingSphere statements

Let’s move on to SQLToken generation

First, locate the key parts of code generated by SQLToken:

Then, to the core:

INSERT INTO t_order (user_id, address_id, status) VALUES (?, ?, ?)

public final class SQLTokenGenerators {
    
    /**
     * Generate SQL tokens.
     *
     * @param sqlStatementContext SQL statement context
     * @param parameters SQL parameters
     * @param schema sShardingSphere schema
     * @return SQL tokens
     */
    @SuppressWarnings("unchecked")
    public List<SQLToken> generateSQLTokens(final SQLStatementContext sqlStatementContext, final List<Object> parameters, final ShardingSphereSchema schema) {
        List<SQLToken> result = new LinkedList<>();
	// Iterate over all sqlTokenGenerators
        for (SQLTokenGenerator each : sqlTokenGenerators) {
            setUpSQLTokenGenerator(each, parameters, schema, result);
	    // This should be similar to whether the condition is met
            if(! each.isGenerateSQLToken(sqlStatementContext)) {continue;
            }
	    // Create a Token
            if (each instanceof OptionalSQLTokenGenerator) {
                SQLToken sqlToken = ((OptionalSQLTokenGenerator) each).generateSQLToken(sqlStatementContext);
                if (!result.contains(sqlToken)) {
                    result.add(sqlToken);
                }
            } else if (each instanceofCollectionSQLTokenGenerator) { result.addAll(((CollectionSQLTokenGenerator) each).generateSQLTokens(sqlStatementContext)); }}returnresult; }}Copy the code

All sqlTokenGenerators are shown below:

SetUpSQLTokenGenerator (each, parameters, schema, result);

public final class SQLTokenGenerators {
    
    private void setUpSQLTokenGenerator(final SQLTokenGenerator sqlTokenGenerator, final List<Object> parameters, final ShardingSphereSchema schema, final List<SQLToken> previousSQLTokens) {
        if (sqlTokenGenerator instanceof ParametersAware) {
            ((ParametersAware) sqlTokenGenerator).setParameters(parameters);
        }
        if (sqlTokenGenerator instanceof SchemaMetaDataAware) {
            ((SchemaMetaDataAware) sqlTokenGenerator).setSchema(schema);
        }
        if (sqlTokenGenerator instanceofPreviousSQLTokensAware) { ((PreviousSQLTokensAware) sqlTokenGenerator).setPreviousSQLTokens(previousSQLTokens); }}}Copy the code

Seeing that there are three kinds of Aware at present, I am not quite sure what their general functions are. But without returning ahead of time, will there be multiple matches at the same time? Forget it for now. Keep going

RemoveTokenGenerator

Debug sqlStatementContext (sqlStatementContext); remove (sqlStatementContext)

After the operation, return false

public final class RemoveTokenGenerator implements CollectionSQLTokenGenerator<SQLStatementContext<? >>{
    
    @Override
    public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
	SqlStatementContext sqlStatementContext sqlStatementContext
        boolean containsRemoveSegment = false;
        if (sqlStatementContext instanceofRemoveAvailable) { containsRemoveSegment = ! ((RemoveAvailable) sqlStatementContext).getRemoveSegments().isEmpty(); }boolean containsSchemaName = false;
        if (sqlStatementContext instanceof TableAvailable) {
            containsSchemaName = ((TableAvailable) sqlStatementContext).getTablesContext().getSchemaName().isPresent();
        }
        returncontainsRemoveSegment || containsSchemaName; }}Copy the code

TableTokenGenerator

This default to return true, and walking is CollectionSQLTokenGenerator branch, then we will interface to look on the token generated:

@Setter
public final class TableTokenGenerator implements CollectionSQLTokenGenerator.ShardingRuleAware {
    
    private ShardingRule shardingRule;
    
    @Override
    public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
        return true;
    }
    
    // It needs to be evaluated before it can be generated
    @Override
    public Collection<TableToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
        return sqlStatementContext instanceof TableAvailable ? generateSQLTokens((TableAvailable) sqlStatementContext) : Collections.emptyList();
    }
    
    private Collection<TableToken> generateSQLTokens(final TableAvailable sqlStatementContext) {
        Collection<TableToken> result = new LinkedList<>();
	// Get the table stuff. This type is InsertStatementContext, so get getOriginalTables
        for (SimpleTableSegment each : sqlStatementContext.getAllTables()) {
	    // Check if any rules match
            if (shardingRule.findTableRule(each.getTableName().getIdentifier().getValue()).isPresent()) {
		// If the token matches, the token is added
                result.add(newTableToken(each.getTableName().getStartIndex(), each.getTableName().getStopIndex(), each, (SQLStatementContext) sqlStatementContext, shardingRule)); }}returnresult; }}Copy the code

From the above code, we can see that a TableToken is generated, and the key variables are as follows:

In the preceding command, only OriginalTables, tables, and shardingRule are traversed. After the conditions are met, the generation token is added directly

IndexTokenGenerator

This one just doesn’t match

public final class IndexTokenGenerator implements CollectionSQLTokenGenerator.ShardingRuleAware.SchemaMetaDataAware {
    
    @Override
    public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
        return sqlStatementContext instanceof IndexAvailable && !((IndexAvailable) sqlStatementContext).getIndexes().isEmpty();
    }
}
Copy the code

ConstraintTokenGenerator

This one doesn’t match either

public final class ConstraintTokenGenerator implements CollectionSQLTokenGenerator.ShardingRuleAware {
    
    @Override
    public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
        return sqlStatementContext instanceof ConstraintAvailable && !((ConstraintAvailable) sqlStatementContext).getConstraints().isEmpty();
    }
}
Copy the code

GeneratedKeyInsertColumnTokenGenerator — BaseGeneratedKeyTokenGenerator

First go to the abstract class, a wave of judgment, successful match

public abstract class BaseGeneratedKeyTokenGenerator implements OptionalSQLTokenGenerator<InsertStatementContext> {
    
    @Override
    public final boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
	// serial triple judgment
        return sqlStatementContext instanceof InsertStatementContext && ((InsertStatementContext) sqlStatementContext).getGeneratedKeyContext().isPresent()
                && ((InsertStatementContext) sqlStatementContext).getGeneratedKeyContext().get().isGenerated() && isGenerateSQLToken((InsertStatementContext) sqlStatementContext);
    }
    
    protected abstract boolean isGenerateSQLToken(InsertStatementContext insertStatementContext);
}
Copy the code

Then comes the subclass, which also makes a wave of judgments and matches successfully

public final class GeneratedKeyInsertColumnTokenGenerator extends BaseGeneratedKeyTokenGenerator {
    
    @Override
    protected boolean isGenerateSQLToken(final InsertStatementContext insertStatementContext) {
        Optional<InsertColumnsSegment> sqlSegment = insertStatementContext.getSqlStatement().getInsertColumns();
        returnsqlSegment.isPresent() && ! sqlSegment.get().getColumns().isEmpty() && insertStatementContext.getGeneratedKeyContext().isPresent() && ! insertStatementContext.getGeneratedKeyContext().get().getGeneratedValues().isEmpty(); }}Copy the code

The contents of insertStatementContext are shown below:

After successful match, go OptionalSQLTokenGenerator branch:

public final class GeneratedKeyInsertColumnTokenGenerator extends BaseGeneratedKeyTokenGenerator {
    @Override
    public GeneratedKeyInsertColumnToken generateSQLToken(final InsertStatementContext insertStatementContext) {
        Optional<GeneratedKeyContext> generatedKey = insertStatementContext.getGeneratedKeyContext();
        Preconditions.checkState(generatedKey.isPresent());
        Optional<InsertColumnsSegment> sqlSegment = insertStatementContext.getSqlStatement().getInsertColumns();
        Preconditions.checkState(sqlSegment.isPresent());
        return newGeneratedKeyInsertColumnToken(sqlSegment.get().getStopIndex(), generatedKey.get().getColumnName()); }}Copy the code

If false, IllegalStateException will be raised. If false, IllegalStateException will be raised. If false, IllegalStateException will be raised

Finally, the Token is generated, which uses only the start and end indexes

GeneratedKeyForUseDefaultInsertColumnsTokenGenerator — BaseGeneratedKeyTokenGenerator

The match failed

public final class GeneratedKeyForUseDefaultInsertColumnsTokenGenerator extends BaseGeneratedKeyTokenGenerator {
    
    @Override
    protected boolean isGenerateSQLToken(final InsertStatementContext insertStatementContext) {
        returninsertStatementContext.useDefaultColumns(); }}Copy the code

ShardingInsertValuesTokenGenerator

Through the matching, and walk into branches: OptionalSQLTokenGenerator

public final class ShardingInsertValuesTokenGenerator implements OptionalSQLTokenGenerator<InsertStatementContext>, RouteContextAware {
    
    private RouteContext routeContext;
    
    @Override
    public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
        return sqlStatementContext instanceofInsertStatementContext && ! (((InsertStatementContext) sqlStatementContext).getSqlStatement()).getValues().isEmpty(); }@Override
    public InsertValuesToken generateSQLToken(final InsertStatementContext insertStatementContext) {
        Collection<InsertValuesSegment> insertValuesSegments = (insertStatementContext.getSqlStatement()).getValues();
	// Select the corresponding maximum and minimum index
        InsertValuesToken result = new ShardingInsertValuesToken(getStartIndex(insertValuesSegments), getStopIndex(insertValuesSegments));
        Iterator<Collection<DataNode>> originalDataNodesIterator = null == routeContext || routeContext.getOriginalDataNodes().isEmpty()
                ? null : routeContext.getOriginalDataNodes().iterator();
	// Add values (); // Add values ()
        for (InsertValueContext each : insertStatementContext.getInsertValueContexts()) {
            List<ExpressionSegment> expressionSegments = each.getValueExpressions();
            Collection<DataNode> dataNodes = null == originalDataNodesIterator ? Collections.emptyList() : originalDataNodesIterator.next();
            result.getInsertValues().add(new ShardingInsertValue(expressionSegments, dataNodes));
        }
        return result;
    }
    
    // take the minimum value
    private int getStartIndex(final Collection<InsertValuesSegment> segments) {
        int result = segments.iterator().next().getStartIndex();
        for (InsertValuesSegment each : segments) {
            result = Math.min(result, each.getStartIndex());
        }
        return result;
    }
    
    // take the maximum value
    private int getStopIndex(final Collection<InsertValuesSegment> segments) {
        int result = segments.iterator().next().getStopIndex();
        for (InsertValuesSegment each : segments) {
            result = Math.max(result, each.getStopIndex());
        }
        returnresult; }}Copy the code

SQL > create token (‘ value ‘, ‘token’);

GeneratedKeyInsertValuesTokenGenerator

Seems to be associated with ShardingInsertValuesTokenGenerator, now don’t know much about

public final class GeneratedKeyInsertValuesTokenGenerator extends BaseGeneratedKeyTokenGenerator implements PreviousSQLTokensAware {
    
    private List<SQLToken> previousSQLTokens;
    
    @Override
    protected boolean isGenerateSQLToken(final InsertStatementContext insertStatementContext) {
        return! insertStatementContext.getSqlStatement().getValues().isEmpty() && insertStatementContext.getGeneratedKeyContext().isPresent() && ! insertStatementContext.getGeneratedKeyContext().get().getGeneratedValues().isEmpty(); }@Override
    public SQLToken generateSQLToken(final InsertStatementContext insertStatementContext) { Optional<InsertValuesToken> result = findPreviousSQLToken(); Preconditions.checkState(result.isPresent()); Optional<GeneratedKeyContext> generatedKey = insertStatementContext.getGeneratedKeyContext(); Preconditions.checkState(generatedKey.isPresent()); Iterator<Comparable<? >> generatedValues = generatedKey.get().getGeneratedValues().iterator();int count = 0;
        List<List<Object>> parameters = insertStatementContext.getGroupedParameters();
        for (InsertValueContext each : insertStatementContext.getInsertValueContexts()) {
            InsertValue insertValueToken = result.get().getInsertValues().get(count);
            DerivedSimpleExpressionSegment expressionSegment = isToAddDerivedLiteralExpression(parameters, count)
                    ? new DerivedLiteralExpressionSegment(generatedValues.next()) : new DerivedParameterMarkerExpressionSegment(each.getParameterCount());
            insertValueToken.getValues().add(expressionSegment);
            count++;
        }
        return result.get();
    }
    
    private Optional<InsertValuesToken> findPreviousSQLToken(a) {
        for (SQLToken each : previousSQLTokens) {
            if (each instanceof InsertValuesToken) {
                returnOptional.of((InsertValuesToken) each); }}return Optional.empty();
    }
    
    private boolean isToAddDerivedLiteralExpression(final List<List<Object>> parameters, final int insertValueCount) {
        returnparameters.get(insertValueCount).isEmpty(); }}Copy the code

After the above Token is generated, the Token that is not added to the Token is eventually generated with one more thing than the previous one, as shown below:

Token upload End

This is the last one, and the final result is as follows:

conclusion

SQLToken generates SQLTokenGenerate. As in the previous article, Index is the key, but in this part of the analysis, Index is actually brought down, as shown in the following figure:

At present, the generation of Token is to judge the type and then get the corresponding data

And then, index, this important raw information, is taken from LogicSQL, and we’re going to follow this code, and see if we can connect these three parts together