The article directories

• One, foreword
• Ii. Data Sources (MyBatis three data sources and three data source factories)
• • 2.1 Three data sources of MyBatis
  • 2.2 Create data source factory and data source when initializing Mybatis
  • 2.3 DataSource delays creating a Connection object until the SQL statement is executed
• Connection pool
• • 3.1 Do not use the UnpooledDataSource. Create a Connection each time
  • 3.2 Experiment: Why use connection pooling? Creating a Connection is too costly
  • 3.3 Use the PooledDataSource class to reduce the cost of creating a Connection
  • • 3.3.1 PooledDataSource class structure: two list: idleConnections and activeConnections
    • 3.3.2 Using Connection Pools: java.sql.Connection object fetching
    • • • 3.3.2.1 Using a Connection pool to create a Connection object
        • 3.3.2.2 popConnection ()
        • 3.3.2.3 getProxyConnection ()
    • 3.3.3 Using Connection Pooling: Collection of java.sql.Connection objects
    • • • 3.3.3.1 Small experiment: When the connection pool is not used, con.close() closes the connection and releases resources
        • 3.3.3.2 Little experiment: Use a Connection pool, instead of conn.close(), to put a Connection object into a Connection pool
        • 3.3.3.3 Source code parsing (very important) : The proxy pattern implements a Connection object that calls the close() method and actually adds it to the Connection pool
• A DataSource of JNDI type
• 5. Interview Goldfinger
• • • Interview Goldfinger 1: Mybatis three data sources and their relationships
    • Interview Goldfinger 2: Create data source factory and data source when Mybatis initializes
    • Mybatis delays creating a Connection object until the SQL statement is executed
    • Interview Goldfinger 4: Connect two lists of pools
    • Interview Goldfinger 5: Creation of the java.sql.Connection object for the Connection pool
    • Interview Goldfinger 6: Collection of java.sqL. Connection objects from the Connection pool
    • Interview Goldfinger 7: DataSource of JNDI type
• Six, the summary

One, foreword

This paper will first describe the classification of data sources of MyBatis;

It then describes how data sources are loaded and used;

This is followed by a breakdown of UNPOOLED, POOLED, and JNDI type data source organizations;

We’ll focus on POOLED data sources and the connection pooling principles they implement.

ORM framework: Hibernate and Mybatis are both ORM frameworks, ORM framework, data source organization is a must, otherwise you need this framework why?

Ii. Data Sources (MyBatis three data sources and three data source factories)

2.1 Three data sources of MyBatis

MyBatis data source implementation is in the following four packages:

The four packages correspond to one base package and three data sources

MyBatis divides DataSource into three types:

(1) UNPOOLED does not use the connection pool data source

(2) POOLED uses the connection pool data source

(3) JNDI A data source implemented using JNDI

That is:

Accordingly, MyBatis defines UnpooledDataSource which implements java.sql.DataSource interface, and PooledDataSource class to represent UNPOOLED and POOLED data sources. As shown below:

For a DataSource of JNDI type, the value is in the JNDI context.

MyBatis DataSource is divided into three types: (1) UNPOOLED does not use the connection pool data source (2) POOLED uses the connection pool data source (3) JNDI uses the JNDI data source PooledDataSource and UnPooledDataSource both implement the java.sql.DataSource interface, and PooledDataSource holds a reference to UnPooledDataSource. When PooledDataSource needs to create a Java.sql.DataSource instance, UnPooledDataSource does so. PooledDataSource simply provides an instance of a cache connection pool.

2.2 Create data source factory and data source when initializing Mybatis

public interface DataSourceFactory { void setProperties(Properties props); // DataSource DataSource getDataSource(); }Copy the code

2.3 DataSource delays creating a Connection object until the SQL statement is executed

MyBatis calls dataSource to create java.sql.Connection when we need to create SqlSession and execute SQL statement. That is, the creation of the java.SQL. Connection object is delayed until the SQL statement is executed.

For example, we have the following method to execute a simple SQL statement:

String resource = "mybatis-config.xml";
InputStream inputStream = Resources.getResourceAsStream(resource);
SqlSessionFactory sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
SqlSession sqlSession = sqlSessionFactory.openSession();
sqlSession.selectList("SELECT * FROM STUDENTS");
Copy the code

Sqlsession. selectList(” SELECT * FROM STUDENTS “), MyBatis creates a java.sql.Connection object by executing the following method underneath:

protected void openConnection() throws SQLException {
    if (log.isDebugEnabled()) {
      log.debug("Opening JDBC Connection");
    }
    connection = dataSource.getConnection();
    if(level ! = null) { connection.setTransactionIsolation(level.getLevel()); } setDesiredAutoCommit(autoCommmit); }Copy the code

How does UnpooledDataSource implement the getConnection() method? Look at the next section.

Summary: From SQLsession.selectList to openConnection() in transactions

Connection pool

3.1 Do not use the UnpooledDataSource. Create a Connection each time

When the type attribute of the configured into UNPOOLED, MyBatis will instantiate an instance UnpooledDataSourceFactory factory first, and then through getDataSource () method returns a UnpooledDataSource instance object references, We assume the dataSource.

With UnpooledDataSource’s getConnection(), a new Connection instance object is created each time it is called.

The UnPooledDataSource getConnection() method is implemented as follows:

/* Public Connection getConnection() throws SQLException {public Connection getConnection() throws SQLException {returndoGetConnection(username, password); } private Connection doGetConnection(String username, Throws SQLException {// Encapsulating username and password into properties properties = new properties ();if(driverProperties ! = null) { props.putAll(driverProperties); }if(username ! = null) { props.setProperty("user", username);
    }
    if(password ! = null) { props.setProperty("password", password);
    }
    returndoGetConnection(props); } /* * Obtain data Connection */ private Connection doGetConnection(Properties Properties) throws SQLException {//1. Initialize the driver initializeDriver(); / / 2. Obtained from the DriverManager Connection, to get a new Connection object Connection Connection = DriverManager. GetConnection (url, properties); Configure the Connection attribute configureConnection(connection). //4. Return the Connection objectreturn connection;
}
Copy the code

As the code above shows, the UnpooledDataSource does the following:

1. Initialize the drive: to determine whether a driver drive has been loaded into memory, if you haven’t loaded, can dynamically load driver class, and instantiate a driver object, use the DriverManager. RegisterDriver () method will be registered into memory, for later use.
2. Create a Connection object: use the DriverManager getConnection () method to create a Connection.
3. Configure the Connection object: set whether to automatically commit autoCommit and the isolationLevel isolationLevel.
4. Return the Connection object.

The sequence diagram above is shown below:

Conclusion: from the above code you can see, each time we call getConnection () method, by DriverManager. GetConnection () returns the new Java SQL. Examples of the Connection.

3.2 Experiment: Why use connection pooling? Creating a Connection is too costly

public static void main(String[] args) throws Exception
{
 
    String sql = "select * from hr.employees where employee_id < ? and employee_id >= ?"; PreparedStatement st = null; ResultSet rs = null; long beforeTimeOffset = -1L; Long afterTimeOffset = -1L; Long executeTimeOffset = -1l; Connection con = null; Class.forName("oracle.jdbc.driver.OracleDriver");
 
    beforeTimeOffset = new Date().getTime();
    System.out.println("before:\t" + beforeTimeOffset);
 
    con = DriverManager.getConnection("JDBC: oracle: thin: @ 127.0.0.1:1521: xe." "."louluan"."123456");
 
    afterTimeOffset = new Date().getTime();
    System.out.println("after:\t\t" + afterTimeOffset);
    System.out.println("Create Costs:\t\t" + (afterTimeOffset - beforeTimeOffset) + " ms"); st = con.prepareStatement(sql); // Set st.setint (1, 101); st.setInt(2, 0); Rs = st.executeQuery(); executeTimeOffset = new Date().getTime(); System.out.println("Exec Costs:\t\t" + (executeTimeOffset - afterTimeOffset) + " ms");
 
}
Copy the code

3.3 Use the PooledDataSource class to reduce the cost of creating a Connection

3.3.1 PooledDataSource class structure: two list: idleConnections and activeConnections

3.3.2 Using Connection Pools: java.sql.Connection object fetching

3.3.2.1 Using a Connection pool to create a Connection object

 public Connection getConnection() throws SQLException {
    return popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();
  }
 
  public Connection getConnection(String username, String password) throws SQLException {
    return popConnection(username, password).getProxyConnection();
  }
Copy the code

3.3.2.2 popConnection ()

Now let’s see what the popConnection() method does:

1. PooledConnection (idle) {PooledConnection (idle) {PooledConnection (idle); Otherwise, proceed to Step 2.
2. Check whether the active PooledConnection activeConnections is full. If not, create a new PooledConnection object, place it in the activeConnections pool, and return the PooledConnection object; Otherwise, proceed to step 3;
3. See if the PooledConnection object that first entered the activeConnections pool has expired: If it has expired, remove this object from the activeConnections pool, create a new PooledConnection object, add it to the activeConnections, and return this object; Otherwise proceed to step 4 (Goldfinger: Step 4 if not expired).
4. The thread waits, loop 2 steps (Goldfinger: step 4 goes to step 3, so PooledConnection activeConnections must be full, so it is necessary to find an expired one in step 3, if there is no expired one, keep looking for it until it is found)

/* * Pass a username and password, Return available PooledConnection */ private PooledConnection popConnection(String username, String password) throws SQLException { boolean countedWait =false;
    PooledConnection conn = null;
    long t = System.currentTimeMillis();
    int localBadConnectionCount = 0;
 
    while(conn == null) // loop {synchronized (state)if(state. IdleConnections. The size () > 0) {/ / have idle connections in the pool, take out the first conn. = the state idleConnections. Remove (0);if (log.isDebugEnabled())
                {
                    log.debug("Checked out connection " + conn.getRealHashCode() + " from pool."); }}else{// If not, create a new PooledConnection object // and place it in the activeConnections pool, then return the PooledConnection objectif(state. ActiveConnections. The size () < poolMaximumActiveConnections) {/ / create a new connection object conn = new PooledConnection(dataSource.getConnection(), this); @SuppressWarnings("unused")
                    //used in logging, if enabled
                    Connection realConn = conn.getRealConnection();
                    if (log.isDebugEnabled())
                    {
                        log.debug("Created connection " + conn.getRealHashCode() + "."); }}else{// Cannot create new connection // When the active connection pool is full and Cannot be created, fetch the first active connection, that is, the PooledConnection object that entered the pool first // calculate its verification time, If the verification time is greater than the maximum verification time specified by the connection pool, it is considered expired. Create a new PooledConnection PooledConnection oldestActiveConnection = using a realConnection within this PoolConnection state.activeConnections.get(0); long longestCheckoutTime = oldestActiveConnection.getCheckoutTime();if(longestCheckoutTime > poolMaximumCheckoutTime) { // Can claim overdue connection state.claimedOverdueConnectionCount++;  state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime; state.accumulatedCheckoutTime += longestCheckoutTime; state.activeConnections.remove(oldestActiveConnection);if(! oldestActiveConnection.getRealConnection().getAutoCommit()) { oldestActiveConnection.getRealConnection().rollback(); } conn = new PooledConnection(oldestActiveConnection.getRealConnection(), this); oldestActiveConnection.invalidate();if (log.isDebugEnabled())
                        {
                            log.debug("Claimed overdue connection " + conn.getRealHashCode() + "."); }}else{// The first one does not expire // If it cannot be released, it Must wait for Mustwait
                        try
                        {
                            if(! countedWait) { state.hadToWaitCount++; countedWait =true;
                            }
                            if (log.isDebugEnabled())
                            {
                                log.debug("Waiting as long as " + poolTimeToWait + " milliseconds for connection.");
                            }
                            long wt = System.currentTimeMillis();
                            state.wait(poolTimeToWait);
                            state.accumulatedWaitTime += System.currentTimeMillis() - wt;
                        }
                        catch (InterruptedException e)
                        {
                            break; }}}} // If the PooledConnection is obtained successfully, update its informationif(conn ! = null) {if (conn.isValid())
                {
                    if(! conn.getRealConnection().getAutoCommit()) { conn.getRealConnection().rollback(); } conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password)); conn.setCheckoutTimestamp(System.currentTimeMillis()); conn.setLastUsedTimestamp(System.currentTimeMillis()); state.activeConnections.add(conn); state.requestCount++; state.accumulatedRequestTime += System.currentTimeMillis() - t; }else
                {
                    if (log.isDebugEnabled())
                    {
                        log.debug("A bad connection (" + conn.getRealHashCode() + ") was returned from the pool, getting another connection.");
                    }
                    state.badConnectionCount++;
                    localBadConnectionCount++;
                    conn = null;
                    if (localBadConnectionCount > (poolMaximumIdleConnections + 3))
                    {
                        if (log.isDebugEnabled())
                        {
                            log.debug("PooledDataSource: Could not get a good connection to the database.");
                        }
                        throw new SQLException("PooledDataSource: Could not get a good connection to the database.");
                    }
                }
            }
        }
 
    }
 
    if (conn == null)
    {
        if (log.isDebugEnabled())
        {
            log.debug("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
        }
        throw new SQLException("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
    }
 
    return conn;
}
Copy the code

The corresponding processing flow chart is as follows:

Question 1: PooledConnection (conn==null); PooledConnection (conn==null); PooledConnection (conn==null); Direct return conn;

As shown above, for the PooledDataSource’s getConnection() method, the popConnection() method of the PooledDataSource class returns a PooledConnection object, PooledConnection’s getProxyConnection() is then called to return the Connection object.

3.3.2.3 getProxyConnection ()

3.3.3 Using Connection Pooling: Collection of java.sql.Connection objects

3.3.3.1 Small experiment: When the connection pool is not used, con.close() closes the connection and releases resources

private void test() throws ClassNotFoundException, SQLException
{
    String sql = "select * from hr.employees where employee_id < ? and employee_id >= ?";
    PreparedStatement st = null;
    ResultSet rs = null;
 
    Connection con = null;
    Class.forName("oracle.jdbc.driver.OracleDriver");
    try
    {
        con = DriverManager.getConnection("JDBC: oracle: thin: @ 127.0.0.1:1521: xe." "."louluan"."123456"); st = con.prepareStatement(sql); // Set st.setint (1, 101); st.setInt(2, 0); Rs = st.executeQuery(); Con.close (); con.close(); } catch (SQLException e) { con.close(); e.printStackTrace(); }}Copy the code

3.3.3.2 Little experiment: Use a Connection pool, instead of conn.close(), to put a Connection object into a Connection pool

Call con.close() : All resources held by the Connection object are freed, and the Connection object can no longer be used.

So, if we use Connection pooling, what if we run out of Connection objects and need to put them in the pool?

The first thought that probably pops into your head is that INSTEAD of calling con.close() when I should be calling con.close(), I’ll replace it with code that puts a Connection object into a Connection pool container!

Ok, we will implement the above idea, first define a simple connection Pool, and then rewrite the above code:

package com.foo.jdbc; import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; import java.util.Vector; /** ** a simple thread-safe connection pooling implementation, This Connection Pool is singleton * putConnection() adds Connection to the Connection Pool * getConnection() returns a Connection object */ Public class Pool {private static Vector<Connection> pool = new Vector<Connection>(); private static int MAX_CONNECTION =100; private static String DRIVER="oracle.jdbc.driver.OracleDriver";
    private static String URL = "JDBC: oracle: thin: @ 127.0.0.1:1521: xe." ";
    private static String USERNAME = "louluan";
    private static String PASSWROD = "123456"; static { try { Class.forName(DRIVER); } catch (ClassNotFoundException e) { e.printStackTrace(); }} public static void putConnection(Connection Connection){synchronized(pool) {synchronized(pool) {if(pool.size()<MAX_CONNECTION) { pool.add(connection); }}} /** * returns a Connection object, popping the first element if there is one in the pool; * If there are no elements in the connection Pool, create a Connection object and add it to the Pool * @return Connection
     */
    public static Connection getConnection(){
        Connection connection = null;
        synchronized(pool)
        {
            if(pool.size()>0)
            {
                connection = pool.get(0);
                pool.remove(0);
            }
            else{ connection = createConnection(); pool.add(connection); }}returnconnection; } /** * create a new Connection object */ private static ConnectioncreateConnection()
    {
        Connection connection = null;
        try {
            connection = DriverManager.getConnection(URL, USERNAME,PASSWROD);
        } catch (SQLException e) {
            e.printStackTrace();
        }
        returnconnection; }}Copy the code

package com.foo.jdbc;
 
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Vector;
 
public class PoolTest
{
 
    private void test() throws ClassNotFoundException, SQLException
    {
        String sql = "select * from hr.employees where employee_id < ? and employee_id >= ?";
        PreparedStatement st = null;
        ResultSet rs = null;
 
        Connection con = null;
        Class.forName("oracle.jdbc.driver.OracleDriver");
        try
        {
            con = DriverManager.getConnection("JDBC: oracle: thin: @ 127.0.0.1:1521: xe." "."louluan"."123456"); st = con.prepareStatement(sql); // Set st.setint (1, 101); st.setInt(2, 0); Rs = st.executeQuery(); Pool.putconnection (con); pool.putConnection (con); } catch (SQLException e) { e.printStackTrace(); }}}Copy the code

Conn. Close () into the Pool. PutConnection (con);

The above code simply puts the Connection object we used into the Pool. If we need a Connection object, we simply use the pool.getConnection () method to fetch it from the Pool.

Yes, the above code can do this, but there is a rather inelegant implementation: we need to manually place the Connection object into the Pool, which is a silly implementation. This also differs from the usual way of using a Connection object, which is to use it and then call the close() method to release the resource.

In keeping with the usual way of using Conneciton objects, we want the close() method to be called when Connection is finished, and the Connection resource is not actually released, but actually added to the Connection pool. Can you do that? The answer is yes. Another way to describe this requirement is to provide a mechanism that lets us know what methods the Connection object calls, so that we can customize the corresponding processing mechanism for different methods. A proxy mechanism does just that (good).

3.3.3.3 Source code parsing (very important) : The proxy pattern implements a Connection object that calls the close() method and actually adds it to the Connection pool

class PooledConnection implements InvocationHandler { //...... private PooledDataSource dataSource; // Data source reference private Connection realConnection; // Real Connection object private Connection proxyConnection; // Proxy own proxy Connection //...... }Copy the code

public PooledConnection(Connection connection, PooledDataSource dataSource) { this.hashCode = connection.hashCode(); // hashcode this.realConnection = connection; // True connection this. DataSource = dataSource; This.createdtimestamp = system.currentTimemillis (); This.lastusedtimestamp = system.currentTimemillis (); // Last modified time this.valid =true; // valid this.proxyConnection = (Connection) Proxy.newProxyInstance(Connection.class.getClassLoader(), IFACES, this); // proxy connection}Copy the code

public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { String methodName = method.getName(); // When the call is closed, reclaim the Connection to PooledDataSourceif(CLOSE.hashCode() == methodName.hashCode() && CLOSE.equals(methodName)) { dataSource.pushConnection(this); // When the call is closed, it is put into the connection poolreturn null;
    } else {  // elseMethod. Invoke (realConnection, args); // Invoke (realConnection, args); // Only close() does good try {if(! Object.class.equals(method.getDeclaringClass())) { checkConnection(); }returnmethod.invoke(realConnection, args); } catch (Throwable t) { throw ExceptionUtil.unwrapThrowable(t); }}}Copy the code

A DataSource of JNDI type

MyBatis defines a JndiDataSourceFactory factory to create a DataSource generated through JNDI form.

Let’s take a look at the key code for JndiDataSourceFactory:

if(properties.containsKey(INITIAL_CONTEXT) && properties.containsKey(DATA_SOURCE)) {// Find the DataSource from the JNDI Context and return the Context ctx = (Context) initCtx.lookup(properties.getProperty(INITIAL_CONTEXT)); dataSource = (DataSource) ctx.lookup(properties.getProperty(DATA_SOURCE)); }else if(properties.containsKey(DATA_SOURCE)) {// // find the DataSource from the JNDI context and return DataSource = (DataSource) initCtx.lookup(properties.getProperty(DATA_SOURCE)); }Copy the code

5. Interview Goldfinger

Interview Goldfinger 1: Mybatis three data sources and their relationships

MyBatis DataSource is divided into three types: (1) UNPOOLED does not use the connection pool data source (2) POOLED uses the connection pool data source (3) JNDI uses the JNDI data source PooledDataSource and UnPooledDataSource both implement the java.sql.DataSource interface, and PooledDataSource holds a reference to UnPooledDataSource. When PooledDataSource needs to create a Java.sql.DataSource instance, UnPooledDataSource does so. PooledDataSource simply provides an instance of a cache connection pool.

Interview Goldfinger 2: Create data source factory and data source when Mybatis initializes

Mybatis delays creating a Connection object until the SQL statement is executed

Summary: From SQLsession.selectList to openConnection() in transactions

Interview Goldfinger 4: Connect two lists of pools

Interview Goldfinger 5: Creation of the java.sql.Connection object for the Connection pool

Using a Connection pool, create a Connection object in two steps. First call popConnection() to get a PooledConnection object (important, see 3.3.2.2 popConnection() for two queues). Then call getProxyConnection() of the PooledConnection class to return a proxyConnection variable

Interview Goldfinger 6: Collection of java.sqL. Connection objects from the Connection pool

public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { String methodName = method.getName(); // When the call is closed, reclaim the Connection to PooledDataSourceif(CLOSE.hashCode() == methodName.hashCode() && CLOSE.equals(methodName)) { dataSource.pushConnection(this); // When the call is closed, it is put into the connection poolreturn null;
    } else {  // elseMethod. Invoke (realConnection, args); // Invoke (realConnection, args); // Only close() does good try {if(! Object.class.equals(method.getDeclaringClass())) { checkConnection(); }returnmethod.invoke(realConnection, args); } catch (Throwable t) { throw ExceptionUtil.unwrapThrowable(t); }}}Copy the code

Interview Goldfinger 7: DataSource of JNDI type

MyBatis defines a JndiDataSourceFactory to create a DataSource generated through the JNDI form.

if(properties.containsKey(INITIAL_CONTEXT) && properties.containsKey(DATA_SOURCE)) {// Find the DataSource from the JNDI Context and return the Context ctx = (Context) initCtx.lookup(properties.getProperty(INITIAL_CONTEXT)); dataSource = (DataSource) ctx.lookup(properties.getProperty(DATA_SOURCE)); }else if(properties.containsKey(DATA_SOURCE)) {// // find the DataSource from the JNDI context and return DataSource = (DataSource) initCtx.lookup(properties.getProperty(DATA_SOURCE)); }Copy the code

Six, the summary