background
Database connection pooling (DBCP, c3P0, druid, etc.) is a very useful tool for Java development, but it is important to understand how to use a database connection pool. The realization principle of database connection pool is briefly analyzed
The version I’m using here is version 2.5.0
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
<version>2.5.0</version>
</dependency>
Copy the codeAnalysis of the
To prepare
Let’s start with a piece of native JDBC code (omitting exception catching) :
Class.forName("com.mysql.cj.jdbc.Driver");
String url = "jdbc:mysql://localhost:3306/xxxxxx";
Connection conn = DriverManager.getConnection(url, "user"."password");
Statement stat = conn.createStatement();
ResultSet set = stat.executeQuery(sql);
while(set.next()) {
System.out.println(set.getString(2));
}
stat.close();
conn.close();
Copy the codeI don’t care about the structure of the code, just one sentence
Connection conn = DriverManager.getConnection(url, "user"."password");
Copy the codeWhy single this one out? Since we also call the getConnection method using the database connection pool, Java advocates using the DataSource interface to retrieve connections in the JDBC 2.0 API
We are now focused on finding the DataSource interface provided by DHCP. If we have used DHCP configuration, we must have seen this configuration
spring:
datasource:
type: org.apache.commons.dbcp2.BasicDataSource
Copy the codeThe BasicDataSource class implements the DataSource interface:
public class BasicDataSource implements DataSource.BasicDataSourceMXBean.MBeanRegistration.AutoCloseable
Copy the codeSince we want to explore the principle, let’s not focus on the details. Let’s go straight to the core getConnection method and ignore all exception catching statements for clarity:
@Override
public Connection getConnection(a) throws SQLException {
if (Utils.IS_SECURITY_ENABLED) {
final PrivilegedExceptionAction<Connection> action = new PaGetConnection();
return AccessController.doPrivileged(action);
}
return createDataSource().getConnection();
}
Copy the codeThe whole code is very simple, according to whether to turn on security management, to choose the way to get the connection, we only look at the way without security management
Data source creation
Let’s start with the createDataSource method:
protected DataSource createDataSource(a) throws SQLException {
if (closed) {
throw new SQLException("Data source is closed");
}
// If the data source already exists, return it directly
if(dataSource ! =null) {
return dataSource;
}
synchronized (this) {
// Use a double-lock design to ensure the dataSource is singleton
if(dataSource ! =null) {
return dataSource;
}
// Register mbeans. This is not the point
jmxRegister();
// Create a factory that returns native connections
final ConnectionFactory driverConnectionFactory = createConnectionFactory();
// Create a connection pool factory
boolean success = false;
PoolableConnectionFactory poolableConnectionFactory;
try {
poolableConnectionFactory = createPoolableConnectionFactory(driverConnectionFactory);
poolableConnectionFactory.setPoolStatements(poolPreparedStatements);
poolableConnectionFactory.setMaxOpenPreparedStatements(maxOpenPreparedStatements);
success = true;
} catch (final SQLException se) {
throw se;
} catch (final RuntimeException rte) {
throw rte;
} catch (final Exception ex) {
throw new SQLException("Error creating connection factory", ex);
}
if (success) {
// The connection pool was created successfully
createConnectionPool(poolableConnectionFactory);
}
// Create a data source pool to manage connections
DataSource newDataSource;
success = false;
try {
newDataSource = createDataSourceInstance();
newDataSource.setLogWriter(logWriter);
success = true;
} catch (final SQLException se) {
throw se;
} catch (final RuntimeException rte) {
throw rte;
} catch (final Exception ex) {
throw new SQLException("Error creating datasource", ex);
} finally {
if (!success) {
closeConnectionPool();
}
}
// If the initial connection number is greater than 0, preload
try {
for (int i = 0; i < initialSize; i++) { connectionPool.addObject(); }}catch (final Exception e) {
closeConnectionPool();
throw new SQLException("Error preloading the connection pool", e);
}
// If space connection collector execution interval is greater than 0, add collector task
startPoolMaintenance();
// Returns the data source
dataSource = newDataSource;
returndataSource; }}Copy the codeThe code looks long, but the process is simple:
- Determine if the result can be returned directly (the data source is closed or already exists), and if so, return directly
- Create a factory that returns a database connection
- Create a connection pool factory
- Wrap the connection pool factory with a database connection factory
- Create a database connection pool
- Create data source instances through connection pooling
- Determine if additional actions (such as preloading database connections) are required based on the parameters you set
- Return data source
We only focus on three things: what a database connection factory is, what a connection pool factory is, and what a connection pool is. Let’s start with the database connection factory
createConnectionFactory
The code for this method is a bit confusing compared to the above, so I’ll ignore all the exception catching code here for the sake of understanding:
protected ConnectionFactory createConnectionFactory(a) throws SQLException {
// Load the JDBC driver
Driver driverToUse = this.driver;
if (driverToUse == null) { Class<? > driverFromCCL =null;
if(driverClassName ! =null) {
if (driverClassLoader == null) {
driverFromCCL = Class.forName(driverClassName);
} else {
driverFromCCL = Class.forName(driverClassName, true, driverClassLoader); }}if (driverFromCCL == null) {
// This is the url where we set up the connection to the database
driverToUse = DriverManager.getDriver(url);
} else {
driverToUse = (Driver) driverFromCCL.getConstructor().newInstance();
if(! driverToUse.acceptsURL(url)) {throw new SQLException("No suitable driver"."08001"); }}}// Set up the connection
final String user = userName;
if(user ! =null) {
connectionProperties.put("user", user);
} else {
log("DBCP DataSource configured without a 'username'");
}
final String pwd = password;
if(pwd ! =null) {
connectionProperties.put("password", pwd);
} else {
log("DBCP DataSource configured without a 'password'");
}
final ConnectionFactory driverConnectionFactory = new DriverConnectionFactory(driverToUse, url,
connectionProperties);
return driverConnectionFactory;
}
Copy the codeWhat this method does is simply load the driver, and the connection factory is actually created in the second to last line of new DriverConnectionFactory, and we end up creating an object of type DriverConnectionFactory. Here we pass attributes like the database driver, connection URL, username, and password to the factory object
For the DriverConnectionFactory class, here’s a snippet of the code, and I’m sure there’s nothing more to say
@Override
public Connection createConnection(a) throws SQLException {
return driver.connect(connectionString, properties);
}
Copy the codePoolableConnectionFactory
Then look at the connection pool factory class, but we are not going to study this class, but returned to createDataSource method, discovered that a createConnectionPool method, introduced into PoolableConnectionFactory class object, Let’s go into this method:
protected void createConnectionPool(final PoolableConnectionFactory factory) {
// Create an object pool
final GenericObjectPoolConfig<PoolableConnection> config = newGenericObjectPoolConfig<>(); updateJmxName(config); config.setJmxEnabled(registeredJmxObjectName ! =null);
final GenericObjectPool<PoolableConnection> gop = createObjectPool(factory, config, abandonedConfig);
gop.setMaxTotal(maxTotal);
gop.setMaxIdle(maxIdle);
gop.setMinIdle(minIdle);
gop.setMaxWaitMillis(maxWaitMillis);
gop.setTestOnCreate(testOnCreate);
gop.setTestOnBorrow(testOnBorrow);
gop.setTestOnReturn(testOnReturn);
gop.setNumTestsPerEvictionRun(numTestsPerEvictionRun);
gop.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
gop.setSoftMinEvictableIdleTimeMillis(softMinEvictableIdleTimeMillis);
gop.setTestWhileIdle(testWhileIdle);
gop.setLifo(lifo);
gop.setSwallowedExceptionListener(new SwallowedExceptionLogger(log, logExpiredConnections));
gop.setEvictionPolicyClassName(evictionPolicyClassName);
factory.setPool(gop);
connectionPool = gop;
}
Copy the code- The first line creates an object pool. Apache provides four object pools. GenericObjectPoolConfig is just one of them
- The next two lines have to do with Jmx, which I don’t care about here
- In the next few lines, create a pool of objects using the connection pool and other configurations we provide, and set the values of the variables, which can be set from our configuration file
- Finally, we put connection pool Settings to PoolableConnectionFactory factory class, and sets the data source connection pool itself to the connection pool
Now understand why don’t we just study PoolableConnectionFactory, because in fact we call connection pool is connectionPool
Get connected
We introduced connection factories, connection pooling factories, and connection pooling. In case you forgot what to do next, here’s the code snippet we started with:
return createDataSource().getConnection();
Copy the codeJust looking at the createDataSource method, which finally returns a data source object (the PoolingDataSource<PoolableConnection> object wrapped in a connectionPool), Let’s look at getConnection, the method that actually gets the connection
This is an abstract method, because createDataSource returns the PoolingDataSource object, so we analyze from this object. For clarity, we omit the exception catch code:
@Override
public Connection getConnection(a) throws SQLException {
final C conn = pool.borrowObject();
if (conn == null) {
return null;
}
return new PoolGuardConnectionWrapper<>(conn);
}
Copy the codeLet’s first address two possible problems:
- C is a subclass of Connection
- PoolGuardConnectionWrapper is to ensure that shut down the connection was disconnected by the reuse of packaging
Now there should be no problem, the whole process is to get the connection, return null if not, otherwise return a wrapper class. The connection is obtained through the borrowObject method. The implementation of this method is in GenericObjectPool (I don’t know why it is this class, it must be that the previous code did not read carefully). The implementation method of this class is not the focus of this paper. I’ll write a separate article about the four object pools Apache provides in the future
conclusion
Finally, according to the convention, to summarize the steps of DBCP connection pool to obtain a connection, here to call getConnection method to obtain a connection as an example analysis:
- Determines whether the data source is closed or exists, throws an exception if it is closed, and returns an exception if it exists
- Create a factory class that returns a database connection and load the database driver
- Create a connection pool factory class using the database connection factory class
- Create a database connection pool and set properties using the connection pool factory class
- The connection pool is used to create the data source and perform some initialization based on the configured properties
- Fetching objects (connections) from the object pool in the data source