This is the second day of my participation in the November Gwen Challenge. Check out the details: the last Gwen Challenge 2021
Introduction to the
In this article, we will try to explore the overall implementation of Connection pooling for Alibaba Druid database
The general idea of connection pool
Query read some data: the following two or write better
- The connection pooling technology is designed and implemented in Python.
- How to design and implement a DB connection pool?
The core of a database connection pool is as follows:
The basic principle of connection pool design is as follows:
(1) Establish the connection pool object (service startup).
(2) Create an initial number of connections (i.e., the number of idle connections) according to the parameters specified in advance.
(3) For an access request, get a connection directly from the connection pool. If there are no free connections in the connection pool object, and the number of connections has not reached the maximum (i.e., the maximum number of active connections), create a new connection; If the maximum is reached, a certain timeout period is set to obtain the connection.
(4) Use connections to access services.
(5) After accessing the service, release the connection (at this time, the release of the connection is not really closed, but put it into the idle queue. If the actual number of free connections is greater than the initial number of free connections, the connection is released.
(6) Release the connection pool object (release the connection pool object and all connections during service stop and maintenance).
Understanding connection pooling Technology principles, Design and Implementation (Python)
The above, combined with the Python code above, feels like the key to implementing a connection pool
The source code to explore
Based on the above ideas, we can see that the connection acquisition and release should be the key implementation, so let’s start to try to find the relevant code
Run the sample, debug entry
First have to find a breakthrough point, double the test file, find the one thing: the SRC/test/Java/com/alibaba/druid/pool/demo/Demo0. Java
public class Demo0 extends TestCase {
private String jdbcUrl;
private String user;
private String password;
private String driverClass;
private int initialSize = 10;
private int minPoolSize = 1;
private int maxPoolSize = 2;
private int maxActive = 12;
protected void setUp(a) throws Exception {
jdbcUrl = "jdbc:h2:file:./demo-db";
user = "sa";
password = "sa";
driverClass = "org.h2.Driver";
}
public void test_0(a) throws Exception {
DruidDataSource dataSource = new DruidDataSource();
JMXUtils.register("com.alibaba:type=DruidDataSource", dataSource);
dataSource.setInitialSize(initialSize);
dataSource.setMaxActive(maxActive);
dataSource.setMinIdle(minPoolSize);
dataSource.setMaxIdle(maxPoolSize);
dataSource.setPoolPreparedStatements(true);
dataSource.setDriverClassName(driverClass);
dataSource.setUrl(jdbcUrl);
dataSource.setPoolPreparedStatements(true);
dataSource.setUsername(user);
dataSource.setPassword(password);
dataSource.setValidationQuery("SELECT 1");
dataSource.setTestOnBorrow(true); Connection conn = dataSource.getConnection(); conn.close(); System.out.println(); }}Copy the codeCan not be run in the source code, you need to make the following modifications to run successfully:
- 1. Set maxActive to a value greater than initSize: The initial number of connections is always displayed at runtime, so we need to change the initial number of connections not less than the maximum number of active connections
- 2. Replace with memory data (can run without replacing, this is just for fun)
We put breakpoints on both lines of code below:
Connection conn = dataSource.getConnection();
conn.close();
Copy the codeCreate and get database connection code
Conn = datasource.getConnection ();
# DruidDataSource.java
public DruidPooledConnection getConnection(long maxWaitMillis) throws SQLException {, # init();if (filters.size() > 0) {
FilterChainImpl filterChain = new FilterChainImpl(this);
return filterChain.dataSource_connect(this, maxWaitMillis);
} else {
// Go here to get the connection
returngetConnectionDirect(maxWaitMillis); }}Copy the codeHaving directly retrieved initialization and database connections in the above code, let’s take a look at initialization
In the initialized DIAM, we see something that fits our supposed view of the type queue initialization, as follows:
# DruidDataSource.java
public void init(a) throws SQLException {# suspected initializing pool connections =new DruidConnectionHolder[maxActive];
evictConnections = new DruidConnectionHolder[maxActive];
keepAliveConnections = new DruidConnectionHolder[maxActive];
SQLException connectError = null;
if(createScheduler ! =null && asyncInit) {
for (int i = 0; i < initialSize; ++i) {
submitCreateTask(true); }}else if(! asyncInit) {// init connections
// This is the connection with the initial amount of data initialized according to the configuration
while (poolingCount < initialSize) {
try {
PhysicalConnectionInfo pyConnectInfo = createPhysicalConnection();
DruidConnectionHolder holder = new DruidConnectionHolder(this, pyConnectInfo);
connections[poolingCount++] = holder;
} catch (SQLException ex) {
LOG.error("init datasource error, url: " + this.getUrl(), ex);
if (initExceptionThrow) {
connectError = ex;
break;
} else {
Thread.sleep(3000);
}
}
}
}
} catch (SQLException e) {
} finally{}}}Copy the codeContinue with the code related to getting connections:
public DruidPooledConnection getConnectionDirect(long maxWaitMillis) throws SQLException {
int notFullTimeoutRetryCnt = 0;
for (;;) {
// handle notFullTimeoutRetry
DruidPooledConnection poolableConnection;
try {
// Get the connection
poolableConnection = getConnectionInternal(maxWaitMillis);
} catch (GetConnectionTimeoutException ex) {
}
returnpoolableConnection; }}private DruidPooledConnection getConnectionInternal(long maxWait) throws SQLException {
DruidConnectionHolder holder;
for (;;) {
if (maxWait > 0) {
holder = pollLast(nanos);
} else {
// take the last one
holder = takeLast();
}
// Calculation of related statistics
if(holder ! =null) {
if (holder.discard) {
continue;
}
activeCount++;
holder.active = true;
if(activeCount > activePeak) { activePeak = activeCount; activePeakTime = System.currentTimeMillis(); }}}catch (InterruptedException e) {
connectErrorCountUpdater.incrementAndGet(this);
throw new SQLException(e.getMessage(), e);
} catch (SQLException e) {
connectErrorCountUpdater.incrementAndGet(this);
throw e;
} finally {
lock.unlock();
}
break;
}
DruidPooledConnection poolalbeConnection = new DruidPooledConnection(holder);
return poolalbeConnection;
}
DruidConnectionHolder takeLast(a) throws InterruptedException, SQLException {
decrementPoolingCount();
// Get the last connection from the variable in init initialization
DruidConnectionHolder last = connections[poolingCount];
connections[poolingCount] = null;
return last;
}
Copy the codeWith the above code, we have a rough idea of the key code to get links:
- Initializing connections: Initializes the number of links based on the number of initial connections set. See that the current use is a simple array structure to save
- Get links: What you see so far is get the last one
Close the connection
Let’s trace the entry to close the connection: conn.close();
# DruidPooledConnection.java
@Override
public void close(a) throws SQLException {
try {
List<Filter> filters = dataSource.getProxyFilters();
if (filters.size() > 0) {}else {
// Recycle the entryrecycle(); }}finally{}}public void recycle(a) throws SQLException {
if (!this.abandoned) {
DruidAbstractDataSource dataSource = holder.getDataSource();
// Enter the recycle logic
dataSource.recycle(this); }}Copy the codeThere is no specific logic to close the physical connection. This logic is used to set some identification bits and put the connection back
# DruidDataSource.java
/** * reclaim connection */
protected void recycle(DruidPooledConnection pooledConnection) throws SQLException {
final DruidConnectionHolder holder = pooledConnection.holder;
lock.lock();
try {
if (holder.active) {
activeCount--;
holder.active = false;
}
closeCount++;
result = putLast(holder, currentTimeMillis);
recycleCount++;
} finally{ lock.unlock(); }}Copy the codeconclusion
In this article, we reviewed some articles on connection pooling implementation, and then located the core code in Alibaba Druid that matches our guess:
- Initializing connections: Initializes the number of links based on the number of initial connections set. See that the current use is a simple array structure to save
- Get links: What you see so far is get the last one
- Reclaim connection: do not close the physical connection, reset some identity bits
Of course, there are other details, which I’ll explore later
Refer to the link
- The connection pooling technology is designed and implemented in Python.
- How to design and implement a DB connection pool?