Go and Redis connection pool

This is the sixth day of my participation in the August More text Challenge. For details, see:August is more challenging

This article appears in my column: Let’s Golang together

What is connection pooling? What is connection pooling

Connection pooling:

The basic idea of connection pool is to store the database connection as an object in memory when the system is initialized. When the user needs to access the database, instead of establishing a new connection, he will take an established free connection object from the connection pool. After use, the user does not close the connection, but puts it back into the connection pool for the next request to access. The establishment and disconnection of connections are managed by the connection pool itself. You can also set connection pool parameters to control the initial number of connections in the connection pool, the upper and lower limits of connections, the maximum number of uses per connection, the maximum idle time, and so on. It also has its own management mechanism to monitor the number of database connections, usage, and so on.

I’ll explain this briefly, because every time a Redis client connects to a Redis server it takes a while, and the processing time is often very short. If you need to reconnect to Redis every time you perform a variety of operations, you will waste a lot of time. Therefore, Redis introduced connection pool, which can realize the establishment of multiple client connections without release, avoid wasting IO resources, when not in use, put in the connection pool, so as to reduce the time needed to connect to the database, improve efficiency.

To connect a pool is to build a pool and a certain number of pipes. Every time the pipe is exhausted, it can no longer consume IO resources, thus ensuring that IO resources are not exhausted.

2. Code presentation

package main

import (
    "fmt"
    "github.com/garyburd/redigo/redis"
    "strconv"
    "time"
)

func main(a) {
    pool := &redis.Pool{
        // Maximum number of connections allocated by the pool at a given time.
        // When zero, there is no limit on the number of connections in the pool.
        // The maximum number of active connections. 0 is infinite
        MaxActive: 888.// The maximum number of idle connections
        // Maximum number of idle connections in the pool.
        MaxIdle: 20.// Idle connection timeout
        // Close connections after remaining idle for this duration. If the value
        // is zero, then idle connections are not closed. Applications should set
        // the timeout to a value less than the server's timeout.
        IdleTimeout: time.Second * 100.// Define the function that dials up to get the connection
        // Dial is an application supplied function for creating and configuring a
        // connection.
        //
        // The connection returned from Dial must not be in a special state
        // (subscribed to pubsub channel, transaction started, ...) .
        Dial: func(a) (redis.Conn, error) {
            return redis.Dial("tcp"."127.0.0.1:6379"),}}// Delay closing the connection pool
    defer pool.Close()
    //I/O concurrent connection
    for i:=0; i<10; i++{go getConnFromPoolAndHappy(pool,i)
    }
    // Keep the main coroutine alive
    time.Sleep(3*time.Second)

}

func getConnFromPoolAndHappy(pool *redis.Pool, i int)  {
    // Get the connection from the connection pool
    conn := pool.Get()
    // Delay closing the connection
    defer conn.Close()
    // Use the connection to manipulate the data
    reply, err := conn.Do("set"."conn"+strconv.Itoa(i), i)
    s, _ := redis.String(reply, err)
    fmt.Println(s)
}
Copy the code

MaxActive: 888
Copy the code

The above code represents the maximum number of connections that can be allocated by the connection pool in a given time. When this value is 0, the number of connections in the connection pool is uncapped.

MaxIdle: 20
Copy the code

Indicates the upper limit of the number of idle connections in the connection pool.

IdleTimeout: time.Second * 100
Copy the code

This value indicates that if the pool’s idle connections exceed this value, the connection will be closed. If the value is zero, idle connections in the connection pool will not be closed. The application should set this limit timeout to not exceed the limit timeout on the server side.

Dial: func(a) (redis.Conn, error) {
   return redis.Dial("tcp"."127.0.0.1:6379")}Copy the code

This value is a function that should be used to create and configure connections. Connections returned from Dial cannot be in a special state.

Create a concurrent coroutine to execute this function, because the range of the for loop is 0 to 9, so the concurrency is 10, which means that 10 people are trying to connect to the pool at the same time.

go getConnFromPoolAndHappy(pool,i)
Copy the code

The following code is used to keep the main coroutine alive.

time.Sleep(3*time.Second)
Copy the code

The following indicates that the connection is obtained through the connection pool. You take a pipe from the pool.

conn := pool.Get()
Copy the code

Then the following code is to use pipes.

reply, err := conn.Do("set"."conn"+strconv.Itoa(i), i)
	s, _ := redis.String(reply, err)
	fmt.Println(s)
Copy the code

If a person occupies the pipe and does not use it, it will be idle. If you stay idle for too long and reach the idle connection timeout, you will be removed.

Because the pools all use the same thing, you need to use the pool pointer.

pool := &redis.Pool
Copy the code

Go and Redis connection pool