Use Go to learn about the Redis communication protocol

Go, PHP, Java… With so many packages to support your use of Redis, have you ever thought about it

With the server, with the client, how do they communicate, and based on what communication protocol do they interact?

introduce

Based on our purpose, this paper mainly explains and practices Redis communication protocol

The client and server of Redis interact data through TCP connection. The default port number of the server is 6379

All commands or data sent by the client and server end with \r\n (CRLF) (this is a convention)

agreement

In Redis, the request protocol is divided into request and reply, and the request protocol is divided into the new version and the old version. The new unified request protocol was introduced in Redis 1.2, and finally became the standard way of communication for Redis server in Redis 2.0

This article is based on the new protocol to implement the function, do not recommend using the old version (1.2 is quite old). Here are various examples of the new protocol:

Request protocol

1. Format example

*< number of bytes > CR LF $< number of bytes > CR LF < number of bytes > CR LF... $< number of bytes of parameter N > CR LF < data of parameter N > CR LFCopy the code

All parameters sent to the Redis server are binary safe under this protocol

2. Print an example

* 3$3
SET
A $5
mykey
$7
myvalue
Copy the code

3. Actual protocol value

"*3\r\n$3\r\nSET\r\nA $5\r\nmykey\r\n$7\r\nmyvalue\r\n"
Copy the code

This is the request protocol specification of Redis, according to example 1 to write the client logic, finally sent is example 3, I believe you have a general concept, Redis protocol is very simple and easy to understand, this is also one of the reasons for good use, you can think of the protocol so defined benefits where?

reply

Redis will return many different types of responses depending on the protocol you requested (and the results of the commands executed). In the reply “protocol”, you can determine the type of reply by examining the first byte as follows:

  • The first byte of a status reply is “+”
  • The first byte of an error reply is “-“.
  • The first byte of integer reply is “:”
  • The first byte of bulk reply is “$”
  • The first byte of multi Bulk Reply is “*”.

With the header identification and the CRLF at the end, you can guess roughly what the reply “protocol” is, but the answer is true according to practice. Can you forget 😀 very soon

practice

Interact with the Redis server

package main

import (
	"log"
	"net"
	"os"

	"github.com/EDDYCJY/redis-protocol-example/protocol"
)

const (
	Address = "127.0.0.1:6379"
	Network = "tcp"
)

func Conn(network, address string) (net.Conn, error) {
	conn, err := net.Dial(network, address)
	iferr ! = nil {return nil, err
	}

	return conn, nil
}

func mainArgs := os.args [1:]if len(args) <= 0 {
		log.Fatalf("Os.Args <= 0"ReqCommand := protocol.GetRequest(args) // Connect to Redis server redisConn, err := Conn(Network, Address)iferr ! = nil { log.Fatalf("Conn err: %v", err)} defer redisconn.close () // Write the request _, err = redisconn.write (reqCommand)iferr ! = nil { log.Fatalf("Conn Write err: %v", err)} // Read the replycommand := make([]byte, 1024)
	n, err := redisConn.Read(command)
	iferr ! = nil { log.Fatalf("Conn Read err: %v"Err := protocol.GetReply(err := protocol.command[:n])
	iferr ! = nil { log.Fatalf("protocol.GetReply err: %v", err)} return log.printf ("Reply: %v", reply) // Original reply log.printf ("Command: %v", string(command[:n]))
}
Copy the code

Here we complete the whole Redis client and server interaction process as follows:

1. Read command line parameters: get the executed Redis command

2. Obtain request protocol parameters

3. Connect to Redis server and obtain connection handle

4. Write the request protocol parameters to the connection: command line parameters that send the request

Read the returned data from the connection: Read the reply data from the previous request

6. Process the reply data set according to the reply “Agreement”

7. Output the processed reply content and original reply content

request

func GetRequest(args []string) []byte {
	req := []string{
		"*" + strconv.Itoa(len(args)),
	}

	for _, arg := range args {
		req = append(req, "$"+strconv.Itoa(len(arg)))
		req = append(req, arg)
	}

	str := strings.Join(req, "\r\n")
	return []byte(str + "\r\n")}Copy the code

Through the analysis of Redis request protocol, we can get its rule, add the flag bit first, calculate the total number of parameters, and then recycle the number of bytes and values of each parameter

reply

func GetReply(reply []byte) (interface{}, error) {
	replyType := reply[0]
	switch replyType {
	case StatusReply:
		return doStatusReply(reply[1:])
	case ErrorReply:
		return doErrorReply(reply[1:])
	case IntegerReply:
		return doIntegerReply(reply[1:])
	case BulkReply:
		return doBulkReply(reply[1:])
	case MultiBulkReply:
		return doMultiBulkReply(reply[1:])
	default:
		return nil, nil
	}
}

func doStatusReply(reply []byte) (string, error) {
	if len(reply) == 3 && reply[1] == 'O' && reply[2] == 'K' {
		return OkReply, nil
	}

	if len(reply) == 5 && reply[1] == 'P' && reply[2] == 'O' && reply[3] == 'N' && reply[4] == 'G' {
		return PongReply, nil
	}

	return string(reply), nil
}

func doErrorReply(reply []byte) (string, error) {
	return string(reply), nil
}

func doIntegerReply(reply []byte) (int, error) {
	pos := getFlagPos('\r', reply)
	result, err := strconv.Atoi(string(reply[:pos]))
	iferr ! = nil {return 0, err
	}

	return result, nil
}

...
Copy the code

Here, we distribute all reply types, and different reply flag bits correspond to different processing methods. Here, we need to pay attention to the following problems:

When the requested value does not exist, the special value -1 will be used as the reply

2. All strings sent by the server are terminated by CRLF

3. Multiple batch replies can be based on batch replies, so pay attention to understanding

4. Multiple batch replies without content exist

Most importantly, the control of the rules for different replies allows you to better understand the interaction between Redis requests and replies 👌

summary

The reason for writing this article is that it is often very uncomfortable to use Redis just to use, you don’t know what protocol it is based on

Through this article, I believe that you have a general understanding of how the Redis client interacts with the server side, and also understand the communication principle used. I hope it can help you!

Finally, if you want to look at the code in detail, right turn the project address

If you can help, please go to Star 👍

reference

  • Communication protocol