In recent years, with the rapid development of the Internet, as programmers, we need to deal with the scale of data is also expanding. If you use Redis as a database, a single Instance of Redis will not be able to cope with the high concurrency of big data. At this time, the Redis cluster scheme came into being. It integrated numerous Instances of Redis to cope with the high concurrency of big data.

Codis is one of the Redis cluster solutions. It was developed by wandoujia’s middleware team, so it has a detailed README in Chinese for everyone to learn.

Its architecture is shown in the figure above. Codis-proxy provides Redis services externally. ZooKeeper stores data routing tables and meta information about the CODIS-proxy node. Codis-proxy listens to all Redis clusters. When the processing capacity of the Redis cluster reaches its upper limit, redis instances can be dynamically added to meet capacity expansion requirements.

Component is introduced

  • Codis Proxy: As mentioned earlier, it provides Redis services externally and behaves the same as native Redis except for some unsupported commands (a list of unsupported commands). Since it is stateless, we can deploy multiple nodes, ensuring availability.
  • Codis Dashboard: a cluster management tool that supports operations such as adding and deleting Codis proxies and data migration. For a Codis cluster, a maximum of one Dashboard can be deployed
  • Codis Admin: command line tool for cluster management
  • Codis FE: cluster management interface. Multiple Codis clusters can share one Codis FE and manage the Codis -dashboard on the backend through configuration files
  • Storage: Provides external Storage for clusters. Currently, it supports ZooKeeper, Etcd, and Fs.
  • Codis Server: Based on 3.2.8 branch development, additional data structures are added to support slot-related operations and data migration instructions.

Principle of Codis sharding

We now know that Codis will forward the Redis command specifying the key to the underlying Redis. So how does Codis know which Redis a key is on?

Codis uses pre-Sharding technology to implement data sharding, which is divided into 1024 slots (0-1023) by default. When the Codis receives the command, it performs crc32 on the key and mod 1024 on the key. The result is the corresponding slot. The command can then be forwarded to the slot’s Redis instance for processing.

Scale operation

Codis’s dynamic capacity to expand/shrink is one of its highlights. It can be transparent to Redis clients. During capacity expansion, Codis provides the SLOTSSCAN directive, which scans all keys in a specified slot and migrates each key. During capacity expansion, if a new key needs to be transferred to the slot that is being migrated, CODIS determines whether the key needs to be migrated. If so, coDIS forcibly migrates the specified key. After the migration, coDIS forwards the command to the new Redis.

Codis makes this decision. It makes automatic balancing. Just click the “Auto Rebalance” button on the screen and make slot transitions automatically. Scaling down is also simple. Simply migrate the slot of the instance that needs to be taken offline to other instances, and then delete the group.

Codis shortcomings

When the master of Redis Group dies, CODIS does not automatically promote a slave to master. Codis provides a tool called CODIS-HA, which provides RESTful APIS through Dashboard to realize automatic master/slave switchover. However, when CODIS promoted one slave to master, the other slaves did not change state and still synchronized data from the old master, resulting in data inconsistency between master and slave. Therefore, when a primary/secondary switchover occurs, the administrator needs to manually create a Sync action to complete data synchronization.

In addition, Codis also faces the awkward situation that since it is not Redis’s “own”, it is always a step behind when Redis releases new features. Therefore, it needs to catch up quickly after Redis releases new features in order to remain competitive.

Build Codis

  1. Install the Go runtime environment

Mac users can refer to this tutorial as well as users of other systems.

Once installed, verify that the installation was successful

$go version go version go1.11.2 Darwin/AMd64Copy the code
  1. Download the Codis source code

Need to download to the specified directory: $GOPATH/src/github.com/CodisLabs/codis

$ mkdir -p $GOPATH/src/github.com/CodisLabs
$ cd The $_ && git clonehttps://github.com/CodisLabs/codis.git - release3.2 bCopy the code
  1. Compile the source code

Enter the codis directory of the source code and execute the make command directly. Once compiled, the structure in the bin directory should look like this

$ ll bin total 178584 drwxr-xr-x 8 jackey staff 256B 11 13 10:57 assets -rwxr-xr-x 1 jackey staff 17M 11 13 10:57 codis-admin -rwxr-xr-x 1 jackey staff 18M 11 13 10:56 codis-dashboard -rw-r--r-- 1 jackey staff 5B 11 21 18:06 codis-dashboard.pid -rwxr-xr-x 1 jackey staff 16M 11 13 10:57 codis-fe -rw-r--r-- 1 jackey staff 5B 11 21 18:24 codis-fe.pid -rwxr-xr-x 1 jackey staff 15M 11 13 10:57 codis-ha -rwxr-xr-x 1 jackey staff 19M 11 13 10:57 codis-proxy -rw-r--r-- 1 Jackey staff 5B 11 21 18:08 COdis-proxys.pid -rwxr-xr-x 1 Jackey staff 1.1m 11 13 10:56 COdis-server -rwxr-xr-x 1 jackey staff 98K 11 13 10:56 redis-benchmark -rwxr-xr-x 1 jackey staff 161K 11 13 10:56 redis-cli -rwxr-xr-x 1 Jackey staff 1.1m 11 13 10:56 Redis-sentinel-RW-r --r-- 1 Jackey staff 170B 11 13 10:56 VersionCopy the code

Up to this point, our preparations have been completed. Let’s look at how to start the test cluster in a standalone environment.

  1. Start the codis – dashboard

Go to the admin directory and run the codis-dashboard-admin.sh script

$ ./codis-dashboard-admin.sh start /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/admin/.. /config/dashboard.toml starting codis-dashboard ...Copy the code

Then view logs to check whether the startup is successful

$ tail -100 .. /log/codis-dashboard.log.2018-11-21
2018/11/21 18:06:57 main.go:155: [WARN] option --pidfile = /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/bin/codis-dashboard.pid
2018/11/21 18:06:57 topom.go:429: [WARN] admin start service on [::]:18080
2018/11/21 18:06:57 fsclient.go:195: [INFO] fsclient - create /codis3/codis-demo/topom OK
2018/11/21 18:06:58 topom_sentinel.go:169: [WARN] rewatch sentinels = []
2018/11/21 18:06:58 main.go:179: [WARN] [0xc000374120] dashboard is working ...
Copy the code
  1. Start codes – proxy

Run the codis-proxy-admin.sh script

$ ./codis-proxy-admin.sh start /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/admin/.. /config/proxy.toml starting codis-proxy ...Copy the code

Check whether the startup is successful

$ tail -100 .. /log/codis-proxy.log.2018-11-21 2018/11/21 18:08:34 proxy_api.go:44: Xc0003262c0 [WARN] [0] API call/API/proxy/start / 212 d13827c84455d487036d4bb07ce15 from 10.1.201.43:58800 [] 2018/11/21 18:08:34 proxy_api.go:44: Xc0003262c0 [WARN] [0] API call/API/proxy/sentinels / 212 d13827c84455d487036d4bb07ce15 from 10.1.201.43:58800 [] 2018/11/21 18:08:34 proxy.go:293: [WARN] [0xc0003262c0]set sentinels = []
2018/11/21 18:08:34 main.go:343: [WARN] rpc online proxy seems OK
2018/11/21 18:08:35 main.go:233: [WARN] [0xc0003262c0] proxy is working ...
Copy the code
  1. Start the codis – server

Run the codis-server-admin.sh script

$ ./codis-server-admin.sh start /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/admin/.. /config/redis.conf starting codis-server ...Copy the code

Check whether the startup is successful

$tail -100 / TMP /redis_6379.log 12854:M 21 Nov 18:09:29.172 * Increased maximum number of open files to 10032 (it was originallyset to 256).
                _._                                                  
           _.-``__ ' '-) _ _) - ` ` `. ` _.' '- _ Redis 3.2.11 (de1ad026/0), 64 - bit. - ` `. - ` ` `. ` ` ` \ / _, _' '-. _ (' , .-` | `, ) Running in standalone mode |`-._`-... - ` __... -. ` ` - _ | '` _....'| Port: 6379 | `-._ `._ / _.-'    |     PID: 12854
  `-._    `-._  `-./  _.-'_. -| ` -) _ ` -) _ ` -. __. -'_. -_. -'| | ` -. _ ` -. _ _. -'_. -' | http://redis.io `-._ `-._`-.__.-'_. -'_. -| ` -) _ ` -) _ ` -. __. -'_. -_. -'| | ` -. _ ` -. _ _. -'_. -'| ` -) _ ` -) _ ` - __. -'_. -'_. -` - _ ` -. __. -'_. -` -. _ _. -'` -. __. -12854: M 21 Nov 18:09:29. 187Redis version 3.2.11
12854:M 21 Nov 18:09:29.187 * The server is now ready to accept connections on port 6379
Copy the code

If an error occurs, check whether the user has read and write permissions to the/TMP /redis_6379.log file.

I’ve started two instances of Codis to test Auto Rebalance. The method is simple: copy the admin/codis-server-admin.sh and config/redis.conf files respectively, modify the port information in the files, and then execute the new script in the same way.

  1. Start the codis – fe

Run the codis-fe-admin.sh script

$ ./codis-fe-admin.sh start

starting codis-fe ... 
Copy the code

Check whether the command is executed successfully

$ tail -100 .. /log/codis-fe.log.2018-11-21 
2018/11/21 18:24:33 main.go:101: [WARN] set ncpu = 4
2018/11/21 18:24:33 main.go:104: [WARN] setListen = 0.0.0.0:9090 2018/11/21 18:24:33 main.go:120: [WARN]setassets = /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/bin/assets 2018/11/21 18:24:33 main.go:162:  [WARN]set --filesystem = /tmp/codis
2018/11/21 18:24:33 main.go:216: [WARN] option --pidfile = /Users/jackey/Documents/go_workspace/src/github.com/CodisLabs/codis/bin/codis-fe.pid
Copy the code

Once you have all started successfully, you can go to http://127.0.0.1:9090 and start setting up the cluster.

  1. Add a group

We just started two CODIS-Servers, so we can new two groups and add coDIS-Server to each group

  1. Initialize the slot.

Initially, all slots are in offline state.

Click the Rebalance All Slots button below. Codis automatically assigns 1024 Slots to two groups of 512 each.

Of course, you can also manually assign slots, e.g. by clicking Migrate Some, you can assign 10 slots from group-1 to group-2.

summary

Codis’s dynamic capacity for expansion is excellent, but there are some problems (as we’ve already covered). So whether your cluster will use Codis depends on the specific requirements. Finally, kudos to the Codis development team for creating a distributed database called TiDB. Those of you who are interested can learn about it.