In the work may encounter a single point Redis to Redis cluster data migration problem, but can not always trouble operation and maintenance to do. In order to facilitate the development of their own migration data, I have written a simple Redis migration tool here, I hope to be useful to those who need it.

This tool supports:

  • Single point Redis to single point Redis migration
  • Single point Redis to Redis cluster migration
  • Redis cluster to Redis cluster
  • Redis cluster to single point Redis migration

The tool has been compiled into multi-platform commands, which can be executed directly by downloading binaries from Github. Project address: https://github.com/icowan/redis-tool, pull down the code to execute commands directly after the make can compile multiple platforms executable file, need to rely on golang compiler.

  • Windows amd64: redis-tool-windows-amd64.exe
  • MacOS amd64: redis-tool-darwin-amd64
  • Linux amd64: redis-tool-linux-amd64
  • Linux arm64: redis-tool-linux-arm64

View the usage method:

	$ chmod a+x redis-tool-linux-amd64

	$ ./redis-tool-linux-amd64 -h
	
Copy the code

Supported data types

  • String string
  • Hash Hash list
  • List the list
  • Sorted -set ordered set

How to use

/redis-tool-linux-amd64 -h You can view all the functions supported by the tool.

$./redis-tool- Darwin -amd64 migrate -h Usage: redis-tool migrate [command] Examples [hash, set, sorted-set, list] Available Commands: All migrate all hash hash list migration list migration set redis set migration sorted-set sorted set migration Flags: -h, --help help for migrate --source-auth string Source password --source-database int source database --source-hosts string Source redis address, Multiple IP addresses are separated by commas (,) (default "127.0.0.1:6379") --source-prefix string Source redis prefix --source-redis-cluster Whether source redis is a cluster --target-auth String Target password --target-database int Target database --target-hosts string Target redis address, Multiple IP addresses are separated by commas (,) (default "127.0.0.1:6379") --target-prefix string Target redis prefix --target-redis-cluster Whether the target redis is cluster Use "redis-tool migrate [command] --help" for more information about a command.Copy the code

Parameter Description:

  • –source-auth: source redis password, if any
  • –source-database: source database, default is 0
  • –source-hosts: specifies the source redis address. Separate multiple IP addresses in the cluster with commas (,). (default “127.0.0.1:6379”)
  • –source-prefix: indicates the prefix of source redis
  • –source-redis-cluster: indicates whether the source redis is a cluster. The default value is false
  • –target-auth: specifies the redis password of the target to be migrated, if any
  • –target-database: migrates the target database. Default is 0
  • –target-hosts: specifies the redis address of the migration target. Use ‘,’ to separate multiple IP addresses in the cluster (default “127.0.0.1:6379”).
  • –target-prefix: indicates the migration target redis prefix. This parameter is optional
  • –target-redis-cluster: specifies whether the target redis is in a cluster. The default value is false

Migrate data for a single key

Here are just two examples, and the rest are not too different.

Hash type

You can run the redis-tool migrate hash -h command to view the usage description

$redis-tool Migrate hash helloworld \ --source-hosts 127.0.0.1:6379 \ --target-redis-cluster true \ --target-hosts 127.0.0.1:6379127.00 0.1: \ 7379 - target - auth 123456Copy the code

An ordered set

You can run the redis-tool migrate sorted-set -h command to view the instructions. The ordered set may have a large amount of data. Therefore, the sorted set is cut in the unit of 50000. Here I have tested migrating nearly 17 million pieces of data in over 40 minutes.

$redis-tool Migrate hash helloworld \ --source-hosts 127.0.0.1:6379 \ --target-redis-cluster true \ --target-hosts 127.0.0.1:6379127.00 0.1: \ 7379 - target - auth 123456Copy the code

Migrating all key data supports wildcard filtering

You can run the redis-tool migrate all -h command to view the usage description

$redis-tool Migrate all "ipdetect:*" \ --source-hosts 127.0.0.1:6379 \ --target-redis-cluster true \ --target-hosts 127.0.0.1:6379127.00 0.1: \ 7379 - target - auth 123456Copy the code

This command compiles all matched keys and migrates them step by step based on the key type.

The tail

A relatively simple tool written using Golang is mainly used when Redis are not persisted or multiple sets of Redis migrate to one set of Redis.

Hope to be useful to you, thank you!

Author: Wang Cong, Creditease Institute of Technology