1. Redis first meeting

  • High-performance key-value server
  • Multiple data structures
  • Rich functionality
  • Highly available distributed support

1. What is Redis

  • Open source
  • Key-value based storage service system
  • Multiple data structures
  • High performance, rich function

2. Redis features

  • Speed is fast
  • persistence
  • Multiple data structures
  • Support for multiple programming languages
  • feature-rich
  • simple
  • A master-slave replication
  • Highly available, distributed

3. Redis features – fast speed

  • 10W OPS
  • Where is the data stored? – In memory
  • What language is written – C language
  • Thread model – Single thread

4. Redis feature – Persistence (power failure without data loss)

  • Redis keeps all data in memory and updates to data are asynchronously saved to hard disk.

5. Redis features – multiple data structures

  • String/Blobs/Bitmaps
  • Hash Tables(Object)
  • Linked Lists
  • Sets
  • Sorted Sets

6. Redis features – Rich in features

  • Release subscription
  • The Lua script
  • The transaction
  • pipeline

7. Typical usage scenarios of Redis

  • Caching system
  • counter
  • Message queue system
  • list
  • The social network
  • Real-time systems

8. Redis is installed and started

1. Redis executable file description

  • Redis -server -> redis server
  • Redis -cli -> redis command line client
  • Redis -benchmark -> Redis performance test tool
  • Redis-check-aof -> aof file repair tool
  • Redis-check-dump -> RDB file check tool
  • Redis-sentinel -> Sentinel server (after 2.8)

2. Redis executable file description

  • Three boot modes
    • The minimalist start
    • Dynamic parameter start
    • Configuration file startup
  • Compare the three boot modes
    • Production environment Select configuration startup
    • Single-machine multi-instance profiles can be separated by ports

3. Verify

  • ps -ef | grep redis
  • netstat -antpl | grep redis
  • redis -cli -h ip -p port -ping

9. Redis common configurations

  • Daemonize -> Whether it is a daemon process
  • Port -> Redis Specifies the external port number
  • Logfile -> Redis system log
  • Dir -> Redis working directory

10. Use the configuration file to start the system

[root @ localhost redis - 4.0.5]# mkdir config[root @ localhost redis - 4.0.5]# cp redis.conf config/
Copy the code
  • Remove all comments and Spaces and redirect the configuration.
[root@localhost config]# cat redis.conf | grep -v "#" | grep -v "^$" > redis-6382.conf
Copy the code
  • Enter the configuration file. Delete The configuration file
[root@localhost config]# vi redis-6382.conf

port 6382
dir "/usr/local/redis/data"
logfile "6382.log"
daemonize yes
Copy the code
  • Start the client and view the logs
[root@localhost redis]# redis-server config/redis-6382.conf

[root@localhost redis]# ps -ef | grep redis-server | grep 6382
root       8447      1  0 09:49 ?        00:00:00 redis-server *:6382

[root@localhost redis]# cd data
[root@localhost data]# llTotal usage 4-rw-r --r--. 1 root root 1354 12月 20 09:49 6382.logCopy the code

2. Understanding and using API

1. General commands

  • Keys: Traverses all keys.
    • Generally not used in the production environment.
    • How to use keys: hot standby slave node. Scan.
  • Dbsize: Calculates the total number of keys.
  • Exists key: Checks whether the key exists.
  • del key[key …] : Deletes the specified key-value
  • Expire key seconds: The key expires after seconds.
    • TTL key: Checks the remaining expiration time of the key.
    • Persist key: deletes the expiration time of the key.
  • Type Key: Return key type.
The command Time complexity
keys O(n)
dbsize O(1)
del O(1)
exists O(1)
expire O(1)
type O(1)

2. Data structure and internal coding

  • Data structures can be implemented in different ways: memory is expensive, and different internal encodings may be used to achieve time-for-space and space-for-time effects in different cases.
  • In fact, the redis source code has a redisObject, or structure, that has a number of properties:
    • Data type (Type)
    • Encoding Method
    • Data pointer (PTR)
    • Virtual Memory (VM)
    • Other information
  • The user does not need to know the internal encoding, which is actually a practice of interface oriented programming.

3. Single thread

  • Redis will only execute one command at a time.
  • Why is a single thread so fast?
    • Pure memory
    • Non-blocking IO
    • Avoid thread switching and static consumption
  • What do you notice about single threads?
    • Run only one command at a time
    • Refused to long (slow) command: keys, flushall, flushdb, missile lua script, mutil/exec, operate the big value (collection)
    • It’s not a single thread: fysNC file Descriptor,close file Descriptor

4. The string

1. Structure and commands

  • Usage Scenarios:
    • The cache
    • counter
    • A distributed lock
  • get set del
    • Get key: obtains the value of the key
    • Set key value: sets key-value
    • Del key: deletes key-value
  • incr decr incrby decrby
    • Incr key: The key is incremented by 1. If the key does not exist, get(key)=1 after the incremented
    • Decr key: decreases the key by 1. If the key does not exist, get(key)=-1 after the increment
    • Incrby key k: the key is automatically added with k. If the key does not exist, get(key)= K
    • Decrby key k: the key decreases by k. If the key does not exist, get(key)=-k after the increment
  • set setnx set xx
    • Set Key value: Sets whether the key exists or not
    • Setnx key Value: Set this parameter only when the key does not exist
    • Set key value xx: Set this parameter only when the key exists
  • mget mset
    • mget key1 key2 key3… : Obtain keys in batches, atomic operation
    • Mset key1 value1 key2 value2 key3 value3: sets key-value in batches

2. Fast combat

  • Achieve the following functions: record the page views of each user on the website
    • Incr userId: pageView
  • Implement the following functions: cache the basic information of the video (data source in MySQL) pseudo code
public VideoInfo get(long id) {
    String redisKey = redisPrefix + id;
    VideoInfo viderInfo = redis.get(redisKey);
    if (videoInfo == null) {
        videoInfo = mysql.get(id);
        if(videoInfo ! = null) {// serialize redis. Set (redisKey, serialize(videoInfo)); }}return videoInfo;
}
Copy the code
  • The following functions are implemented: Distributed ID generator
    • Incr ID (Atomic operation)

3. Check for gaps

  • getset append strlen
    • Getset Key newValue: Set key newValue and return the old value
    • Append Key Value: Appends the value to the old value
    • Strlen key: Returns the length of the string
  • incrbyfloat getrange setrange
    • Incrybyfloat key 3.5: Increases the value of key 3.5
    • Getrange: Gets all values of the specified subscript of the string
    • Setrange: Sets all values of the specified subscript
The command meaning Time complexity
set key value Example Set key-value O(1)
get key value Get key-value O(1)
del key value Example Delete key-value O(1)
setnx set xx Set key-value based on whether the key exists O(1)
incr decr count O(1)
mget mset Example Run the key-value command in batches O(n)

5. The hash

1. Structure and commands

  • All hashing commands begin with an H
  • hget hset hdel
    • Hget key field: obtains the value of the field corresponding to the hash key
    • Hget Key field value: Sets the value of the hash key field
    • Hget key field: deletes the value of the hash key field
  • hexists hlen
    • Hexists Key field: determines whether the hash key has a field
    • Hlen key: Gets the number of hash key fields
  • hmget hmset
    • hmget key field1 field2 … FieldN: obtains hash key field values in batches
    • hmset key field1 value1 field2 value2 … FieldN valueN: Sets the hash key field values in batches
  • hgetall hvals hkeys
    • Hgetall Key: Returns all fields and values corresponding to the Hash key
    • Hvals key: returns the values of all fields corresponding to the Hash key
    • Hkeys key: Returns all fields corresponding to the Hash key

2. Fast combat

  • Achieve the following functions: record the page views of each user on the website
    • hincrby user:1:info pageview count
  • Implement the following functions: cache the basic information of the video (data source in MySQL) pseudo code
public VideoInfo get(long id) {
    String redisKey = redisPrefix + id;
    Map<String, String> hashMap = redis.hgetAll(redisKey)
    VideoInfo viderInfo = transferMapToVideo(hashMap);
    if (videoInfo == null) {
        videoInfo = mysql.get(id);
        if (videoInfo != null) {
            redis.hmset(redisKey, transferVideoToMap(videoInfo));
        }
    }
    return videoInfo;
}
Copy the code

3. Check for gaps

  • string vs hash
    • A similar API
string hash
get hget
set setnx hset hsetnx
del hdel
incr incyby decr decrby hincrby
mset hmset
mget hmget
  • How do I update user attributes
  • The string to achieve
    • key : user:1
    • value : (serializable:json,xml,protobuf) {“id”:1, “name”:”ronaldo”, “age”:40, “pageView”:5000000}
  • The string to achieve
    • key : user:1:name user:1:age user:1:pageView
    • value : world 40 500000
  • Hash implementation
    • key : hset user:1:info age 41
    • value :
The command advantages disadvantages
string v1 Easy to program and may save memory Serialization overhead, setting properties to do the whole data wrong
string v2 Intuitive, can be partially updated Large memory usage and scattered keys
hash Intuitive, space saving, can be partially updated Programming is slightly complicated, TTL is not easy to control
  • hsetnx hincrby hincrbyfloat
    • Hsetnx key field Value: Sets the value of the field corresponding to the hash key (fails if the field already exists)
    • Hincrby Key field intCounter: Indicates that the value of the field corresponding to the hash key increases intCounter
    • Hincrbyfloat Key Field floatCounter: HincrBY floating-point version
The command Time complexity
hget hset hdel O(1)
hexists O(1)
hincrby O(1)
hgetall hvals hkeys O(1)
hmget hmset O(n)

A list of 6.

1. Structure and commands

  • The characteristics of
    • The orderly
    • Can be repeated
    • Left and right side insert pop up
  • Linked list apis all start with L
  • rpush
    • rpush key value1 value2 … ValueN: Insert values from the right end of the list (1-N)
  • lpush
    • lpush key value1 value2 … ValueN: Inserts values from the left side of the list (1-N)
  • linsert
    • Linsert key before | agter value newValue: the values specified in the list before | after insert newValue
  • lpop
    • Lpop Key: Pops an item from the left side of the list
  • rpop
    • Rpop Key: Pops an item from the right side of the list
  • lrem
    • Lrem key count value: Deletes all entries with equal value from the list based on the count value
      • If count>0, delete at most entries whose value is equal from left to right
      • Math.abs(count) delete Max math.abs (count) entries with equal value from right to left
      • If count=0, delete all value entries
  • ltrim
    • Ltrim Key start end: Trim the list based on the index range
  • lrange
    • Lrange key start end(including end) : Gets all items in the specified index range
  • lindex
    • Lindex key index: gets the specified index item of the list
  • llen
    • Llen key: Obtains the list length
  • lset
    • Lset Key index newValue: Sets the specified index value of the list to newValue

2. Fast combat

  • Weibo – TimeLine
    • People you follow update their tweets, LPUSH

3. Check for gaps

  • blpop brpop
    • Blpop Key timeout: lPOP blocked version. Timeout is the blocking timeout period. Timeout =0 indicates that lPOP will never be blocked
    • Brpop Key timeout: rPOP blocked version. Timeout is the blocking timeout period. Timeout =0 indicates that rPOP will never be blocked
  • TIPS
    • LPUSH + LPOP = Stack
    • LPUSH + RPOP = Queue
    • LPUSH + LTRIM = Capped Collection
    • LPUSH + BRPOP = Message Queue

Collection of 7.

1. Structure and commands

  • The characteristics of
    • A disorderly
    • No repeat
    • Interset operation
  • All apis start with an S
  • sadd srem
    • Sadd Key Element: Add element to set key (if element already exists, add fails)
    • Srem key Element: Removes an element from the set key
  • scard sismember srandmember smembers
    • Scard key: Count the number of elements in the set
    • Sismember Key Element: Determines whether the element is in the collection
    • Srandmember Key count: Fetch count randomly from the set
    • Smembers key: Fetches all elements of an element
  • sdiff sinter sunion
    • Sdiff key1 Key2: set difference
    • Sinter key1 key2: set intersection
    • Sunion key1 KEY2: indicates the set union
    • Sdiff | sinter | sunion + store destkey: difference set, intersection, and stored in destkey set of results

2. Fast combat

  • Lottery system
  • I Like it very much
  • Label (tag)
  • Mutual concern

3. Check for gaps

  • TIPS:
    • SADD = Tagging
    • SPOP/SRANDMEMBER = Random item
    • SADD + SINTER = Social Graph

8. Ordered collections

1. Structure and commands

  • All apis start with Z
  • zadd
    • Zadd Key Score Element (can be multiple pairs) : Add score and Element
  • zrem
    • Zrem Key Element (can be multiple pairs) : Deletes elements
  • zscore
    • Zscore key Element: Returns the score of the element
  • zincrby
    • Zincrby key increScore Element: Increases or decreases the score of an element
  • zcard
    • Zcard key: Returns the total number of elements
  • zrange
    • Zrange key start end [WITHSCORES] : returns the ascending element within the specified index range.
  • zrangebyscore
    • Zrangebyscore key minScore maxScore [WITHSCORES] : returns the ascending element within the specified score range.
  • zcount
    • Zcount key minScore maxScore: Returns the number of scores within the specified range in the ordered set
  • zremrangebyrank
    • Zremrangebyrank key start end: Deletes ascending elements in the specified rank
  • zremrangebyscore
    • Zremrangebyscore key minScore maxScore: deletes the ascending elements from the specified score

2. Fast combat

  • list

3. Check for gaps

  • zrevrank
  • zrevrange
  • zvevrangebyscore
  • zinterstore
  • zunionstore

3. Redis client

1. Java client: Jedis

// Generate a Jedis object that communicates with the specified Redis node. Jedis jedis = new Jedis("127.0.0.1", 6379); / / jedis executionsetOperating jedis. Set ("hello"."world"); String value = jedis.get("hello");
Copy the code
  • Jedis(String host, int port, int connectionTimeout, int soTimeout)
    • Host: indicates the IP address of the Redis node
    • Port: indicates the port of the Redis node
    • ConnectionTimeout: Client connection times out
    • SoTimeout: Read/write timeout of the client
  • Jedis directly connected
    • Generate a Jedis object
    • Jedis executes the command
    • Return execution result
    • Close the Jedis connection
  • Jedis connection pool
    • Borrow a Jedis object from the resource pool
    • Jedis executes the command
    • Return execution result
    • Return the Jedis object to the connection pool

4. Some other features of Redis

1. Slow query

  • Slow queries occur in phase 3.
  • Client timeouts are not necessarily slow queries, but slow queries are a possible factor in client timeouts.

Slowlog-max-len: what is the length of our slow query queue

  • Fifo queue
  • Fixed length
  • Save it in memory

2. Two configurations – slowlog-log-slower than: What is considered slow when the query time is less than

  • Slow query threshold (in microseconds)
  • Slowlog-log-slower than = 0, record all commands
  • Slowlog-log-slower than < 0, no commands are recorded

3. Configuration method

  • The default value
    • config get slowlog-max-len = 128
    • config get slowlog-log-slower-than = 10000
  • Modify the configuration file to restart
  • Dynamic configuration

4. Slowly query commands

  • Slowlog Get [n] : Slowlog query queue is obtained
  • Slowlog len: obtains the length of the slow query queue
  • Slowlog Reset: clears slow query queues

5. Operation and maintenance experience

  • Do not set slowlog-max-len to too small, usually around 1000.
  • Slowlog-log-slower than do not set too much, default 10ms, usually 1ms.
  • Understand the command life cycle.
  • Persistent periodic slow queries.

2. pipeline

  • 1 time = 1 network time + 1 command time
  • N times = N network times + N command times

1. What is assembly line

  • A batch of commands is packaged in bulk, computed in bulk on the server, and the results are returned to us in order.
  • 1 pipeline(N commands) = 1 network time + N command time
  • Pay attention to two points
    • Redis command times are all microseconds.
    • To control the number of pipelines per run (network).

2. The pipeline – Jedis implementation

public void PipelineTest() {
    Jedis jedis = new Jedis("192.168.242.129", 6379);
    for (int i = 0; i < 100; i++) {
        Pipeline pipeline = jedis.pipelined();
        for (int j = i * 100; j < (i + 1) * 100; j++) {
            pipeline.hset("hashkey:" + j, "field" + j, "value"+ j); } pipeline.syncAndReturnAll(); }}Copy the code

3. Operate with native M?

  • M operations are atomic operations
  • Pipelines are not atomic operations

4. Usage suggestions

  • Note the amount of data carried per Pipeline.
  • Pipeline can only work on one Redis node at a time.
  • The difference between M operation and Pipeline.

3. Publish subscriptions

1. The role

  • The publisher (publisher)
  • The subscriber (the subscriber)
  • Channel (channel)

2. API

  • Publish
    • publish channel message
  • The subscribe (subscription)
    • Subscribe [channel] # One or more
  • Unsubscribe
    • Unsubscribe [channel] # One or more

3. Message queues

  • Publish subscribe features that all other subscribers can get the message after it is published.
  • Message queuing is a grab process in which a publisher publishes a message and only one subscriber gets it. (Redis does not provide this functionality.)

4. Publish a subscription summary

  • Roles in publish subscribe schema
  • Important API

4. Bitmap

127.0.0.1:6379 >setHello big OK 127.0.0.1:6379> getbit hello 0 (integer) 0
127.0.0.1:6379> getbit hello 1
(integer1)Copy the code
  • For Redis, I can manipulate bits directly.

1. The relevant API

  • setbit
    • Setbit key offset value: specifies the index value for the bitmap
127.0.0.1:6379> setbit hello 7 1
(integer) 0
127.0.0.1:6379> get hello
"cig"
Copy the code
  • getbit
    • Getbit key offset: gets the specified offset
  • bitcount
    • Bitcount key [start end] : Retrieves the number of bitmap bits with a value of 1 in the specified range (start to end, if not specified)
  • bitop
    • bitop op destkey key [key …] : Perform and(intersection), OR (union), NOT (not), xOR (xor) operations on multiple bitmaps and save the result in destkey
  • bitpos
    • Bitpos key targetBit [start] [end] : Computes the position where the first offset is equal to the targetBit in the specified bitmap range (start to end, in bytes, if not specified)

2. Independent user statistics

  • Use sets and bitmaps
  • 100 million users, 50 million unique visitors a day
The data type Each userID takes up space The number of users to be stored Total memory
set 32-bit (integer) 50000000 200MB
bitmap 1 a 100000000 12.5 MB

3. Use experience

  • Type = string. The maximum value is 512 MB
  • Note the offset in setbit, which can be time-consuming
  • Bitmaps are not always good

5. HyperLogLog

1. Based on HyperLogLog algorithm: complete independent quantity statistics in minimal space

  • It’s essentially a string

2. Three commands

  • pfadd key element [element …] : Adds elements to hyperLogLog
  • pfcount key [key …] : Calculates the independent total of hyperloglog
  • pfmerge destkey sourcekey [sourcekey …] : Merges multiple Hyperloglogs

3. Use experience

  • Are you tolerant of mistakes? (Error rate: 0.81%)
  • Do you need a single piece of data?

6. GEO

1. GEO(geographic information positioning) : store longitude and latitude, calculate the distance between two places, calculate the range, etc

2. Application scenarios

  • shake
  • Surrounding restaurants, hotels

3. API

  • geoadd
    • geo key longitude latitude member [longitude latitude member …] : Adds location information
  • geopos
    • geopos key member [member …] : Get location information
  • geodisk
    • Geodisk key member1 member2 [unit] : Get the distance between two geographical locations (unit: m…)
  • georadius

4. Relevant instructions

  • Since 3.2 +
  • type geoKey = zset

The last

You can follow my wechat public number to learn and progress together.