Analyze and explore the serialization and deserialization + generics mechanism of RedisTemplate

Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

The premise is introduced

The last article introduced Java’s many tools and excellent frameworks for serialization. Now we combine these excellent frameworks for serialization and deserialization.

Redis serialization and deserialization

Redis stores content in binary/string form at the bottom;

serialization

Convert Java objects to binary/strings and store them in memory;

deserialization

Read binary/strings in memory and convert them to Java objects;

RedisTemplate generics

Usually use

RedisTemplate<String, String> indicates that the operation key and val are both strings.

@Resource
RedisTemplate<String, String> redisTemplate;

public void doTest(a){
  redisTemplate.opsForValue().setIfAbset("key"."val", Duration.ofSecend(100));
}
Copy the code

If val is Integer

@Resource
RedisTemplate<String, Integer> redisTemplate;
public void doTest(a){
  redisTemplate.opsForValue().setIfAbset("key".100, Duration.ofSecend(100));
}
Copy the code

This might be an error because of the serialization operation when SpringBoot accesses Redis.

Serializer Serializer

Springboot interacts with Redis in binary mode (byte[]). In order to support data types in Java, the object (key, value, hashKey, hashValue… Do serialization. RedisTemplate Sets serializer for key value hashKey hashValue only

Springboot provides several serializations

• JdkSerializationRedisSerializer (default)
• StringRedisSerializer
• Other or custom

JdkSerializationRedisSerializer

public byte[] serialize(@Nullable Object object)
Copy the code

StringRedisSerializer

Using StringRedisSerializer only supports string, so if you use RedisTemplate< string, Integer> will report an error.

public byte[] serialize(@Nullable String string)
Copy the code

• If use JdkSerializationRedisSerializer, not only supports RedisTemplate < String, Integer >, also can support any custom type RedisTemplate < String, Person >.

• However, this default serialization method results in poor readability of keys and values stored in Redis, with some unreadable hexadecimal characters

RedisTemplate serialization and deserialization implementation

RedisSerializer Specifies the properties of the serialization component

AfterPropertiesSet is called to initialize the property. The redis command will determine whether the property has been initialized
private boolean initialized = false;

// Enable default serialization
private boolean enableDefaultSerializer = true;

// The default serializer
private @NullableRedisSerializer<? > defaultSerializer;// Key serializer
@SuppressWarnings("rawtypes") 
private @Nullable RedisSerializer keySerializer = null;

//value serializer
@SuppressWarnings("rawtypes") 
private @Nullable RedisSerializer valueSerializer = null;

// Hash key serializer, used in hash keys and stream fields
@SuppressWarnings("rawtypes")
private @Nullable RedisSerializer hashKeySerializer = null;

Hash Value serializer, used in hash value and stream value
@SuppressWarnings("rawtypes") 
private @Nullable RedisSerializer hashValueSerializer = null;

// String serializer that serializes channels when Redis publishes order mode messages
private RedisSerializer<String> stringSerializer = RedisSerializer.string();

// Operation type string
private final ValueOperations<K, V> valueOps = new DefaultValueOperations<>(this);

// Operate on the list type
private final ListOperations<K, V> listOps = new DefaultListOperations<>(this);

// Operate the set type
private final SetOperations<K, V> setOps = new DefaultSetOperations<>(this);

// Operate on the stream type
private finalStreamOperations<K, ? ,? > streamOps =new DefaultStreamOperations<>(this.new ObjectHashMapper());

// Operate the zset type
private final ZSetOperations<K, V> zSetOps = new DefaultZSetOperations<>(this);

// Operate the geo location type
private final GeoOperations<K, V> geoOps = new DefaultGeoOperations<>(this);

// Operate the hyperLogLog type
private final HyperLogLogOperations<K, V> hllOps = new DefaultHyperLogLogOperations<>(this);

// Run the cluster command
private final ClusterOperations<K, V> clusterOps = new DefaultClusterOperations<>(this);

@Override
public <T> T execute(SessionCallback<T> session) {
    //** After executing the redis command, the system will determine whether it has been initialized. AfterPropertiesSet is required for initialization
	Assert.isTrue(initialized, "template not initialized; call afterPropertiesSet() before using it"); . }Copy the code

AfterPropertiesSet initializes the serialization property

@Override
public void afterPropertiesSet(a) {
    //** Call the afterPropertiesSet method of the parent class to determine whether to initialize the Redis connection factory
	super.afterPropertiesSet();

	boolean defaultUsed = false;
    //** If the default serializer is not set, use the JDK serializer to set the default serializer
	if (defaultSerializer == null) {
		defaultSerializer = newJdkSerializationRedisSerializer( classLoader ! =null ? classLoader : this.getClass().getClassLoader());
	}

	if (enableDefaultSerializer) {
        //** If default serialization is enabled and no key serializer is set, the default serializer is used as the serializer for key
		if (keySerializer == null) {
			keySerializer = defaultSerializer;
			defaultUsed = true;
		}
		//** If default serialization is enabled and value serializer is not set, the default serializer is used as value serializer
		if (valueSerializer == null) {
			valueSerializer = defaultSerializer;
			defaultUsed = true;
		}
		//** If default serialization is enabled and no key serializer is set, the default serializer is used as the hash key serializer
		if (hashKeySerializer == null) {
			hashKeySerializer = defaultSerializer;
			defaultUsed = true;
		}
		//** If default serialization is enabled and no value serializer is set, the default serializer is used as the hash value serializer
		if (hashValueSerializer == null) {
			hashValueSerializer = defaultSerializer;
			defaultUsed = true; }}//** If default serialization is enabled, the default serializer cannot be null
	if (enableDefaultSerializer && defaultUsed) {
		Assert.notNull(defaultSerializer, "default serializer null and not all serializers initialized");
	}
    //** If the redis script executor is not set, set the redis script executor to DefaultScriptExecutor
	if (scriptExecutor == null) {
		this.scriptExecutor = new DefaultScriptExecutor<>(this);
	}
    //** The Settings are initialized
	initialized = true;
}
//** Sets the default serializer
public void setDefaultSerializer(RedisSerializer
        serializer) {
	this.defaultSerializer = serializer;
}
Copy the code

RedisTemplate use

• To perform writing, the corresponding serializer is first called to serialize the key/value into binary code and save it in Redis.
• To perform reading, first obtain the binary code of value based on key, and call the value serializer to reverse sequence it into a Java object

String serialization and deserialization

First call DefaultValueOperations set/get methods save/key/value pair, create an anonymous inner class implements ValueDeserializingRedisCallback, then call redistemplate the execute method

RawKey Serialization key

The key serializer is used to convert the key to binary

@SuppressWarnings("unchecked")
private byte[] rawKey(Object key) {
	Assert.notNull(key, "non null key required");
	if (keySerializer == null && key instanceof byte[]) {
		return (byte[]) key;
	}
	return keySerializer.serialize(key);
}
Copy the code

RawValue Serialization key

Use the value serializer to convert a value to binary

@SuppressWarnings("unchecked")
private byte[] rawValue(Object value) {
	if (valueSerializer == null  && value instanceof byte[]) {
		return (byte[]) value;
	}
	return valueSerializer.serialize(value);
}
Copy the code

Set to save

@Override
public void set(K key, V value) {
    //** Convert value to binary using value serializer
	byte[] rawValue = rawValue(value);
	execute(new ValueDeserializingRedisCallback(key) {
        //** Save the binaries of the serialized key and value to redis
		@Override
		protected byte[] inRedis(byte[] rawKey, RedisConnection connection) {
			connection.set(rawKey, rawValue);
			return null; }},true);
}
Copy the code

Get access to

Create an anonymous inner class implements ValueDeserializingRedisCallback, then call redistemplate the execute method

@Override
public V get(Object key) {
	return execute(new ValueDeserializingRedisCallback(key) {
        // Get the binary code from the serialized key and value
		@Override
		protected byte[] inRedis(byte[] rawKey, RedisConnection connection) {
			returnconnection.get(rawKey); }},true);
}
Copy the code

Execute execution method

Redistemplate the execute method call ValueDeserializingRedisCallback doInRedis method, called ValueDeserializingRedisCallback doInRedis method, Returns the result of the execution.

@Nullable
public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {... T result = action.doInRedis(connToExpose); . }Copy the code

InRedis executes the method

ValueDeserializingRedisCallback doInRedis method call inRedis method, call inRedis method, the key and the value of serialized binary code stored in redis (set method will return null), Deserialize binary values into Java objects using the Value serializer.

public final V doInRedis(RedisConnection connection) {
	byte[] result = inRedis(rawKey(key), connection);
	// Use the value serializer to deserialize binary values into Java objects
	return deserializeValue(result);
}

@SuppressWarnings("unchecked")
V deserializeValue(byte[] value) {
	if (valueSerializer() == null) {
		return (V) value;
	}
	return (V) valueSerializer().deserialize(value);
}
Copy the code

Hash serialization and deserialization

The Redistemplate does not set a member attribute for the hash type

@Override
public <HK, HV> HashOperations<K, HK, HV> opsForHash(a) {
	return new DefaultHashOperations<>(this);
}
Copy the code

To obtain

We first call the Put /get methods of DefaultHashOperations to save /get the key-value pairs.

1. The key serializer is used to convert the key to binary
2. Convert a hashKey to binary using the Hash key serializer
3. Lambda expressions implement the RedisCallback interface and then call the Execute method of the redistemplate to get the value binary based on the key and hashKey
4. Deserialize the binary value into a Java object using the Hash Value serializer

@Override
@SuppressWarnings("unchecked")
public HV get(K key, Object hashKey) {
	byte[] rawKey = rawKey(key);
	byte[] rawHashKey = rawHashKey(hashKey);
	byte[] rawHashValue = execute(connection -> connection.hGet(rawKey, rawHashKey), true);
	return(HV) rawHashValue ! =null ? deserializeHashValue(rawHashValue) : null;
}
Copy the code

save

1. The key serializer is used to convert the key to binary
2. Convert a hashKey to binary using the Hash key serializer
3. Use the Hash Value serializer to convert the value to binary
4. Lambda expressions implement the RedisCallback interface and then call the execute method of the redisTemplate to save the serialized binaries of keys, hashkeys, and values into redis

@Override
public void put(K key, HK hashKey, HV value) {
	byte[] rawKey = rawKey(key);
	byte[] rawHashKey = rawHashKey(hashKey);
	byte[] rawHashValue = rawHashValue(value);
	execute(connection -> {
		connection.hSet(rawKey, rawHashKey, rawHashValue);
		return null;
	}, true);
}
Copy the code

Convert a hashKey to binary using the Hash key serializer

@SuppressWarnings("unchecked")
<HK> byte[] rawHashKey(HK hashKey) {
	Assert.notNull(hashKey, "non null hash key required");
	if (hashKeySerializer() == null && hashKey instanceof byte[]) {
		return (byte[]) hashKey;
	}
	return hashKeySerializer().serialize(hashKey);
}
Copy the code

Use the Hash Value serializer to convert the value to binary

@SuppressWarnings("unchecked")
<HV> byte[] rawHashValue(HV value) {

	if (hashValueSerializer() == null && value instanceof byte[]) {
		return (byte[]) value;
	}
	return hashValueSerializer().serialize(value);
}
Copy the code

Deserialize the binary value into a Java object using the Hash Value serializer

@SuppressWarnings("unchecked")
<HV> HV deserializeHashValue(byte[] value) {
	if (hashValueSerializer() == null) {
		return (HV) value;
	}
	return (HV) hashValueSerializer().deserialize(value);
}
Copy the code

The execute method of Redistemplate calls the doInRedis method of the RedisCallback interface

Custom serialization

• JdkSerializationRedisSerializer although in redis saved data is unreadable, but it is convenient to operation and can specify the type of the return value, directly from the conversion of trival again. The principle is that the data stored in Redis contains data types.

• In Redis, data does not hold type information. By specifying the type of value for template, the desired type value is obtained. Use StringRedisSerializer, but can achieve JdkSerializationRedisSerializer effect.

@Resource
RedisTemplate<String,Integer> tplInt;

@Resource
RedisTemplate<String,Person> tplPerson;

public void testGet(a){
  Integer x = tplInt.get("valOfX");
  Person p = tplPerson.get("valOfPerson");
}
Copy the code

• Redis returns byte[], which requires a serializer for deserialization.

• The Serializer in the RedisTemplate must bea bean, that is, an instantiated object.

To implement the RedisSerializer interface, this object must be bound to a fixed type. If it is a String, it cannot be an Integer. So you can’t pass it in on demand.

• StringRedisSerializer Specifies RedisSerializer

• RedisSerializer JdkSerializationRedisSerializer implementation

  The latter is set to Object in order to be compatible with all types, and the deserialized data is an Object, thus losing all the original information. So if you want to return an external required type, you can only cast the value once after serialization. The essential logic is as follows:

  public Object get(String key){
    //get data from redis
    return (Object)(new Person())
  }
  Person p = (Person)get();
  Copy the code

  The Serialization of Springboot can be customized and matched by itself. Such as

  // StringRedisSerializer is used as the key class by default
  template.setDefaultSerializer(new StringRedisSerializer());
  
  // Support complex types for value
  JdkSerializationRedisSerializer jdkSerializer = newJdkSerializationRedisSerializer(); template.setValueSerializer(jdkSerializer); template.afterPropertiesSet(); So key, hashKey, hashValue are readable. Value to support complex typesCopy the code