above
This is the first time for the author to publish a paper, maybe there are some improper expression and typesetting, hope readers forgive. The purpose of the post is to discuss with you and make some records. I hope you can find mistakes in the process of reading, please kindly comment and correct them. Thank you very much
This article related the super simple demo git address: [email protected]: Kwin1113 / redis_abstract git
The body of the
Caching should be a cliche in back-end projects, and when it comes to improving interface concurrency and optimizing project performance, it’s probably the first thing that comes to mind. Whether it is interview questions or daily development, are often encountered. In recent development, I ran into a problem.
The scenario looks like this: a simple query interface, the relevant business logic is to query the data from the database, and then perform a simple processing of the data before paging back, and the service layer returns the results cached using Redis. For business reasons, data paging is not carried out directly through the DAO layer, but is done in the business layer after the data is queried, and the relevant paging tool class is an abstract class written by itself.
The problem was that the utility class was instantiated not through subclass inheritance but through the double curly braces of an anonymous inner class. Part of the code is shown below, where irrelevant related code has been hidden.
@Data
public abstract class PageEntity<T> {
/** Paging related fields */
private Integer currentPage, pageSize, pageNum, total, index;
/** Sort related fields */
private String orderBy; private OrderType order; / * * * / data private List<T> data; public abstract Class<T> findTClass(a); / * ** ordering * @paramOrderByProperty Sort field* / private void sort(String orderByProperty) { // Sort by reflecting the get method corresponding to the sort field in the class... {@code #findTClass()} {@code #findTClass()} {@code #findTClass()}} } /** Sort type enumeration class */ public enum OrderType { ASC, DESC,; } } Copy the codeThe relevant business codes are simplified as follows:
@Service
public class EventService {
private final RedisTemplate<String, Object> redisTemplate;
public EventService(RedisTemplate<String, Object> redisTemplate) { this.redisTemplate = redisTemplate; } public PageEntity<Event> getEvent(a) { String key = "pageEntity"; PageEntity<Event> result = (PageEntity<Event>) redisTemplate.opsForValue().get(key); if (null == result) { result = new PageEntity<Event>() { @Override public Class<Event> findTClass(a) { return Event.class; } }; // Query data - simple processing // result.setData(datas); redisTemplate.opsForValue().set(key, result); } return result; } } Copy the codeHere’s a simple test class:
@SpringBootTest
@RunWith(SpringRunner.class)
public class EventServiceTest {
@Resource
private EventService eventService; @Test public void abstractServiceMethod(a) { PageEntity<Event> event = eventService.getEvent(); Assert.assertEquals(event.toString(), Event.class.getCanonicalName()); } } Copy the codeThis process should be easy to understand, is the simplest query data interface. Execute a single test and it will obviously succeed.
The first time the logic is executed and the associated pageEntity class is cached in Redis, it should be fine, so let’s do a single test again.
However, this time it failed, so let’s look at the exception message.
org.springframework.data.redis.serializer.SerializationException: Could not read JSON: Cannot deserialize Class xyz.kwin.redisabstract.service.EventService$1 (of type local/anonymous) as a Bean at [Source: (byte[])"["xyz.kwin.redisabstract.service.EventService$1",{"currentPage":null,"pageSize":null,"pageNum":null,"total":nul l,"index":null,"orderBy":null,"order":null,"data":null}]"; line: 1, column: 50]; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Cannot deserialize Class xyz.kwin.redisabstract.service.EventService$1 (of type local/anonymous) as a Bean ... . Caused by: com.fasterxml.jackson.databind.JsonMappingException: Cannot deserialize Class xyz.kwin.redisabstract.service.EventService$1 (of type local/anonymous) as a Bean at [Source: (byte[])"["xyz.kwin.redisabstract.service.EventService$1",{"currentPage":null,"pageSize":null,"pageNum":null,"total":nul l,"index":null,"orderBy":null,"order":null,"data":null}]"; line: 1, column: 50] ... . Caused by: java.lang.IllegalArgumentException: Cannot deserialize Class xyz.kwin.redisabstract.service.EventService$1 (of type local/anonymous) as a Bean ... .Copy the codeObviously, read relevant json string from redis deserialized into Java objects times wrong (RedisTemplate valueSerializer jackson2JsonRedisSerializer), The newspaper always deserialize a Class xyz. Kwin. Redisabstract. Service. EventService $1 (of type local/anonymous) as a Bean.
We to analyse, the deserialization types according to the logic, we cached PageEntity class, should be xyz. Kwin. Redisabstract. Util. PageEntity type, Why is the xyz. Kwin. Redisabstract. Service. EventService $1 this type (in redis saved related cache as shown in figure).
In the process of analysis, because it was not mentioned before that an abstract class was instantiated and cached in this way, the author temporarily solved the problem through concrete subclass inheritance as the reason of abstract class. EventService$1 is the first anonymous inner class in the EventService class. It is the first anonymous inner class in the EventService class. At this time, I looked back at the relevant exception information (of type local/anonymous), dare to throw the exception has clearly written to me the reason of the exception, I did not see… Obviously, this method of instantiating an abstract class produces an anonymous inner class, and the associated anonymous inner class name is EventService$1. However, when I looked for the class file of the anonymous inner class in the compiled class file, I could not find the relevant class information of the class, so I assumed that the cause of the problem was that the specific type could not be found during deserialization.
At this point, things are moving a bit further, but I’m starting to wonder why the inner class isn’t. So I wrote the simplest demo to verify that.
public class AnonymousInnerClass {
public void test(a) {
Thread t = new Thread() {
@Override
public void run(a) { super.run(); } }; } } Copy the codeThis is one of the most typical ways to use anonymous inner classes. After using the Build Project of IDEA, we will take a look at the output directory of this class.
AnonymousInnerClass$1.class = AnonymousInnerClass$1.class = AnonymousInnerClass$1.class But when I compile the AnonymousInnerClass.java file directly from the command line using Javac, things change!
After compilation, class files for the external and inner classes are successfully generated in the same directory as the Java file. So the problem moved forward a little bit and got stuck again. In the following day, with this question in mind, I could not help checking relevant articles on various blogs and forums during my time at work. Among them, I was attracted by a comment on an anonymous interior class article (the specific article has disappeared from the historical record…). The comment wrote that he looked at the compiled class file of the anonymous inner class in the IDE tool and could only find the class object of the outer class, but when he looked at the project structure in explorer, he found the anonymous inner class file with the word $1. After secretly writing down the content of this comment, I will forget it. But when I got home and looked back at the question, I suddenly remembered this comment, so I went straight to Visit and swiped down to the target directory of the demo output.
Sure enough, class files for anonymous inner classes exist, and even EventService$1.class is perfectly fine. As for why the target of IDEA is not displayed, it should be the IDE that has his own IDEA (if you know, please let me know in the comment section ~).
This disproves the claim that the anonymous inner class files cannot be found during deserialization.
Naturally, then, I turned my attention to the redis serialization approach.
@Configuration
public class RedisConfig {
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
/ / use Jackson2JsonRedisSerializer to serialization and deserialization redis value value Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class); ObjectMapper om = new ObjectMapper(); om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY); om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL); jackson2JsonRedisSerializer.setObjectMapper(om); RedisTemplate<String, Object> redisTemplate = new RedisTemplate<String, Object>(); redisTemplate.setConnectionFactory(redisConnectionFactory); // Set the key serialization mode redisTemplate.setKeySerializer(new StringRedisSerializer()); redisTemplate.setHashKeySerializer(new StringRedisSerializer()); // Set the value serialization method redisTemplate.setValueSerializer(jackson2JsonRedisSerializer); redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer); redisTemplate.afterPropertiesSet(); return redisTemplate; } } Copy the codeUsing RedisTemplate configuration as above, the conventional configuration, set the value of the serialization to Jackson2JsonRedisSerializer.
I didn’t know much about this, so I programmed for Google and found relevant information on Stack Overflow. According to the description of the problem, this guy should have encountered the same problem as me. The problem mentioned that Jackson could serialize anonymous inner classes, but not deserialize them, and some people proposed a reasonable explanation.
An inner class instance needs its external class instance object to be instantiated, and Jackson cannot create its external class instance object during deserialization
We open the class file of the anonymous inner class in the IDE tool.
As you can see in the class file of an anonymous inner class, it has only one constructor and its input parameter is its outer class. When Jackson deserialized the json above, he would not have successfully deserialized it (after all, there is no external class type stored in the JSON string).
At this point, there’s probably a conclusion to the problem: When Jackson deserializes a JSON string into a Java object, he can’t instantiate the object using the constructor provided by the anonymous inner class. When deserialized so throws always deserialize a Class xyz. Kwin. Redisabstract. Service. EventService $1 (of type local/anonymous) as a Bean exception information.
In fact, if you look closely, you can see that, except for the Anonymous class, the local inner class is also not allowed, because the local inner class also holds references to its external class.
So, of course, the normal inner class does not work either, it also holds a reference to the outer class.
So, static inner classes should work, right? It does not hold references to external classes.
Indeed, static inner classes have only one no-parameter constructor implemented by default. Of course, lip service doesn’t count, so let’s test it out.
Verify that the demo is going to write an InnerClassService, and this time we’re going to make it as simple as possible, because we already know what’s going on. The relevant codes are as follows.
@Service
public class InnerClassService {
private final RedisTemplate<String, Object> redisTemplate;
public InnerClassService(RedisTemplate<String, Object> redisTemplate) { this.redisTemplate = redisTemplate; } public Object anonymousInner(a) { String key = "anonymousInner"; Object result = redisTemplate.opsForValue().get(key); if (null == result) { result = new PageEntity<Event>() { @Override public Class<Event> findTClass(a) { return Event.class; } }; redisTemplate.opsForValue().set(key, result); } return result; } public Object localInner(a) { @Data @AllArgsConstructor @NoArgsConstructor class A { private String name; } String key = "localInner"; Object result = redisTemplate.opsForValue().get(key); if (null == result) { result = new A("localInner"); redisTemplate.opsForValue().set(key, result); } return result; } public Object normalInner(a) { String key = "normalInner"; Object result = redisTemplate.opsForValue().get(key); if (null == result) { result = new B("normalInner"); redisTemplate.opsForValue().set(key, result); } return result; } public Object staticInner(a) { String key = "staticInner"; Object result = redisTemplate.opsForValue().get(key); if (null == result) { result = new C("staticInner"); redisTemplate.opsForValue().set(key, result); } return result; } @Data @AllArgsConstructor @NoArgsConstructor public class B { private String name; } @Data @AllArgsConstructor @NoArgsConstructor public static class C { private String name; } } Copy the codeWe then performed a single test and, of course, the first request was all successful.
Data in Redis is properly cached. A single test is then performed, this time returning the result of deserializing the JSON string. As expected, only the static inner class succeeded in deserializing the four inner classes. The deserialization of the other three inner classes all threw the same exception as above.
Single test results confirm our conclusion that Jackson failed to successfully deserialize the inner class, the local inner class, and the anonymous inner class when deserializing the inner class because it could not be created by the constructor because it held a reference to the outer class. Static inner classes that do not hold references to external classes can be deserialized normally (static inner classes simply “hide” ordinary classes in external classes). As you can see from the compiled class file, the three inner class constructor arguments that cannot be deserialized by Jackson are automatically added with instances of the outer class, so you can verify for yourself.
conclusion
Through this problem investigation, a little review of Java basic knowledge.
Each of the four inner classes has its own features:
- (Member) Inner class: Holds a reference to an external class and can access properties of the instance object through this reference; And the inner class must be accessed through an external class instance.
- Local inner class: exists in a method; Holds a reference to the external class.
- Anonymous inner classes: instantiated with double curly braces; Holds a reference to an external class object.
- Static inner class: does not hold a reference to an external class object; The equivalent of hiding a normal class in an external class; It can be accessed directly from Outer.Inner.
extension
In the stack Overflow question, Jackson, Kyro, and XStream are serialized, and Kyro and XStream can serialize and deserialize inner classes (Kyro does not).
Kyro
We customized a KyroRedisSerializer (code from Baidu). Replace valueSerializer for RedisTemplate with KyroRedisSerializer in RedisConfig, and then perform a single test to see the result.
public class KyroRedisSerializer<T> implements RedisSerializer<T> {
private static final Logger logger = LoggerFactory.getLogger(KyroRedisSerializer.class);
public static final byte[] EMPTY_BYTE_ARRAY = new byte[0];
private static final ThreadLocal<Kryo> kryos = ThreadLocal.withInitial(Kryo::new); private Class<T> clazz; public KyroRedisSerializer(Class<T> clazz) { super(a); this.clazz = clazz; } @Override public byte[] serialize(T t) throws SerializationException { if (t == null) { return EMPTY_BYTE_ARRAY; } Kryo kryo = kryos.get(); kryo.setReferences(false); kryo.register(clazz); try (ByteArrayOutputStream baos = new ByteArrayOutputStream(); Output output = new Output(baos)) { kryo.writeClassAndObject(output, t); output.flush(); return baos.toByteArray(); } catch (Exception e) { logger.error(e.getMessage(), e); } return EMPTY_BYTE_ARRAY; } @Override public T deserialize(byte[] bytes) throws SerializationException { if (bytes == null || bytes.length <= 0) { return null; } Kryo kryo = kryos.get(); kryo.setReferences(false); kryo.register(clazz); try (Input input = new Input(bytes)) { return (T) kryo.readClassAndObject(input); } catch (Exception e) { logger.error(e.getMessage(), e); } return null; } } Copy the codeThe first time, of course, everything passed.
The following is the result of the second single test. It looks like they all passed, but clicking on the single test item still throws the corresponding exception, but the exception information is different from Jackson’s.
Interrupt point to view deserialization results, deserialization is fine, this exception does not seem to affect deserialization.
XStream
It seems that there is little information about this serialization method on the Internet, and I have no idea about this serialization method. Even if it is the first time to hear about it, I will not bother to try it