After covering two common serialization methods in Java, JDK serialization and Hessian2 serialization, in the previous article, we’ll take a look at an upstart, Kryo serialization, which claims to be the fastest serialization framework in Java. So without further ado, let’s take a look at what this upstart is capable of.

Kryo serialization

Kryo is a fast serialization/deserialization tool that relies on a bytecode generation mechanism (underlying use of ASM libraries), so it has some advantages in serialization speed, but because of this, its use is limited to JVM-based languages.

There is a lot of information on the Web that says Kryo is only available in Java, which is not true. In fact, besides Java, JVM-based languages such as Scala and Kotlin can also be serialized using Kryo.

Like Hessian, Kryo’s serialized results are in its own custom, unique format. Since the serialized results are binary (byte[]), storage engines that store binary data, such as Redis, can directly store Kryo’s serialized data. Of course, you can also choose to convert it to a String and store it in another storage engine (at a performance cost).

Due to its excellent performance, Kryo has become the underlying serialization protocol for several well-known Java frameworks, including but not limited to 👇

  • Apache Fluo (Kryo is default serialization for Fluo Recipes)
  • Apache Hive (query plan serialization)
  • Apache Spark (shuffled/cached data serialization)
  • Storm (distributed realtime computation system, in turn used by many others)
  • Apache Dubbo (high performance, open source RPC framework)
  • …

The official website is at: github.com/EsotericSof…

Basic usage

With that said, let’s take a look at the basics of Kryo. For serialization frameworks, the apis are pretty much the same, since the input and output parameters are usually determined (which object to serialize/which result to serialize). Before using Kryo, we need to introduce dependencies

< the dependency > < groupId > com. Esotericsoftware < / groupId > < artifactId > kryo < / artifactId > < version > 5.2.0 < / version > </dependency>Copy the code

The basic usage is shown as 👇

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import java.io.*;
​
public class HelloKryo {
   static public void main(String[] args) throws Exception {
       Kryo kryo = new Kryo();
       kryo.register(SomeClass.class);
​
       SomeClass object = new SomeClass();
       object.value = "Hello Kryo!";
​
       Output output = new Output(new FileOutputStream("file.bin"));
       kryo.writeObject(output, object);
       output.close();
​
       Input input = new Input(new FileInputStream("file.bin"));
       SomeClass object2 = kryo.readObject(input, SomeClass.class);
       input.close();
       System.out.println(object2.value);
  }
​
   static public class SomeClass {
       String value;
  }
}
​
Copy the code

The Kryo class performs the serialization automatically. The Output and Input classes are responsible for handling the buffered bytes and writing them to the stream.

Serialization of Kryo

As a flexible serialization framework, Kryo doesn’t care about reading and writing data. As a developer, you are free to use the serializers that Kryo provides out of the box.

Kryo registered

Like many other serialization frameworks, Kryo provides a way to register classes of serialized objects in order to provide performance and reduce the size of serialization results. At registration, an int ID is generated for the serialized class, and an INT ID is used to uniquely identify the type during subsequent serialization. The registration method is as follows:

kryo.register(SomeClass.class);
Copy the code

or

kryo.register(SomeClass.class, 1);
Copy the code

You can explicitly specify the int ID of the registration class, but the ID must be greater than or equal to 0. If not, an ordered int ID generation will be maintained internally using int++.

Kryo’s serializer

Kryo supports a variety of serializers, as we can see from the source code 👇

For details, see 👉 “Serialization Types supported by Kryo.”

Although the serializers provided by Kryo can read and write most objects, developers can easily customize their own serializers. Space is limited, so I won’t go into it here, but take the default serializer as an example.

Object reference

In the new version of Kryo, object references are not enabled by default. This means that if an object appears multiple times in an object graph, it will be written multiple times and will be deserialized into multiple different objects.

For example, when reference properties are turned on, the first time each object appears in the object graph, a varint is written to the record, which is used as a marker. When the same object appears later, only a varint is recorded to save space. While this saves serialization space, it is a time-for-space approach that affects serialization performance, since tracking is required when an object is written/read.

Developers can use kryo’s built-in setReferences method to determine whether to enable kryo’s reference function.

Thread unsafe

Kryo is not thread-safe. Each thread should have its own Kryo object, input, and output instances.

So in a multithreaded environment, consider using ThreadLocal or object pooling to ensure thread safety.

ThreadLocal + Kryo resolve thread insecurity

ThreadLocal is a classic way to sacrifice space for concurrency safety by creating a separate kryo object for each thread. Execution is sequential for each Kryo object for each thread, thus naturally avoiding concurrency security issues. The creation method is as follows:

static private final ThreadLocal<Kryo> kryos = new ThreadLocal<Kryo>() { protected Kryo initialValue() { Kryo kryo = new  Kryo(); // Return kryo; // Return kryo; }; }; Kryo kryo = kryos.get();Copy the code

After that, you just need to pull the object from the thread context through the kryos.get() method.

Object pool + Kryo resolve thread insecurity

* * “pool” is a kind of very important programming ideas, a connection pool, thread pool, pool, etc are the embodiment of “reuse” * * ideas, by will create “object” stored in a single “container”, so that subsequent repeated use, avoid to create, destroy, produce the performance of the loss, in order to enhance the overall performance.

The same is true of Kryo object pooling. The Kryo framework comes with its own implementation of object pools. The entire process involves creating the pool, obtaining the object from the pool, and returning the object.

// Pool constructor arguments: thread safe, soft references, maximum capacity Pool<Kryo> kryoPool = new Pool<Kryo>(true, false, 8) { protected Kryo create () { Kryo kryo = new Kryo(); // return Kryo; }}; // Obtain the Kryo object in the pool. Kryo Kryo = kryopol.obtain (); // Return the kryo object to the pool kryopool.free (kryo);Copy the code

When creating a Kryo Pool, you need to pass in three parameters. The first parameter specifies whether synchronization is used within the Pool. If true, it allows concurrent access by multiple threads. The third argument, which is used to specify the size of the object pool, is easier to understand, so the second argument is important.

If the second parameter is set to true, Kryo pool will use Java. Lang. Ref. The SoftReference object to store. This allows objects in the pool to be garbage collected when the JVM is under heavy memory pressure. Pool Clean deletes all soft references to objects that have been garbage collected. This can reduce the size of the pool when no maximum capacity is set. There is no need to call Clean when the Pool has maximum capacity, because Pool free will attempt to remove an empty reference if the maximum capacity is reached.

Once the play Kryo pool is created, working with Kryo becomes extremely easy. You simply call the kryopool.obtain () method, and when you’re done, you call kryopool.free (Kryo) to return the object to complete a full lease.

In theory, as long as the object pool size is properly evaluated, the concurrency security problem can be solved perfectly with minimal memory footprint. If you want to encapsulate a Kryo serialization method, you can refer to the following code 👇

public static byte[] serialize(Object obj) { Kryo kryo = kryoPool.obtain(); Try (Output opt = new Output(1024, 0, 0) try (Output opt = new Output(1024, 0, 0) -1)) { kryo.writeClassAndObject(opt, obj); opt.flush(); return opt.getBuffer(); }finally { kryoPool.free(kryo); }}Copy the code

summary

Kryo delivers faster performance than the JDK’s built-in serialization, and has less storage overhead because Kryo allows for multiple references and circular references.

However, while Kryo has excellent performance, it leaves out many features such as thread-safety, field modifications to serialized objects, and so on. Although these disadvantages can be satisfied by Kryo’s good scalability, it is still difficult for developers to get started, but this does not affect its status in Java.

Kryo also has many great features, which you can find at 👉EsotericSoftware/ Kryo

Finally, if you find this post helpful, feel free to share your attention and likes, and let us know in the comments section