Distributed Services Framework – serialization and deserialization

This is the 18th day of my participation in the August Challenge

This article uses the source address: simple-rpc

The process of converting an object into a sequence of bytes is called object serialization. The process of restoring a sequence of bytes to an object is called deserialization of an object.

The main purpose of serialization:

• By serializing objects into byte arrays, objects can be transferred between systems that do not share memory over a network.
• An object can be permanently stored to a storage device by serializing it to a byte array.
• Fixed memory sharing between remote interface calls to JVMS.

See the open source project jVM-Serializers for performance comparisons of various commonly used serializers.

Only part of this is implemented in Simple-RPC:

To abstract a generic serialization/deserialization service, first define a generic serialization/deserialization interface. The code is as follows:

public interface ISerializer {

    /** * serialize *@param obj
     * @param <T>
     * @return* /
    <T> byte[] serialize(T obj);

    /** * deserialize *@param data
     * @param clazz
     * @param <T>
     * @return* /
    <T> T deserialize(byte[] data, Class<T> clazz);
}

Copy the code

The following five common serialization/deserialization methods are implemented based on this interface.

Java default serialization

The JDK provides a way to serialize Java objects. Java serialization mainly through the output stream object Java. IO. ObjectOutputStream and Java object input stream. IO. ObjectInputStream, which is serialized class needs to implement Java IO. The Serializable interface. The code is as follows:

@Slf4j
public class DefaultSerializer implements ISerializer {

    @Override
    public <T> byte[] serialize(T obj) {
        if (obj == null) {
            throw new NullPointerException("DefaultSerializer serialize data is null");
        }
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();

        try {
            ObjectOutputStream objectOutputStream = new ObjectOutputStream(byteArrayOutputStream);
            objectOutputStream.writeObject(obj);
            objectOutputStream.close();
        } catch (IOException e) {
            log.error("DefaultSerializer serialize error, obj={}", obj, e);
            throw new SRpcException("DefaultSerializer serialize error", e);
        }

        return byteArrayOutputStream.toByteArray();
    }

    @Override
    public <T> T deserialize(byte[] data, Class<T> clazz) {
        if (data == null) {
            throw new NullPointerException("DefaultSerializer deserialize data is null");
        }
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);

        try {
            ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream);
            return (T) objectInputStream.readObject();
        } catch (Exception e) {
            log.error("DefaultSerializer deserialize error, data={}", data, e);
            throw new SRpcException("DefaultSerializer deserialize error", e); }}}Copy the code

For Java serialization there are the following:

• In serialization, only the state of the object is saved, regardless of the object’s delivery method.
• When a parent class implements serialization, the subclass automatically implements serialization without the need to display the implementationjava.io.SerializableInterface.
• When an instance variable of an object refers to another object, the reference object is also serialized when the object is serialized.
• When an object is declared astransientThe default serialization mechanism ignores this field.

The advantages and disadvantages of Java’s default serialization mechanism are obvious:

Advantages:

• The Java language comes with it and is easy to use without introducing third party dependencies.

Disadvantages:

• Only Java language is supported, not cross-language.
• Java default serialization performance is poor, the code flow generated after serialization is large, for the object serialization of too deep reference is prone to OOM exception.

Json serialization

Json is a lightweight data interchange format. The Json stream is small and readable. Json common open source serialization tools are as follows:

• Jackson
• fastjson
• Gson

In order to control the length, let’s look at the implementation of fastjson, the code is as follows:

public class FastJsonSerializer implements ISerializer {

    static {
        ParserConfig.getGlobalInstance().addAccept("com.miracle.srpc");
    }

    @Override
    public <T> byte[] serialize(T obj) {
        if (obj  == null) {
            throw new NullPointerException("FastJsonSerializer serialize data is null");
        }
        JSON.DEFFAULT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss";
        return JSON.toJSONString(obj, SerializerFeature.WriteDateUseDateFormat, SerializerFeature.WriteClassName).getBytes();
    }

    @Override
    public <T> T deserialize(byte[] data, Class<T> clazz) {
        if (data  == null) {
            throw new NullPointerException("FastJsonSerializer deserialize data is null");
        }
        return JSON.parseObject(newString(data), clazz); }}Copy the code

Hessian serialization

Hessian is a binary serialization protocol that supports cross-language transport. Hessian offers better performance and ease of use than Java’s default serialization mechanism, and supports many different languages. Serialization and deserialization are implemented as follows:

@Slf4j
public class HessianSerializer implements ISerializer {

    @Override
    public <T> byte[] serialize(T obj) {
        if (obj == null) {
            throw new NullPointerException("HessianSerializer serialize obj is null");
        }
        try {
            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            HessianOutput output = new HessianOutput(byteArrayOutputStream);
            output.writeObject(obj);
            return byteArrayOutputStream.toByteArray();
        } catch (IOException e) {
            log.error("HessianSerializer serialize error. obj = {}", obj, e);
            throw new SRpcException("HessianSerializer serialize error", e); }}@Override
    public <T> T deserialize(byte[] data, Class<T> clazz) {
        if (data  == null) {
            throw new NullPointerException("HessianSerializer deserialize data is null");
        }
        try {
            ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
            HessianInput hessianInput = new HessianInput(byteArrayInputStream);
            return (T) hessianInput.readObject();
        } catch (IOException e) {
            log.error("HessianSerializer deserialize error. data = {}", data, e);
            throw new SRpcException("HessianSerializer deserialize error", e); }}}Copy the code

Protobuf serialization

Protobuf is an open source data exchange format from Google. It supports multiple languages and includes a compiler and library file for each implementation. Protobuf has low space overhead, high parsing performance, and low amount of data after serialization, but it requires writing *. Proto IDL files.

The steps for using Protobuf are as follows:

1. Configure the development environment and install the Protocol Compiler code Compiler.
2. Write a. Proto file that defines the data structure of the serialized object.
3. Based on writing. Proto files, the corresponding serialization/antisequence utility classes are compiled using Protocol Compiler.
4. Write your own serialization application based on the generated code.

Serialization and deserialization are implemented as follows:

@Slf4j
public class ProtoBufSerializer implements ISerializer {

    @Override
    public <T> byte[] serialize(T obj) {
        try {
            if (obj instanceof GeneratedMessageV3) {
                throw new UnsupportedOperationException("ProtoBufSerializer serialize not support obj type");
            }
            return (byte[]) MethodUtils.invokeMethod(obj, "toByteArray");
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
            log.error("ProtoBufSerializer serialize error, obj={}", obj, e);
            throw new SRpcException("ProtoBufSerializer serialize error", e); }}@Override
    public <T> T deserialize(byte[] data, Class<T> clazz) {
        try {
            if(! GeneratedMessageV3.class.isAssignableFrom(clazz)) {throw new UnsupportedOperationException("ProtoBufSerializer deserialize not support obj type");
            }
            return (T) MethodUtils.invokeStaticMethod(clazz, "getDefaultInstance");
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
            log.error("ProtoBufSerializer deserialize error", e);
            throw new SRpcException("ProtoBufSerializer deserialize error", e); }}}Copy the code

Serialization engine

The above serialization and deserialization implementations are provided according to different serialization frameworks. To make the serialization tool easier to use, the above implementations are integrated into a general serialization tool engine, which can be arbitrarily selected by entering different configurations. The code implementation is as follows:

@Slf4j
public class SerializerEngine {

    private static final Map<SerializeType, ISerializer> SERIALIZER_MAP = Maps.newHashMap();

    static {
        SERIALIZER_MAP.put(SerializeType.DefaultSerializer, new DefaultSerializer());
        SERIALIZER_MAP.put(SerializeType.JacksonSerializer, new JacksonSerializer());
        SERIALIZER_MAP.put(SerializeType.FastJsonSerializer, new FastJsonSerializer());
        SERIALIZER_MAP.put(SerializeType.HessianSerializer, new HessianSerializer());
        SERIALIZER_MAP.put(SerializeType.ProtoBufSerializer, new ProtoBufSerializer());
    }

    /** * serialize *@paramObject obj *@paramSerializeType Serialization mode *@param <T>
     * @return* /
    public static <T> byte[] serialize(T obj, String serializeType) {
        SerializeType serialize = SerializeType.getByType(serializeType);
        if (null == serialize) {
            throw new SRpcException("SerializerEngine serialize error, serializeType is not support");
        }
        ISerializer iSerializer = SERIALIZER_MAP.get(serialize);
        if (null == iSerializer) {
            throw new SRpcException("SerializerEngine serialize error, get ISerializer fail");
        }
        try {
            return iSerializer.serialize(obj);
        } catch (Exception e) {
            log.error("SerializerEngine serialize error, obj={}", obj, e);
            throw new SRpcException("SerializerEngine serialize error"); }}/** * deserialize *@paramThe data data *@param clazz
     * @param serializeType
     * @param <T>
     * @return* /
    public static <T> T deserialize(byte[] data, Class<T> clazz, String serializeType) {
        SerializeType serialize = SerializeType.getByType(serializeType);
        if (null == serialize) {
            throw new SRpcException("SerializerEngine deserialize error, serializeType is not support");
        }
        ISerializer iSerializer = SERIALIZER_MAP.get(serialize);
        if (null == iSerializer) {
            throw new SRpcException("SerializerEngine deserialize error, get ISerializer fail");
        }
        try {
            return iSerializer.deserialize(data, clazz);
        } catch (Exception e) {
            log.error("SerializerEngine deserialize error, data={}", data, e);
            throw new SRpcException("SerializerEngine deserialize error"); }}}Copy the code

SerializerEngine provides configurable serialization and deserialization through static code blocks that add serialization algorithms to the local cache at class load time, provide corresponding serialization and deserialization methods, and provide serializeType as a passable parameter.