In RPC framework, sticky packet and unpacket must be solved, because in RPC framework, each microservice maintains a TCP long connection with each other, for example, Dubbo is a full-duplex long connection. Since microservices send messages to each other using the same connection, sticky packets and unpacket problems will occur. This article will first describe the sticky and unpack problems, then introduce common solutions, and finally explain several solutions provided by Netty. Here to clarify, as OSChina identifies “Jie Ma Qi” as a sensitive character, this paper uses “Decoder one” to express this meaning

1. Glue and unpack

Produce glue bag and unpacking is the main reason of the problem, the operating system on the sending TCP data, the bottom there will be a buffer, such as the size of 1024 bytes, if a request is sent the amount of data is small, not up to buffer size, TCP will merge multiple requests for the same request to send, this creates a stick pack problem; If the amount of data sent in one request is too large to exceed the buffer size, TCP will split it into multiple packets. This is called unpacking, that is, a large packet is split into multiple packets for sending. The following figure shows a schematic of sticking and unpacking:

The three cases of sticking and unpacking are illustrated in the figure above:

  • Both packets A and B meet the size of the TCP buffer, or their waiting time has reached the TCP waiting time. Therefore, two independent packets are still sent.

  • The interval between two requests of A and B is short and the data packets are small. Therefore, the two requests are combined into the same packet and sent to the server.

  • Packet B is large, so it is split into two packets B_1 and B_2 for sending. In this case, since the split B_2 is small, it is combined with packet A for sending.

2. Common solutions

There are four common solutions to sticky and unpack problems:

  • When the client sends data packets, each packet is of a fixed length, for example, 1024 bytes. If the length of data sent by the client is less than 1024 bytes, a space is added to fill the data packet to a specified length.

  • The client uses fixed delimiters at the end of each packet, such as \r\n. If a packet is split, it will find \r\n after the next packet is sent, and then merge its split header with the rest of the previous packet, so as to obtain a complete packet.

  • The message is divided into header and body, and the length of the current whole message is kept in the header. A complete message is read only after the message with sufficient length is read.

  • You can glue or unpack packets using user-defined protocols.

3. Netty provides a solution for sticking and unpacking packets

3.1 FixedLengthFrameDecoder

For sticky and unpacked scenarios that use fixed length, you can use FixedLengthFrameDecoder, which reads a fixed length message each time, and waits for the next message to arrive if the current read is less than the specified length. It is also relatively simple to use, specifying the length of each message in the constructor. Note here that FixedLengthFrameDecoder is only a decoder, Netty also provides only a decoder, this is because the decoder needs to wait for the next package to complete, the code is relatively complex, but for encoders, users can write their own, Because the encoding only needs to complete the part that is less than the specified length. The following example shows how to use FixedLengthFrameDecoder for sticky and unpack processing:

public class EchoServer { public void bind(int port) throws InterruptedException { EventLoopGroup bossGroup = new NioEventLoopGroup(); EventLoopGroup workerGroup = new NioEventLoopGroup(); try { ServerBootstrap bootstrap = new ServerBootstrap(); bootstrap.group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG, 1024) .handler(new LoggingHandler(LogLevel.INFO)) .childHandler(new ChannelInitializer<SocketChannel>() { @Override Protected void initChannel(SocketChannel CH) throws Exception { Pipeline ().addLast(new FixedLengthFrameDecoder(20)); Ch.pipeline ().addLast(new StringDecoder()); Ch.pipeline ().addLast(new FixedLengthFrameEncoder(20)); Ch.pipeline ().addLast(new EchoServerHandler()); }}); ChannelFuture future = bootstrap.bind(port).sync(); future.channel().closeFuture().sync(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } public static void main(String[] args) throws InterruptedException { new EchoServer().bind(8080); }}Copy the code

In the above pipeline, FixedLengthFrameDecoder and StringDecoder are mainly added for the pushed data. The first one deals with sticky and unpacked messages of fixed length, and the second one converts the processed messages to strings. Finally, the EchoServerHandler processes the final data. After processing, the processed data is processed by FixedLengthFrameEncoder, which is our self-defined implementation. The main function is to complete the space of messages with length less than 20. The following is a FixedLengthFrameEncoder:

public class FixedLengthFrameEncoder extends MessageToByteEncoder<String> { private int length; public FixedLengthFrameEncoder(int length) { this.length = length; } @override protected void encode(ChannelHandlerContext CTX, String MSG, ByteBuf out) throws Exception {// For messages exceeding the specified length, You throw an exception if directly (MSG. Length () > length) {throw new UnsupportedOperationException (" the message length is too large, it's limited " + length); If (MSG. Length () < length) {MSG = addSpace(MSG); } ctx.writeAndFlush(Unpooled.wrappedBuffer(msg.getBytes())); } private String addSpace(String MSG) {StringBuilder Builder = new StringBuilder(MSG); for (int i = 0; i < length - msg.length(); i++) { builder.append(" "); } return builder.toString(); }}Copy the code

Here FixedLengthFrameEncoder implements the decode() method, in which messages of less than 20 messages are mostly whitespace completed. The purpose of the EchoServerHandler is to print the received message and send the response to the client:

public class EchoServerHandler extends SimpleChannelInboundHandler<String> {

  @Override
  protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
    System.out.println("server receives message: " + msg.trim());
    ctx.writeAndFlush("hello client!");
  }
}
Copy the code

For the client, the implementation is basically the same as for the server, except that the final message is sent in a different way. Here is the code for EchoClient:

public class EchoClient { public void connect(String host, int port) throws InterruptedException { EventLoopGroup group = new NioEventLoopGroup(); try { Bootstrap bootstrap = new Bootstrap(); bootstrap.group(group) .channel(NioSocketChannel.class) .option(ChannelOption.TCP_NODELAY, true) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception {// Paste and unpack the message sent by the server, because the message sent by the server has been completed with space, and the length is 20, Pipeline ().addLast(new FixedLengthFrameDecoder(20)) Ch.pipeline ().addLast(new StringDecoder()); Pipeline ().addLast(new FixedLengthFrameEncoder(20)); Ch.pipeline ().addlast (new EchoClientHandler()); // The client sends a message to the server and processes the response message from the server. }}); ChannelFuture future = bootstrap.connect(host, port).sync(); future.channel().closeFuture().sync(); } finally { group.shutdownGracefully(); }} public static void main(String[] args) throws InterruptedException {new EchoClient().connect("127.0.0.1", 8080); }}Copy the code

For the client, its message processing process is actually similar to that of the server. For the inbound message, it needs to paste and unpack it, and then transcode it into a string. For the outbound message, it needs to complete the message with space less than 20. The main difference between client-side and server-side processing is the final message handler, called EchoClientHandler. Here is the source code for this handler:

public class EchoClientHandler extends SimpleChannelInboundHandler<String> {

  @Override
  protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
    System.out.println("client receives message: " + msg.trim());
  }

  @Override
  public void channelActive(ChannelHandlerContext ctx) throws Exception {
    ctx.writeAndFlush("hello server!");
  }
}
Copy the code

The client’s main processing here is to override the channelActive() and channelRead0() methods. The main purpose of both methods is that channelActive() executes when the client is connected to the server, that is, it sends a message to the server once it is connected. ChannelRead0 (), on the other hand, is executed when the server sends a response to the client, which prints the server’s response message. On the server side, we saw earlier that the EchoServerHandler just overrides the channelRead0() method, because the server just waits for the client to send a message, processes it in the method, and then sends the response directly to the client. Here is the data printed by the console after starting the server and client respectively:

// server
server receives message: hello server!

// client
client receives message: hello client!
Copy the code

3.2 LineBasedFrameDecoder与DelimiterBasedFrameDecoder

For through the separator glue bag and unpacking problem handling, Netty provides two codec class, LineBasedFrameDecoder and DelimiterBasedFrameDecoder. The main function of LineBasedFrameDecoder is to process the data with the newline character \n or \r\n. And DelimiterBasedFrameDecoder role by the user to specify a delimiter data package and unpacking processing. Again, both classes are decoder classes, and the encoding of the data, that is, adding a line break or specifying a separator at the end of each packet, needs to be handled by the user. Here talking DelimiterBasedFrameDecoder, for example, the following is the use of the class EchoServer code snippet, the rest are in complete accord with the previous example of:

@Override protected void initChannel(SocketChannel ch) throws Exception { String delimiter = "_$"; / / set the delimiter to DelimiterBasedFrameDecoder, after the decoding a device for processing, the source data will be carried out in accordance with the _ $/ / space, here refers to the maximum length of the space, 1024 when read 1024 bytes of data, If the delimiter is still not read, discard the current data segment, Because it is likely to be caused by code flow disorder ch. Pipeline () addLast (new DelimiterBasedFrameDecoder (1024, Unpooled.wrappedBuffer(delimiter.getBytes()))); // Convert delimited byte data to string data ch.pipeline().addLast(new StringDecoder()); / / this is our custom an encoder, main effect is in the returned response data finally add separator ch. Pipeline () addLast (new DelimiterBasedFrameEncoder delimiter ()); // The handler that finally processes the data and returns the response ch.pipeline().addLast(new EchoServerHandler()); }Copy the code

Pipeline above Settings, add a decoding device mainly DelimiterBasedFrameDecoder and StringDecoder, after the two processing, the received bytes will be separated, and converted to the string data, It is ultimately handled by the EchoServerHandler. DelimiterBasedFrameEncoder here is our custom encoder, its main function is added after the returned response data separator. The following is the source code of the encoder:

public class DelimiterBasedFrameEncoder extends MessageToByteEncoder<String> { private String delimiter; public DelimiterBasedFrameEncoder(String delimiter) { this.delimiter = delimiter; } @Override protected void encode(ChannelHandlerContext ctx, String msg, ByteBuf out) throws Exception {// Add the delimiter ctx.writeAndFlush(Unpooled. WrappedBuffer ((MSG +) after the data in the response delimiter).getBytes())); }}Copy the code

For the client side, the processing here is similar to that of the server side, and its pipeline is added as follows:

@Override protected void initChannel(SocketChannel ch) throws Exception { String delimiter = "_$"; // The messages returned by the server are separated by _$, And every time to find the maximum size of 1024 bytes ch. Pipeline () addLast (new DelimiterBasedFrameDecoder (1024, Unpooled.wrappedBuffer(delimiter.getBytes()))); // Convert delimited byte data to string ch.pipeline().addLast(new StringDecoder()); / / to encode the client sends data, here is mainly in the client sends data finally add delimiters ch. Pipeline () addLast (new DelimiterBasedFrameEncoder delimiter ()); Ch.pipeline ().addlast (new EchoClientHandler()); // The client sends data to the server and processes the data from the server. }Copy the code

Here the client handles the same as the server, and the code that is not shown here is exactly the same as the code in Example 1, which is not shown here.

3.3 LengthFieldBasedFrameDecoder与LengthFieldPrepender

Here LengthFieldBasedFrameDecoder and LengthFieldPrepender need to cooperate together to use, in fact, essentially, is both a decoding, one is the relationship of the code. The main idea of sticky unpacking is to add a length field to the generated packet to record the length of the current packet. LengthFieldBasedFrameDecoder will be in accordance with the parameters specified in the received data packet length offset data decoding, data to get the target message body; LengthFieldPrepender, on the other hand, prefixes the data in the response with the specified byte data, which holds the overall byte data length of the current message body. LengthFieldBasedFrameDecoder decoding process as shown in the figure below:

The encoding process of LengthFieldPrepender is shown in the figure below:

About LengthFieldBasedFrameDecoder, need its constructor parameters were introduced here:

  • MaxFrameLength: specifies the maximum packet size that can be delivered per packet.

  • LengthFieldOffset: Specifies the offset of the length field in the bytecode;

  • LengthFieldLength: Specifies the length of the length field in bytes;

  • LengthAdjustment: Adjusts the length of the header for data that contains not only the header and the body, so that only the body data can be obtained. LengthAdjustment specifies the length of the header.

  • Initialbytesttestis: To testtesttestis in the middle of the message header, testtesttestis ignored.

Here we are in json serialization, for example to explain LengthFieldBasedFrameDecoder and LengthFieldPrepender way of use. Here is the source code for EchoServer:

public class EchoServer { public void bind(int port) throws InterruptedException { EventLoopGroup bossGroup = new NioEventLoopGroup(); EventLoopGroup workerGroup = new NioEventLoopGroup(); try { ServerBootstrap bootstrap = new ServerBootstrap(); bootstrap.group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG, 1024) .handler(new LoggingHandler(LogLevel.INFO)) .childHandler(new ChannelInitializer<SocketChannel>() { @Override Protected void initChannel SocketChannel (ch) throws the Exception {/ / here will LengthFieldBasedFrameDecoder added to the pipeline of the first, Because it need a length field received data / / decoding, here also will stick the data package and unpacking processing ch. Pipeline () addLast (new LengthFieldBasedFrameDecoder (1024, 0, 2, 0, 2)); LengthFieldPrepender is an encoder that adds a byte length field ch.pipeline().addLast(new LengthFieldPrepender(2)) in front of the response byte data; Ch.pipeline ().addLast(new JsonDecoder())); // Serialize the User object to json ch.pipeline().addlast (new JsonEncoder()); Ch.pipeline ().addLast(new EchoServerHandler()); }}); ChannelFuture future = bootstrap.bind(port).sync(); future.channel().closeFuture().sync(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } public static void main(String[] args) throws InterruptedException { new EchoServer().bind(8080); }}Copy the code

Here, EchoServer mainly adds two encoders and two decoders in the pipeline. The encoder is mainly responsible for serializing the User object in response to json object, and then adds a byte array of length field in front of its byte array. The decoder decodes the length field of the received data and then deserializes it into a User object. JsonDecoder JsonDecoder

public class JsonDecoder extends MessageToMessageDecoder<ByteBuf> { @Override protected void decode(ChannelHandlerContext ctx, ByteBuf buf, List<Object> out) throws Exception { byte[] bytes = new byte[buf.readableBytes()]; buf.readBytes(bytes); User user = JSON.parseObject(new String(bytes, CharsetUtil.UTF_8), User.class); out.add(user); }}Copy the code

JsonDecoder first reads the byte array from the received data stream and then deserializes it into a User object. JsonEncoder JsonEncoder

public class JsonEncoder extends MessageToByteEncoder<User> {

  @Override
  protected void encode(ChannelHandlerContext ctx, User user, ByteBuf buf)
      throws Exception {
    String json = JSON.toJSONString(user);
    ctx.writeAndFlush(Unpooled.wrappedBuffer(json.getBytes()));
  }
}
Copy the code

JsonEncoder converts the resulting User object into a JSON object and writes it to the response. For EchoServerHandler, its main function is to receive client data and respond to it.

public class EchoServerHandler extends SimpleChannelInboundHandler<User> { @Override protected void channelRead0(ChannelHandlerContext ctx, User user) throws Exception { System.out.println("receive from client: " + user); ctx.write(user); }}Copy the code

For the client side, its main logic is basically similar to that of the server side. Here, it mainly shows the way of pipeline addition and the process of sending the request and processing the server response:

@Override protected void initChannel(SocketChannel ch) throws Exception { ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024, 0, 2, 0, 2)); ch.pipeline().addLast(new LengthFieldPrepender(2)); ch.pipeline().addLast(new JsonDecoder()); ch.pipeline().addLast(new JsonEncoder()); ch.pipeline().addLast(new EchoClientHandler()); } public class EchoClientHandler extends SimpleChannelInboundHandler<User> { @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { ctx.write(getUser()); } private User getUser() { User user = new User(); user.setAge(27); user.setName("zhangxufeng"); return user; } @Override protected void channelRead0(ChannelHandlerContext ctx, User user) throws Exception { System.out.println("receive message from server: " + user); }}Copy the code

Here the client first sends a User object data to the server when it connects to the server, and then prints the data for the server response when it receives it.

3.4 Customizing sticky packages and Unpacking devices

For sticky and unpack problems, the first three are generally sufficient for most situations, but for more complex protocols, there may be some customization requirements. For these scenarios, in essence, actually we don’t need to manually from scratch a stick pack and unpacking the processor, but through inheritance LengthFieldBasedFrameDecoder and LengthFieldPrepender glue bag and unpacking processing.

If users really need to implement their own sticky and unpack handlers without inheritance, this can be done by implementing MessageToByteEncoder and ByteToMessageDecoder. Here MessageToByteEncoder encodes the response data into a ByteBuf object, while ByteToMessageDecoder converts the received ByteBuf data into some object data. By implementing these two abstract classes, users can implement custom sticky and unpack processing. Here are the declarations of the two classes and their abstract methods:

public abstract class ByteToMessageDecoder extends ChannelInboundHandlerAdapter {
    protected abstract void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) 
        throws Exception;
}

public abstract class MessageToByteEncoder<I> extends ChannelOutboundHandlerAdapter {
    protected abstract void encode(ChannelHandlerContext ctx, I msg, ByteBuf out) 
        throws Exception;
}
Copy the code

4. Summary

This paper first describes the principle of sticking and unpacking problems to help readers understand sticking and unpacking problems. Then several common solutions to deal with sticky package and unpack are explained. Then several solutions to sticky and unpack problems provided by Netty are explained in detail through examples.