What is sticky half pack

TCP is a connection-oriented, reliable protocol that supports byte stream transport. In other words, when TCP remains connected, the message has no boundary, and the sender can continuously send the packet to the receiver. If the packet sent is too small, TCP will integrate the smaller packet based on Nagle algorithm. Therefore, using TCP as the transmission protocol, there will be the following situations

As shown above

  1. The packet received by the receiver is an independent and complete packet
  2. Packets sent by the sender are stuck together. Since the length of each packet is variable, the server cannot open it, which is the phenomenon of sticky packets. There are two reasons for sticky packets. Two: the receiver does not process the data in time and the receiving window is large enough so that the buffer buffers multiple packets
  3. 3, 4, and 5 are all cases of packet disassembly. The causes for packet disassembly are as follows: The amount of data sent by the sender is greater than the MSS or MTU of the protocol, and the packet must be disassembled. (MSS is the maximum segment size of the TCP layer. The data sent by the TCP layer to the IP layer cannot exceed this value. MTU is the maximum transmission unit (MTU). It is the maximum size of data that the physical layer provides for the upper layer to transmit at a time. It is used to limit the size of data transmitted at the IP layer. Two: The buffer of the receiver cannot accept the amount of data of the sender at this time

Code demo

Stick package
public class NettyServer { public static void main(String[] args) throws InterruptedException { // 1. Create two thread groups: bossGroup and workerGroup // 2. BossGroup only handles connection requests. The real and client business processing will be assigned to workerGroup // 3. BossGroup = new NioEventLoopGroup(); EventLoopGroup workerGroup = new NioEventLoopGroup(); ServerBootstrap = new ServerBootstrap(); ServerBootstrap = new ServerBootstrap(); // Set two thread groups bootstrap.group(bossGroup, WorkerGroup) / / using NioServerSocketChannel as the server channel implementation. The channel (NioServerSocketChannel. Class) / / create a channel test object .childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new MyLoggingHandler()); }}); System.out.println("server is ready"); ChannelFuture cf = bootstrap.bind(6668).sync(); ChannelFuture cf = bootstrap.bind(6668).sync() Cf.channel ().closeFuture().sync(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); }}}Copy the code

In order to see the information received by the server more clearly, we add a Handler that prints logs

Public class MyLoggingHandler extends ChannelInboundHandlerAdapter {/ * * * @ param CTX context object, * @param MSG client sends data, Default Object * @override public void channelRead(ChannelHandlerContext CTX, Object msg) throws Exception { System.out.println("server ctx = " + ctx); // convert MSG to a ByteBuf ByteBuf = (ByteBuf) MSG; int length = byteBuf.readableBytes(); int rows = length / 16 + (length % 15 == 0? 0:1) + 4; StringBuilder str = new StringBuilder(rows * 80 * 2) .append("read index:").append(byteBuf.readerIndex()) .append(" write index:").append(byteBuf.writerIndex()) .append(" capacity:").append(byteBuf.capacity()) .append(NEWLINE); appendPrettyHexDump(str, byteBuf); System.out.println(str.toString()); } // Handle exceptions, @Override public void exceptionCaught(ChannelHandlerContext CTX, Throwable cause) throws Exception {ctx.close(); }}Copy the code

Example Set a client to send messages 10 times

public class NettyClient { public static void main(String[] args) throws InterruptedException { send(); } private static void send() throws InterruptedException { EventLoopGroup eventExecutors = new NioEventLoopGroup(); Try {// Instead of using ServerBootstrap, use Bootstrap = new Bootstrap(); bootstrap.group(eventExecutors) .channel(NioSocketChannel.class) .handler(new ChannelInboundHandlerAdapter() { @Override Public void channelActive(ChannelHandlerContext CTX) throws Exception {// Send the message 10 times. For (int I = 0; i < 10; i++) { ByteBuf buffer = ctx.alloc().buffer(16); buffer.writeBytes(new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}); ctx.writeAndFlush(buffer); }}}); System.out.println(" client OK "); ChannelFuture = bootstrap.connect("127.0.0.1", 6668).sync(); channelFuture.channel().closeFuture().sync(); } finally { eventExecutors.shutdownGracefully(); }}}Copy the code

As you can see, the server receives all the messages at once

unpacking

To see how the server receives packets, set the server receive buffer to 16 bytes and the client sends packets of 18 bytes at a time

public class NettyServer { public static void main(String[] args) throws InterruptedException { // 1. Create two thread groups: bossGroup and workerGroup // 2. BossGroup only handles connection requests. The real and client business processing will be assigned to workerGroup // 3. BossGroup = new NioEventLoopGroup(); EventLoopGroup workerGroup = new NioEventLoopGroup(); ServerBootstrap = new ServerBootstrap(); ServerBootstrap = new ServerBootstrap(); // Set two thread groups bootstrap.group(bossGroup, WorkerGroup) / / using NioServerSocketChannel as the server channel implementation. The channel (NioServerSocketChannel. Class) / / set the receiver ByteBuf numerical, Fixed buffer receives the number of bytes for 16. ChildOption (ChannelOption RCVBUF_ALLOCATOR, new AdaptiveRecvByteBufAllocator (16, 16, ChildHandler (new ChannelInitializer<SocketChannel>() {@override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new MyLoggingHandler()); }}); System.out.println("server is ready"); ChannelFuture cf = bootstrap.bind(6668).sync(); ChannelFuture cf = bootstrap.bind(6668).sync() Cf.channel ().closeFuture().sync(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); }}}Copy the code
public static void main(String[] args) throws InterruptedException { send(); } private static void send() throws InterruptedException { EventLoopGroup eventExecutors = new NioEventLoopGroup(); Try {// Instead of using ServerBootstrap, use Bootstrap = new Bootstrap(); bootstrap.group(eventExecutors) .channel(NioSocketChannel.class) .handler(new ChannelInboundHandlerAdapter() { @Override  public void channelActive(ChannelHandlerContext ctx) throws Exception { for (int i = 0; i < 1; i++) { ByteBuf buffer = ctx.alloc().buffer(16); WriteBytes (new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17}); ctx.writeAndFlush(buffer); }}}); System.out.println(" client OK "); ChannelFuture = bootstrap.connect("127.0.0.1", 6668).sync(); channelFuture.channel().closeFuture().sync(); } finally { eventExecutors.shutdownGracefully(); }}Copy the code

As you can see, the 18 bytes are received in two separate packets

How to solve the problem of sticking half package

The root cause of sticky packet and half packet problem is that the server does not know how to parse the sent packet, so we need to give the rules for packet parsing. Netty has many built-in handlers to solve the sticky packet unpacking problem

FixedLengthFrameDecoder

As shown in the figure above, divide the data by specifying the length of the data

public class NettyServer { public static void main(String[] args) throws InterruptedException { EventLoopGroup bossGroup  = new NioEventLoopGroup(); EventLoopGroup workerGroup = new NioEventLoopGroup(); try { ServerBootstrap bootstrap = new ServerBootstrap(); bootstrap.group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .childHandler(new ChannelInitializer<SocketChannel>() { @Override Protected void initChannel(SocketChannel SocketChannel) throws Exception {// Add a fixed-length decoder of specified length, Socketchannel.pipeline ().addLast(new FixedLengthFrameDecoder(9)); socketChannel.pipeline().addLast(new MyLoggingHandler()); }}); System.out.println("server is ready"); ChannelFuture cf = bootstrap.bind(6668).sync(); cf.channel().closeFuture().sync(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); }}}Copy the code
public class NettyClient { public static void main(String[] args) throws InterruptedException { send(); } private static void send() throws InterruptedException { EventLoopGroup eventExecutors = new NioEventLoopGroup(); try { Bootstrap bootstrap = new Bootstrap(); bootstrap.group(eventExecutors) .channel(NioSocketChannel.class) .handler(new ChannelInboundHandlerAdapter() { @Override  public void channelActive(ChannelHandlerContext ctx) throws Exception { for (int i = 0; i < 1; i++) { ByteBuf buffer = ctx.alloc().buffer(16); WriteBytes (new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17}); ctx.writeAndFlush(buffer); }}}); System.out.println(" client OK "); ChannelFuture = bootstrap.connect("127.0.0.1", 6668).sync(); channelFuture.channel().closeFuture().sync(); } finally { eventExecutors.shutdownGracefully(); }}}Copy the code

As you can see, it was received in two batches

Specify the line Base framedecoder

The packet is segmented according to \n or \r\n. The LineBasedFrameDecoder constructor has a maxLength parameter, meaning that an exception will be thrown if no separator is found within this maximum length

The server code is the same as before, but with a new Handler

.childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new LineBasedFrameDecoder(1024)); socketChannel.pipeline().addLast(new MyLoggingHandler()); }});Copy the code

The client code modifies the packets sent by adding \n to each String

bootstrap.group(eventExecutors)
        .channel(NioSocketChannel.class)
        .handler(new ChannelInboundHandlerAdapter() {
          @Override
          public void channelActive(ChannelHandlerContext ctx) throws Exception {
            ByteBuf buffer = ctx.alloc().buffer(38);
​
            for (int i = 0; i < 3; i++) {
              StringBuilder sb = new StringBuilder();
              sb.append(UUID.randomUUID().toString());
              sb.append("\n");
              buffer.writeBytes(sb.toString().getBytes(StandardCharsets.UTF_8));
            }
​
            ctx.writeAndFlush(buffer);
          }
  });
Copy the code

As you can see, they are separated by UUID

Of course, we also can specify separator, DelimiterBasedFrameDecoder this class to implement the use of Netty

LengthFieldBasedFrameDecoder

This decoder is powerful, can customize message content, and is complex to use

As shown in the figure above, the sent message consists of the above parts associated with the parameters in the following constructors

MaxFrameLength, lengthfield testset, lengthFieldLength, lengthAdjustment, and initialby teststrip

  1. Length mark: LengthFieldOffset +lengthFieldLength specifies the length of the stored data. For example, lengthFieldOffset=0 and lengthFieldLength=2 indicate that the first two bytes store the length of the stored data. If the data stored in these two bytes is0x0009, indicates that the length of the specific data is 9
  1. Additional information: The message version number or other information can be added here, which is not specific data, so when we actually cut the data, we need to cut this part of the data, we can use lengthAdjustment to adjust, which means where do I need to start reading the data
  1. Concrete data: the actual content of the packet
  2. Unused space: maxFrameLength is used to define the maximum length. The unused space corresponds to this
  3. InitialBytesToStrip: because the packet contains the length of logo, additional information such as data, when we really read if you don’t need this information, you can use this field is removed
Public class EncoderTest {public static void main(String[] args) {// Test handler EmbeddedChannel using EmbeddedChannel channel = new EmbeddedChannel( new LengthFieldBasedFrameDecoder(1024, 0, 4, 1, 5), new MyLoggingHandler() ); / / simulation client, write data ByteBuf buffer = ByteBufAllocator. DEFAULT. Buffer (); send(buffer, "Hello"); channel.writeInbound(buffer); send(buffer, "World"); channel.writeInbound(buffer); } private static void send(ByteBuf buf, String message) {private static void send(ByteBuf buf, String message) {int length = message.length(); byte[] bytes = message.getBytes(StandardCharsets.UTF_8); // The data length is 4 bytes buf.writeint (length); // Additional information buf.writebyte (0x01); // Specific data buf.writebytes (bytes); }}Copy the code

As you can see, the resolution is based on the constructor parameters

The resources

www.bilibili.com/video/BV1py…