This article has participated in the third “topic writing” track of the Denver Creators Training Camp. For details, check out: Digg Project | Creators Training Camp third is ongoing, “write” to make a personal impact.

TCP packet sticking and unpacking are low-level network problems, which may occur at the data link layer, network layer, and transport layer. Most of the daily network application development occurs at the transport layer, while UDP is protected by the message boundary, there is no sticky packet, unpack problem, only occurs in TCP protocol. Suppose the client sends two consecutive packets Packet1 and Packet2 to the server.

There are three possible scenarios in this process:

  • Normal: Two packets are sent separately one by one
  • Sticky packets: two packets sent together,
  • Unpacking: The Server receives incomplete or partial packets

As shown below:

Reasons for sticking/unpacking

There are many reasons for TCP sticky or unpacked packets. Here are the common ones:

  1. Packet unpacking occurs when the data to be sent is larger than the remaining space of the TCP send buffer.
  2. If the data to be sent is larger than MSS (maximum packet length), TCP unpacks the data before transmission.
  3. The size of the data to be sent is smaller than the size of the TCP send buffer. TCP sends the data that has been written into the buffer for several times. Packet sticking occurs.
  4. The application layer at the receiving end does not read the data in the receiving buffer in time, and sticky packets occur.

Sticking package, unpacking solutions

TCP itself is stream-oriented, as a network server, how to split or merge meaningful information from this continuous flow of data? There are some common methods:

  1. The sender adds a header to each packet, and the header should contain at least the length of the packet. In this way, after receiving data, the receiver can know the actual length of each packet by reading the length field in the header.
  2. The sender encapsulates each packet into a fixed length (if not, it can be filled by padding zeros), so that the receiver automatically splits each packet by reading a fixed length of data from the receive buffer.
  3. A boundary can be set between packets, such as a special symbol, so that the receiver can separate different packets through this boundary.

For example

The following is the data package of my previous project for reference only:

Header definition

The serial number field Length (bytes) instructions
1 headFlag 4 0 to 3 Flag bit: fixed four characters’ AABB ‘nuggets community: fixed XTXT
2 version 2, 4-5 Major version. Minor version, one byte each. This issue is 00 01
3 packetNo 4, 6-9 Unsigned integer, loop between 1-int.Max, reply must be the same as request packet number
4 length 4 10 to 13 Data length of data area
5 direction 1 to 14 0: request, 1: reply
6 command 2 15 to 16 The interface number is 1-66536
7 data N 17-length+16 Length Specifies the length. The value is in JSON format and utF-8 encoding format
8 crc16 4 length+17 – 18 Verify all data from headFlag to data(including) CRC16 (CCITT-Xmodem)

In fact, this design is the solution using method 1 of the solution:

First, I will set a message header flag to a fixed length of 4 bytes, such as “XTXT” for mining community, and then fix the first 17 bits in order: identifier, version number, serial number, data length, transmission direction, packet type. The last part is the data section and the last CRC16 check code.

Here is the unpacking logic, the code is relatively simple:

Try {// TODO to be optimized byte[] head = new byte[17]; in.readBytes(head); byte[] h = new byte[4]; System.arraycopy(head, 0, h, 0, 4); // flag 4 String f = new String(h, "UTF-8"); // version 2 short v = ByteUtil.byteArrayToShort(new byte[]{head[4], head[5]}); // serial number 4 int no = ByteUtil.byteArrayToInt(new byte[]{head[6], head[7], head[8], head[9]}); // length 4 int len = ByteUtil.byteArrayToInt(new byte[]{head[10], head[11], head[12], head[13]}); // direction byte d = head[14]; // command short com = ByteUtil.byteArrayToShort(new byte[]{head[15], head[16]}); byte[] content = new byte[len]; in.readBytes(content); int cc = in.readInt(); byte[] packet = new byte[17 + len]; System.arraycopy(head, 0, packet, 0, head.length); System.arraycopy(content, 0, packet, 17, len); int i = CRC16.crc16CcittXmodem(packet); if (cc ! = i) { logger.error("decode crc16 fail {}", cc); } // MessageProtocol protocol = new MessageProtocol(); protocol.setHeadFlag(f); protocol.setVersion(v); protocol.setPacketNo(no); protocol.setLength(len); protocol.setDirection(d); protocol.setCommand(com); protocol.setData(new String(content, "UTF-8")); protocol.setCrc16(cc); logger.info("encode MessageProtocol:{}", protocol); out.add(protocol); } catch (Throwable t) { logger.error("decode fail", t); throw t; }Copy the code

The resources

  • Blog.csdn.net/weixin_4499…
  • Github.com/zhengsh/ssm…