Introduction to the
Unpacking and sticky packets are common in socket programming. During socket communication, if one end of communication sends multiple packets continuously, TCP packets are packaged into one TCP packet and sent out, which is called sticky packets. If the number of packets sent by one end exceeds the maximum number of TCP packets, a packet is split into multiple TCP packets of the maximum TCP length for transmission. This is called packet unpacking.
Some basic concepts
MTU
The maximum transmission unit in a communication protocol. It is used to describe the maximum transmission unit of the data link layer in the TCP/IP layer 4 protocol. The MTU of different types of networks varies. The MTU of Ethernet is 1500, that is, it can transmit a maximum of 1500 bytes of data frames. You can run the ifconfig command to view the MTU of each network adapter on a PC.
MSS
This parameter specifies the maximum length of TCP packets that can be transmitted after a TCP connection is established. It is used by TCP to limit the maximum number of bytes that can be sent at the application layer. If the MTU at the bottom is 1500 bytes, MSS = 1500-20 (IP Header) -20 (TCP Header) = 1460 bytes.
Schematic diagram
As shown in the figure, the channel between the client and the server represents the TCP transmission channel, and the square between the two arrows represents a TCP packet. Normally, a TCP packet transmits one application data. Sticky packet is when two or more application packets are glued together for transmission through a TCP connection. In the case of unpacking, one application packet is split into two segments and transmitted separately, and the other segments may be bonded with other application packets.
Scenario instance
The following shows the process of packet sticking and unpacking by simply realizing the communication between two sockets. Both clients and servers communicate on the local computer, with the server listening on the client using 127.0.0.1 and the client initiating a connection using 127.0.0.1.
1. The package
A. Implement server code, the server listens on port 55533, does not specify the IP address default is localhost, that is, the local IP loopback address 127.0.0.1, then wait for the client to connect, the code is as follows:
public class SocketServer {
public static void main(String[] args) throws Exception {
// Listen on the specified port
int port = 55533;
ServerSocket server = new ServerSocket(port);
// The server will wait for the connection to arrive
System.out.println("The server will wait for the connection to arrive");
Socket socket = server.accept();
// After the connection is established, the input stream is fetched from the socket and the buffer is set up for reading
InputStream inputStream = socket.getInputStream();
byte[] bytes = new byte[1024 * 1024];
int len;
while((len = inputStream.read(bytes)) ! = -1) {
// Specify the encoding format. The sender and receiver must be the same. Utf-8 is recommended
String content = new String(bytes, 0, len,"UTF-8");
System.out.println("len = " + len + ", content: "+ content); } inputStream.close(); socket.close(); server.close(); }}Copy the codeB. Realize the client code to connect the server. After the connection is established, the client will send 100 consecutive same strings;
public class SocketClient {
public static void main(String[] args) throws Exception {
// The IP address and port of the server to be connected
String host = "127.0.0.1";
int port = 55533;
// Establish a connection with the server
Socket socket = new Socket(host, port);
// Get the output stream after the connection is established
OutputStream outputStream = socket.getOutputStream();
String message = "It's a whole bag!!";
for (int i = 0; i < 1; i++) {
//Thread.sleep(1);
outputStream.write(message.getBytes("UTF-8"));
}
Thread.sleep(20000); outputStream.close(); socket.close(); }}Copy the codeC. Run the server code first, block when running to server.accept(), print “Server will always wait for connection” to wait for the connection to the client, and then run the client code;
D. After the client code is run, you can see the following output from the server console:
The server will wait for the connection len =21Content: It's a whole package!! len =168Content: It's a whole package!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! len =105Content: It's a whole package!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! len =42Content: It's a whole package!! It's a whole pack!! len =42Content: It's a whole package!! It's a whole pack!! len =63Content: It's a whole package!! It's a whole pack!! It's a whole pack!! len =42Content: It's a whole package!! It's a whole pack!! len =21Content: It's a whole package!! len =42Content: It's a whole package!! It's a whole pack!! len =21Content: It's a whole package!! len =147Content: It's a whole package!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! len =63Content: It's a whole package!! It's a whole pack!! It's a whole pack!! len =21Content: It's a whole package!! len =252Content: It's a whole package!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!! It's a whole pack!!Copy the codeIn the original understanding, the client sends the string “this is a whole package!!” one at a time. , each sent 50 times. The server should also receive it 50 times, printing 50 lines of the same string. But it turned out to be an unusual result, the result of sticky bags.
The reasons for sticky packets are as follows: 1. The size of the data to be sent is smaller than the size of the TCP send buffer. TCP sends the data that has been written into the buffer for several times. 2. The application layer of the receiving end fails to read the data in the receiving buffer in time. 3. Data is sent too fast, and data packets are piled up in the buffer before being sent out at a time (if the client sleeps for a period of time after sending a data packet, the sticky packet will not occur);
2. Unpacking
If the packet is too large, exceeding the size of the MSS, the packet is split into multiple TCP packets and transmitted separately. So to demonstrate unpacking, you need to send a data larger than the MSS size, which depends on the MTU size of the network through which the data passes. Because the IP address of the client and server in the socket is 127.0.0.1, data is transmitted only between loopback nics. Therefore, the MSS of the client and server are mTU-20 (IP Header) -20 (TCP Header) of the loopback NICS. The following are the procedures for unpacking.
A. MAC can check the MTU of each local network adapter through ifconfig. The following is part of the output of my computer after running ifconfig, where LO0 is the loopback network adapter, and the MTU is 16384:
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
inet 127.0. 01. netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
nd6 options=201<PERFORMNUD,DAD>
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 88:e9:fe:76:dc:57
inet6 fe80::18d4:84fb:fa10:7f8%en0 prefixlen 64 secured scopeid 0x6
inet 192.1681.8. netmask 0xffffff00 broadcast 192.1681.255.
inet6 240e:d2:495f:9700:182a:c53f:c720:5f63 prefixlen 64 autoconf secured
inet6 240e:d2:495f:9700:d96:48f2:8108:2b33 prefixlen 64 autoconf temporary
nd6 options=201<PERFORMNUD,DAD>
media: autoselect
status: active
en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=60<TSO4,TSO6>
ether 7a:00:5c:40:cf:01
media: autoselect <full-duplex>
status: inactive
en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=60<TSO4,TSO6>
ether 7a:00:5c:40:cf:00
media: autoselect <full-duplex>
status: inactive
......
Copy the codeB. The server code is the same as when the packet is attached. Change the client code to send a string of more than 16384 bytes. I’m just going to replace it with a little text. The client code is as follows:
public class SocketClient {
private final static String CONTENT = "This is a very, very long string this is a very, very long string this is a very, very long string this is a very..... It's a very, very long string it's a very, very long string it's a very, very long string it's a very, very long string it's a very, very long string";// The test is larger than 5461 characters. Due to space constraints, only this paragraph is used as a representative
public static void main(String[] args) throws Exception {
// The IP address and port of the server to be connected
String host = "127.0.0.1";
int port = 55533;
// Establish a connection with the server
Socket socket = new Socket(host, port);
// Get the output stream after the connection is established
OutputStream outputStream = socket.getOutputStream();
outputStream.write(CONTENT.getBytes("UTF-8"));
Thread.sleep(20000); outputStream.close(); socket.close(); }}Copy the codeC. As in the sticky package code example, run the original server code first, then run the client code, and see the server printout. Len = 22328, content: This is a very, very long string this is a very, very long string this is a very, very long string this is a very….. It’s a very, very long string it’s a very, very long string it’s a very, very long string it’s a very, very long string it’s a very, very long string it’s a very, very long string Through the output log, it can be found that the string sent by the client is not broken at the server, but read the whole string sent by the client at one time. This is because the string is split into two TCP packets and sent to the server’s buffer data stream. The server reads the data in the stream once, and the result is that the two TCP datagrams are concatenated. Tcpdump can be used to capture packets to see the data transfer details:
Run sudo tcpdump -i lo0 ‘port 55533’ on the console to monitor packets transmitted on port 55533 on loopback NETWORK interface card (NIC) LO0. Packets coming in and out of port 55533 will be captured and printed out. This command requires administrator permission. At this time, we run it again according to the test steps just now, and the result of packet capture is as follows:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo0, link-type NULL (BSD loopback), capture size 262144 bytes
23:15:44.641208 IP 192.168.1.8.58748 > 192.168.1.8.55533: Flags [S], seq 2331897419, win 65535, options [mss 16344,nop,wscale 6,nop,nop,TS val 261991443 ecr 0,sackOK,eol], length 0
23:15:44.641261 IP 192.168.1.8.55533 > 192.168.1.8.58748: Flags [S.], seq 3403812509, ack 2331897420, win 65535, options [mss 16344,nop,wscale 6,nop,nop,TS val 261991443 ecr 261991443,sackOK,eol], length 0
23:15:44.641270 IP 192.168.1.8.58748 > 192.168.1.8.55533: Flags [.], ack 1, win 6379, options [nop,nop,TS val 261991443 ecr 261991443], length 0
23:15:44.641279 IP 192.168.1.8.55533 > 192.168.1.8.58748: Flags [.], ack 1, win 6379, options [nop,nop,TS val 261991443 ecr 261991443], length 0
23:15:44.644808 IP 192.168.1.8.58748 > 192.168.1.8.55533: Flags [.], seq 1:16333, ack 1, win 6379, options [nop,nop,TS val 261991446 ecr 261991443], length 16332
23:15:44.644812 IP 192.168.1.8.58748 > 192.168.1.8.55533: Flags [P.], seq 16333:22329, ack 1, win 6379, options [nop,nop,TS val 261991446 ecr 261991443], length 5996
23:15:44.644835 IP 192.168.1.8.55533 > 192.168.1.8.58748: Flags [.], ack 22329, win 6030, options [nop,nop,TS val 261991446 ecr 261991446], length 0
Copy the code1. In the third line, the client initiates a connection request, and there is a parameter of MSS 16344 in the options parameter, which indicates the maximum size of TCP packets that the client can receive after the connection is established. If the maximum size is exceeded, the packets will be unpacked and transmitted separately. 2. The first four lines are connected at both ends; 3. The client port 58748 in the fifth row transmits data packets with the size of 16,332 bytes to the service port 55533. 4. In the sixth line, client port 58748 transmits data packets of 5996 bytes to service port 55533.
As can be seen from the packet capture process, the client sends a string, which is split into two TCP datagrams for transmission.
The solution
In the case of sticky packages, unpack the packages that are stuck together. In the case of unpacking, it is necessary to glue the unpacked package, that is, to combine a disassembled complete application package into a complete package. It is common practice to add a four-byte packet length value before each application packet is sent, indicating the true length of the application packet. The following figure shows the application packet format.
Below, I modify the previous code examples to solve the problem of unpacking and sticking packages. There are two ways to do it: 1. One way is to introduce netTY library, netTY encapsulates a variety of unpacking sticky package way, only need to be familiar with the interface and call, reduce their own processing data protocol tedious process; 2. Write the protocol encapsulation and parsing process by yourself, which is equivalent to realizing the simple version of Netty library unsticking package. This article is to learn the needs of this way:
A. Client. Each time a string is sent, the string is converted into a byte array, and a four-byte representation of the length of the data is added before the original data byte array, and then the combined byte array is sent out;
public class SocketClient {
public static void main(String[] args) throws Exception {
// The IP address and port of the server to be connected
String host = "127.0.0.1";
int port = 55533;
// Establish a connection with the server
Socket socket = new Socket(host, port);
// Get the output stream after the connection is established
OutputStream outputStream = socket.getOutputStream();
String message = "It's a whole bag!!";
byte[] contentBytes = message.getBytes("UTF-8");
System.out.println("contentBytes.length = " + contentBytes.length);
int length = contentBytes.length;
byte[] lengthBytes = Utils.int2Bytes(length);
byte[] resultBytes = new byte[4 + length];
System.arraycopy(lengthBytes, 0, resultBytes, 0, lengthBytes.length);
System.arraycopy(contentBytes, 0, resultBytes, 4, contentBytes.length);
for (int i = 0; i < 10; i++) {
outputStream.write(resultBytes);
}
Thread.sleep(20000); outputStream.close(); socket.close(); }}public final class Utils {
//int is converted to a byte array
public static byte[] int2Bytes(int i) {
byte[] result = new byte[4];
result[0] = (byte) (i >> 24 & 0xFF);
result[1] = (byte) (i >> 16 & 0xFF);
result[2] = (byte) (i >> 8 & 0xFF);
result[3] = (byte) (i & 0xFF);
return result;
}
// The byte array is converted to an int value
public static int bytes2Int(byte[] bytes){
int num = bytes[3] & 0xFF;
num |= ((bytes[2] < <8) & 0xFF00);
num |= ((bytes[1] < <16) & 0xFF0000);
num |= ((bytes[0] < <24) & 0xFF000000);
returnnum; }}Copy the codeB. Server. After receiving the byte array sent from the client, extract the first four bytes and convert them into int values, then take the number of bytes of the int value back, and then convert them into a string, which is the data sent from the client side, see the code:
public class SocketServer {
public static void main(String[] args) throws Exception {
// Listen on the specified port
int port = 55533;
ServerSocket server = new ServerSocket(port);
// The server will wait for the connection to arrive
System.out.println("The server will wait for the connection to arrive");
Socket socket = server.accept();
// After the connection is established, the input stream is fetched from the socket and the buffer is set up for reading
InputStream inputStream = socket.getInputStream();
byte[] bytes = new byte[1024 * 128];
int len;
byte[] totalBytes = new byte[] {};int totalLength = 0;
while((len = inputStream.read(bytes)) ! = -1) {
//1. Combine the read data with the data left over from the previous one
int tempLength = totalLength;
totalLength = len + totalLength;
byte[] tempBytes = totalBytes;
totalBytes = new byte[totalLength];
System.arraycopy(tempBytes, 0, totalBytes, 0, tempLength);
System.arraycopy(bytes, 0, totalBytes, tempLength, len);
while (totalLength > 4) {
byte[] lengthBytes = new byte[4];
System.arraycopy(totalBytes, 0, lengthBytes, 0, lengthBytes.length);
int contentLength = Utils.bytes2Int(lengthBytes);
//2. If the remaining data is smaller than the length of the data header, the packet is unpacked and the data connection is obtained again
if (totalLength < contentLength + 4) {
break;
}
//3. Data with a specified length on the data header label is application data
byte[] contentBytes = new byte[contentLength];
System.arraycopy(totalBytes, 4, contentBytes, 0, contentLength);
// Specify the encoding format. The sender and receiver must be the same. Utf-8 is recommended
String content = new String(contentBytes, "UTF-8");
System.out.println("contentLength = " + contentLength + ", content: " + content);
//4. Remove the read data
totalLength -= (4 + contentLength);
byte[] leftBytes = new byte[totalLength];
System.arraycopy(totalBytes, 4 + contentLength, leftBytes, 0, totalLength); totalBytes = leftBytes; } } inputStream.close(); socket.close(); server.close(); }}Copy the codeC. Print results:
The server will wait for the connection to arrive contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!! contentLength =21Content: It's a whole package!!Copy the codeThe client sends ten strings in a row, and the server receives a separate ten strings. Multiple packets are no longer connected.
In the Netty application layer, data is sent according to the unit of ByteBuf, but in the underlying operating system, data is still sent according to the byte stream. Therefore, a second assembly is needed from the bottom layer to the application layer.
At the bottom of the operating system, it is read in the way of byte stream. At the Netty application level, it needs to assemble ByteBuf twice.
This is the root of sticky packets and half packets.
At the Netty level, ByteBuf is a read of the underlying buffer when assembled, and there is a problem.
First of all, the data capacity of the upper layer application layer to read the data from the bottom buffer is limited. When the TCP bottom buffer data packet is large, it is divided into multiple reads, resulting in packet break. For the application layer, it is half packet.
Second, if the upper application layer reads multiple low-level buffered packets at once, it is sticky.
How to solve it?
The basic idea is that at the receiving end, we need to read the underlying packets and reassemble the packets of our application layer according to the custom protocol. This process is usually called unpacking at the receiving end.
Principle of packet unpacking The basic principle of packet unpacking is as follows: The receiving application layer continuously reads data from the underlying TCP buffer. After each reading, check whether it is a complete application layer packet. If yes, the upper-layer application layer data packet is read. If not, the data is kept in the application-layer buffer and read from the TCP buffer until a complete application-layer packet is returned. At this point, the half pack problem can be solved.
If multiple application layer packets are read from TCP, the whole application layer buffer is split into independent application layer packets and returned to the calling program.
At this point, sticky bag problem to be solved.
Netty unpack this job, Netty has prepared a lot of different unpacks for you. In the spirit of not reinventing the wheel, we went straight to Netty’s off-the-shelf unpacker.
The unpacking devices in Netty are as follows:
The length framedecoder for each application layer packet is a fixed length size, such as 1024 bytes. This is obviously not suitable for practical use in Java chat programs. 2. LineBasedFrameDecoder. Each application layer data packet is separated by a newline character. This is obviously not suitable for practical use in Java chat programs. 3. The delimiter unpacker DelimiterBasedFrameDecoder, each application layer packet, through custom delimiter, dividing a split. This version, which is the generic version of Linebase Framedecoder, is essentially the same. This is obviously not suitable for practical use in Java chat programs. 4. Based on the packet length unpacker LengthFieldBasedFrameDecoder, application layer packet length, as the receiver application layer packet split basis. Unpack packets based on the size of application layer packets. The unpacker has a requirement that the application layer protocol include the length of the packet. This is obviously more suitable for practical application in Java chat programs. Now let’s apply the splitter.
Unpacking before the news of the package in use LengthFieldBasedFrameDecoder unpacker before, the sender need to round the news of the protobuf package packing.
The method of sending side packaging is:
Prefixes four bytes to the actual Protobuf binary message packet.
The first two bytes are the version number and the last two bytes are the length of the actual sent protobuf message.
Binary message wrapping, again, happens at the sending end.
Modify the encoder ProtobufEncoder of the sending end, the code is as follows:
/** * encoder */
public class ProtobufEncoder extends MessageToByteEncoder<ProtoMsg.Message>
{
@Override
protected void encode(ChannelHandlerContext ctx, ProtoMsg.Message msg, ByteBuf out)
throws Exception
{
byte[] bytes = msg.toByteArray();// Convert the object to byte
int length = bytes.length;// Read the length of the ProtoMsg message
ByteBuf buf = Unpooled.buffer(2 + length);
// Write the version of the message protocol first, that is, the header
buf.writeShort(Constants.PROTOCOL_VERSION);
// Write the length of the ProtoMsg message
buf.writeShort(length);
// Write the body of the ProtoMsg message
buf.writeBytes(bytes);
/ / sendout.writeBytes(buf); }}Copy the code
The steps on the sending side are:
The version of the message protocol is written first, which is the header
buf.writeShort(Constants.PROTOCOL_VERSION);
Write the length of the ProtoMsg message to buf.writeShort(length);
Finally, write the body of the ProtoMsg message buf.writeBytes(bytes);
Develop a custom unpacker at the receiving end use Netty, based on the length of the domain unpacker LengthFieldBasedFrameDecoder, according to the actual application layer packet length to break up.
Two things need to be done:
Sets the location of the length information (length field) in the packet.
Sets the length of the message (length field) itself, that is, the number of bytes.
In the previous section, our length information (length field) takes up 2 bytes; The position in a message where the length information (length field) comes after the version number.
The version number is 2 bytes, counting from 0, and the length information (length field) is 2 in the packet.
This data is defined in the Constansts constant class.
public class Constants
{
// Protocol version number
public static final short PROTOCOL_VERSION = 1;
// Header length: version number + packet length
public static final short PROTOCOL_HEADLENGTH = 4;
// Length offset
public static final short LENGTH_OFFSET = 2;
// The number of bytes in length
public static final short LENGTH_BYTES_COUNT = 2;
}
Copy the code
With these data, can be based on the length of the Netty unpacker LengthFieldBasedFrameDecoder, develop their own length divider.
The newly developed splitter is PackageSpliter, and the code is as follows:
package com.crazymakercircle.chat.common.codec;
public class PackageSpliter extends LengthFieldBasedFrameDecoder
{
public PackageSpliter(a) {
super(Integer.MAX_VALUE, Constants.LENGTH_OFFSET,Constants.LENGTH_BYTES_COUNT);
}
@Override
protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
return super.decode(ctx, in); }}Copy the code
Divider PackageSpliter inherited LengthFieldBasedFrameDecoder, introduced to the three parameters.
Length offset, here Constants.LENGTH_OFFSET, value 2
Number of bytes of length, here Constants.LENGTH_BYTES_COUNT, value 2
The maximum application package length, in this case integer. MAX_VALUE, is unlimited
After the splitter is written, just add the splitter to the front of the pipeline to use the splitter (custom unpacker).
The actual application of a custom unpacker is to add this splitter to the front of the server-side pipeline, with the following code:
package com.crazymakercircle.chat.server;
/ /...
@Service("ChatServer")
public class ChatServer
{
static final Logger LOGGER = LoggerFactory.getLogger(ChatServer.class);
/ /...
// Create a channel when a connection arrives
protected void initChannel(SocketChannel ch) throws Exception
{ // Apply a custom unpacker
ch.pipeline().addLast(new PackageSpliter());
ch.pipeline().addLast(new ProtobufDecoder());
ch.pipeline().addLast(new ProtobufEncoder());
// Manage the Handler in the pipeline channel
// Add a handler to the channel queue to process the business
ch.pipeline().addLast("serverHandler", serverHandler); }});//....
}
Copy the code
Add the splitter to the front of the pipeline on the sending side, and the code is similar and won’t be described here. You can download the source code to view.