Analysis of TCP sticky packet problem and its solution

Recently I have been doing middleware related things, so I have come into contact with a variety of protocols, generally SPEAKING, TCP, UDP, HTTP and other network transmission protocols, so I want to start from the most basic PROTOCOL TCP sticky packet problem, the computer network this part of the foundation of a solid.

This section describes TCP

TCP is a connection-oriented transport layer protocol

In simple terms, a TCP connection must be established before TCP can be used, which is often referred to as the three-way handshake. After data transfer is complete, the established TCP connection must be released. Otherwise, unexpected problems may occur, causing service unavailability.

Every TCP connection is a reliable connection and has only two endpoints

The TCP connection is point-to-point from the Server to the Client. Data is transmitted through TCP without error, repetition, and loss.

TCP communication is full-duplex

TCP allows applications on both sides of the communication to send data at any time. Both ends of the TCP connection are equipped with send buffer and receive buffer to temporarily store bidirectional communication data. When sending data, the application sends the data to TCP’s buffer and then does its own thing, while TCP sends the data out when appropriate. Upon receiving, TCP puts the received data into the receive buffer, and the upper layer application reads the data when appropriate.

TCP is byte stream oriented

A stream in TCP is a sequence of bytes flowing into or out of a process. Therefore, TCP communication to Java, Golang and other high-level languages requires the corresponding entity serialization before transmission. Also, when we use Redis to cache, we need to serialize the data into Redis, because the bottom layer of Redis is the TCP protocol.

TCP does not know the meaning of the byte stream being transmitted, and TCP does not guarantee that the block of data sent by the receiver application and the sender application will have a corresponding size relationship (this is the sticky packet problem during TCP transmission). But the application receiver ends up receiving the same byte stream that the sender sends. Therefore, when using TCP, we should formulate a reasonable sticky packet unpacking policy.

The following figure shows the whole process of TCP protocol transmission:

TCP Byte – oriented stream

TCP sticky packets reappear

The theory of scrutiny

TCP sticky packet unpacking code practice

The following code mainly demonstrates the use of a specified message header, message body to solve the TCP sticky packet, unpack the problem.

Server-side code: The main logic of server-side code is to receive the message sent from the client, reassemble the message, and print it out.

import java.io.*; import java.net.InetSocketAddress; import java.net.ServerSocket; import java.net.Socket; /** * @author wuhf * @Date 2018/7/16 15:50 **/ public class TestSocketServer { public static void main(String args[]) { ServerSocket serverSocket; try { serverSocket = new ServerSocket(); serverSocket.bind(new InetSocketAddress(8089)); while (true) { Socket socket = serverSocket.accept(); new ReceiveThread(socket).start(); } } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } static class ReceiveThread extends Thread { public static final int PACKET_HEAD_LENGTH = 2; // Header length private Socket Socket; private volatile byte[] bytes = new byte[0]; public ReceiveThread(Socket socket) { this.socket = socket; } public byte[] mergebyte(byte[] a, byte[] b, int begin, int end) { byte[] add = new byte[a.length + end - begin]; int i = 0; for (i = 0; i < a.length; i++) { add[i] = a[i]; } for (int k = begin; k < end; k++, i++) { add[i] = b[k]; } return add; } @Override public void run() { int count = 0; while (true) { try { InputStream reader = socket.getInputStream(); if (bytes.length < PACKET_HEAD_LENGTH) { byte[] head = new byte[PACKET_HEAD_LENGTH - bytes.length]; int couter = reader.read(head); if (couter < 0) { continue; } bytes = mergebyte(bytes, head, 0, couter); if (couter < PACKET_HEAD_LENGTH) { continue; Byte [] temp = new byte[0]; temp = mergebyte(temp, bytes, 0, PACKET_HEAD_LENGTH); String templength = new String(temp); int bodylength = Integer.parseInt(templength); If (bytes.length - PACKET_HEAD_LENGTH < bodylength) {byte[] body = new byte[bodyLength + PACKET_HEAD_LENGTH - bytes.length]; Int couter = reader.read(body); if (couter < 0) { continue; } bytes = mergebyte(bytes, body, 0, couter); if (couter < body.length) { continue; } } byte[] body = new byte[0]; body = mergebyte(body, bytes, PACKET_HEAD_LENGTH, bytes.length); count++; System.out.println("server receive body: " + count + new String(body)); bytes = new byte[0]; } catch (Exception e) { e.printStackTrace(); } } } } }Copy the code

Client-side code: The main logic of the client-side code is to assemble the message to be sent, determine the message header, the message body, and then send to the server.

import java.io.*; import java.net.InetSocketAddress; import java.net.Socket; /** * @author wuhf * @Date 2018/7/16 15:45 **/ public class TestSocketClient { public static void main(String args[]) throws IOException { Socket clientSocket = new Socket(); clientSocket.connect(new InetSocketAddress(8089)); new SendThread(clientSocket).start(); } static class SendThread extends Thread { Socket socket; PrintWriter printWriter = null; public SendThread(Socket socket) { this.socket = socket; try { printWriter = new PrintWriter(new OutputStreamWriter(socket.getOutputStream())); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); }} @override public void run() {String message = "HelloWorld! from clientsocket this is test half packages!" ; for (int i = 0; i < 100; i++) { sendPacket(reqMessage); } if (socket ! = null) { try { socket.close(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } } public void sendPacket(String message) { try { OutputStream writer = socket.getOutputStream(); writer.write(message.getBytes()); writer.flush(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); }}}}Copy the code

summary

Refer to the article

• Netty: TCP sticky packet unpacking problem
• Computer Networks – Xie Xiren – 7th edition