preface
I remember some time ago, one of the gateways on our production failed.
The gateway logic is very simple. It receives the request from the client, parses the message, and sends the SMS.
However, this request is not a common HTTP request, but uses Netty’s custom protocol.
The premise is that the gateway needs to read a complete packet before performing the following logic.
The problem is that one day, the gateway suddenly fails to parse packets. After checking the sending logs of the client, the problem is not found. Finally, many incomplete packets are received through the logs, and some of them are too many.
So I thought it might be the problem caused by TCP unpacking and sticky packets. Finally, I solved the problem by using Netty’s own unpacking tool.
Hence the article.
TCP protocol
The problem is solved, but you still have to think about why. Why? Asking questions is a good programmer.
This starts with the TCP protocol.
TCP is a byte stream oriented protocol that is streaming in nature, so it is not segmented. Like a current, you never know when it’s going to start and when it’s going to end.
So it will unpack or glue packets according to the current socket buffer.
The following figure shows the process of TCP transmission:
The byte stream of the sending end is first passed into the buffer, and then passed into the buffer of the receiving end through the network, and finally obtained by the receiving end.
When we send two complete packets to the receiver:
Normally, two complete packets are received.
But there are also the following:
The received message is a single message, which is made up of two messages sent, making it difficult for an application to process (thus called sticky packets).
It is also possible to receive two packages, but the contents are mutually contained and still cannot be parsed by the application (unpack).
This problem can be solved only by upper-layer applications. Common methods are as follows:
- Adding a newline character to the end of a message indicates that the message is complete, so that the receiver can use the newline character to determine whether the message is complete.
- The message is divided into header and body. You can declare the length of the message in the header, and get the message based on this length (such as the 808 protocol).
- The length of the packet is specified. If the gaps are insufficient, the packet can be intercepted according to the length.
These methods can be manually implemented by adding the corresponding decoder in Netty’s Pipline.
But Netty has done it for us right out of the box.
Such as:
LineBasedFrameDecoderCan be solved based on a newline character.DelimiterBasedFrameDecoderCan be resolved based on delimiters.FixedLengthFrameDecoderLength can be specified.
Unpack and paste strings
Let’s simulate the simplest string transfer.
It was before
Github.com/crossoverJi…
Demonstrate.
Netty client added a portal to loop 100 string packets to the receiver:
/** * Sends a message to the server@param stringReqVO
* @return* /
@ApiOperation("Client sends message, string")
@RequestMapping(value = "sendStringMsg", method = RequestMethod.POST)
@ResponseBody
public BaseResponse<NULLBody> sendStringMsg(@RequestBody StringReqVO stringReqVO){
BaseResponse<NULLBody> res = new BaseResponse();
for (int i = 0; i < 100; i++) {
heartbeatClient.sendStringMsg(stringReqVO.getMsg()) ;
}
// Use the actuator to add
counterService.increment(Constants.COUNTER_CLIENT_PUSH_COUNT);
SendMsgResVO sendMsgResVO = new SendMsgResVO() ;
sendMsgResVO.setMsg("OK"); res.setCode(StatusEnum.SUCCESS.getCode()) ; res.setMessage(StatusEnum.SUCCESS.getMessage()) ;return res ;
}
/** * Send message string **@param msg
*/
public void sendStringMsg(String msg) {
ByteBuf message = Unpooled.buffer(msg.getBytes().length) ;
message.writeBytes(msg.getBytes()) ;
ChannelFuture future = channel.writeAndFlush(message);
future.addListener((ChannelFutureListener) channelFuture ->
LOGGER.info("Client succeeded in sending message manually ={}", msg));
}
Copy the codeThe server can print it directly:
@Override
protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
LOGGER.info("Receive MSG = {}", msg);
}
Copy the codeBy the way, there is a StringDecoder added here:.addlast (new StringDecoder()) essentially parses the message into a string.
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
out.add(msg.toString(charset));
}
Copy the codeThe interface that calls the client in Swagger sends messages to the server 100 times:
Normally, the receiver should print hello 100 times, but if you look at the log, you will find:
The received content is complete, more, less, spliced; This also corresponds to the above mentioned unpacking, sticky package.
How to solve it? This can be done using the LineBasedFrameDecoder mentioned earlier using a newline character.
Use linebase Framedecoder to solve the problem
The LineBasedFrameDecoder is very simple to use and just needs to be added to the Pipline chain.
// String parsing, newline anti-unpacking
.addLast(new LineBasedFrameDecoder(1024))
.addLast(new StringDecoder())
Copy the code1024 is passed in the constructor to indicate that the maximum length is not greater than this value, as can be seen in the source code analysis below.
Then we run the test again to see the results:
Note that since the LineBasedFrameDecoder decoder is determined by a newline character, a complete message needs to be preceded by \n when sent.
The end result:
Careful observation of the log, it was found that no one was removed, sticky package.
The principle of LineBasedFrameDecoder
Here’s how it works:
- The first step is basically
findEndOfLineMethod to find if there is a delimiter in the current message. If so, the location of the delimiter is returned. - To determine whether discarding is required, the default is false, and this logic is used the first time (see below to determine if it needs to be changed to true).
- If a newline character exists in a message, the data is intercepted to that location.
- If there is no newline character, check whether the length of the current packet is larger than the preset length. If the value is greater than that, cache the packet length and set discarding to true.
- If it is necessary to discard, determine whether the newline character is found, then discard the length of the previous record and intercept the data.
- If no newline character is found, the packet length is accumulated and discarded next time.
From this logic, it can be seen that it is to find whether the message contains a newline character and intercept it accordingly.
Because it is read through the buffer, even if there is no newline character, as long as the next packet has a newline character, the data of the previous round will not be lost.
Efficient encoding method Google Protocol
In fact, the above mentioned operation is in the decoding, we can also customize their own unpacking, sticky package tools.
The main purpose of codec is to encode byte streams for transmission and persistent storage in the network.
The Serializable interface can also be implemented in Java to implement serialization, but it is rarely used in some RPC calls due to performance and other reasons.
Google Protocol is an efficient serialization framework, and here’s how to use it in Netty.
The installation
The first step, of course, is installation:
Download the corresponding package from the official website.
Local configuration environment variables:
The installation is successful if the following information is displayed when protoc –version is executed:
Define your own protocol format
Next, you need to define your own protocol format according to the officially required syntax.
For example, I need to define an input/output message format:
BaseRequestProto.proto:
syntax = "proto2";
package protocol;
option java_package = "com.crossoverjie.netty.action.protocol";
option java_outer_classname = "BaseRequestProto";
message RequestProtocol {
required int32 requestId = 2;
required string reqMsg = 1;
}
Copy the codeBaseResponseProto.proto:
syntax = "proto2";
package protocol;
option java_package = "com.crossoverjie.netty.action.protocol";
option java_outer_classname = "BaseResponseProto";
message ResponseProtocol {
required int32 responseId = 2;
required string resMsg = 1;
}
Copy the codethrough
protoc --java_out=/dev BaseRequestProto.proto BaseResponseProto.proto
Copy the codeThe protoc command converts the protocol format just defined into Java code, which is generated in the /dev directory.
Just copy the generated code into our project and introduce dependencies:
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.4.0</version>
</dependency>
Copy the codeCodec using Protocol is also very simple:
public class ProtocolUtil {
public static void main(String[] args) throws InvalidProtocolBufferException {
BaseRequestProto.RequestProtocol protocol = BaseRequestProto.RequestProtocol.newBuilder()
.setRequestId(123)
.setReqMsg("Hello?")
.build();
byte[] encode = encode(protocol);
BaseRequestProto.RequestProtocol parseFrom = decode(encode);
System.out.println(protocol.toString());
System.out.println(protocol.toString().equals(parseFrom.toString()));
}
/** * code *@param protocol
* @return* /
public static byte[] encode(BaseRequestProto.RequestProtocol protocol){
return protocol.toByteArray() ;
}
/** * decode *@param bytes
* @return
* @throws InvalidProtocolBufferException
*/
public static BaseRequestProto.RequestProtocol decode(byte[] bytes) throws InvalidProtocolBufferException {
returnBaseRequestProto.RequestProtocol.parseFrom(bytes); }}Copy the codeBaseRequestProto is used for a demonstration. The final result is the same after coding and decoding. The answer must be the same.
Java files generated using protoc commands have helped us to encapsulate all the encoding and decoding, just a simple call on the line.
You can see that Protocol creates objects using the Builder pattern, which is clear and readable to consumers. More on builders can be found here.
See the official development documentation for more information about Google Protocol.
Combining with the Netty
Netty already has a Built-in Codec for Google Protobuf, which you only need to add to pipline.
Server side:
// Google Protobuf codec
.addLast(new ProtobufDecoder(BaseRequestProto.RequestProtocol.getDefaultInstance()))
.addLast(new ProtobufEncoder())
Copy the codeClient:
// Google Protobuf codec
.addLast(new ProtobufDecoder(BaseResponseProto.ResponseProtocol.getDefaultInstance()))
.addLast(new ProtobufEncoder())
Copy the codeA slight note is that when building a ProtobufDecoder you need to explicitly specify what type the decoder needs to be decoded into.
The client receives the BaseResponseProto response from the server, so I set the corresponding instance.
It also provides an interface to send messages to the server, and when the server receives a particular command it returns the message to the client:
@Override
protected void channelRead0(ChannelHandlerContext ctx, BaseRequestProto.RequestProtocol msg) throws Exception {
LOGGER.info("Receive MSG = {}", msg.getReqMsg());
if (999 == msg.getRequestId()){
BaseResponseProto.ResponseProtocol responseProtocol = BaseResponseProto.ResponseProtocol.newBuilder()
.setResponseId(1000)
.setResMsg("Server Response") .build(); ctx.writeAndFlush(responseProtocol) ; }}Copy the codeCall the relevant interface in Swagger:
In the log, you can see that the server received the message, and the client also received the return:
Although Netty is packaged with Google Protobuf codec tools, if you look at Netty’s codec tools, you will see that they are implemented using the API mentioned above.
Protocol Unpack or glue the package
Google Protocol is really quite simple to use, but there are some important points to be aware of, such as unpacking and sticky packages.
Try a simulation:
Send messages 100 times in a row to see how the server receives them:
Will find that the server at the time of decoding error, in fact, is removed, sticky package.
Netty has also taken this into account, so it has provided relevant tools.
// Unpack and decode
.addLast(new ProtobufVarint32FrameDecoder())
.addLast(new ProtobufVarint32LengthFieldPrepender())
Copy the codeJust add these two codecs to the server and client and send it a hundred times.
The logs show that all 100 messages are received without any exception.
The codec tool can be simply interpreted as adding a 32-bit shaping field to the message body to indicate the current message length.
conclusion
Network is also the basis of the computer, because I am doing related work recently, so I have more contact with it, which is also a make-up lesson for the university.
We will continue to update netty-related content and finally produce a high-performance HTTP and RPC framework, so stay tuned.
The relevant code above:
Github.com/crossoverJi…
extra
Recently in the summary of some Java related knowledge points, interested friends can maintain together.
Address: github.com/crossoverJi…
Welcome to pay attention to the public account to communicate:
