Small knowledge, big challenge! This article is part of the “Programmer’s Essentials
This article also participated in the “Digitalstar Project” to win a creative gift package and creative incentive money
Netty’s name I want to do Java basic know, because he is really too famous, now a lot of famous software are using Netty as the basis of communication, today we will talk about Netty, hope to explain clearly, if you are too lazy to see the theory, you can directly pull back to see Hello World. Copy the code and run it.
1. What is Netty
Netty is a high-performance, asynchronous event-driven NIO framework based on API implementations provided by JAVA NIO. It provides support for TCP, UDP, and file transfers
As an asynchronous NIO framework, all I/O operations of Netty are asynchronous and non-blocking. Through the Future-Listener mechanism, users can obtain the RESULTS of I/O operations proactively or through notification mechanism.
As the most popular NIO framework, Netty has been widely used in the field of Internet, big data distributed computing, game industry, communication industry, etc. Some well-known open source components in the industry are also built based on Netty NIO framework.
Netty: netty. IO /
2. The advantages of Netty
The disadvantages of Netty aside, Netty has many advantages:
Unified API that supports multiple transport types, blocking and non-blocking.
Powerful, built in a variety of decoding encoders, support for a variety of protocols, such as the yellow area on the right, universal text, binary protocol, Google Protobuf, etc.
High performance. Netty has the best performance compared to other mainstream NIO frameworks.
Active community, found bugs will be timely repair, iteration cycle is short, constantly add new features.
Simple and powerful threading model.
Built-in codec to solve TCP sticky/unpack problem.
It comes with various protocol stacks, such as SSL.
Zero-copy Byte Buffers have higher throughput, lower latency, lower resource consumption, and less memory replication than using Java core apis directly.
Good security, full SSL/TLS and StartTLS support.
Mature and stable, it has been used in large projects and many open source projects use Netty, such as Dubbo, RocketMQ,Elasticsearch, etc.
3. Core components
3.1 Netty thread model
The thread model of Netty is very important. Only when you understand the thread model of Netty, can you use Netty well. There are three common thread models of Netty:
1. Single-threaded model
The single-thread model, in which all I/O operations are performed on the same NIO thread, is not used in game development, is not reasonable, does not expand NIO thread responsibilities include: receive new connection requests, read and write operations.
2.Reactor multithreaded model
The first is not reasonable, and to upgrade, a thread receiving a connection, all the I/O operations are done in the same NIO thread pool. This thread model works for most cases, but if you need to do some validation while connecting, it will block the thread. Performance is going to be a problem, server
3.Reactor master-slave multithreading model
Instead of a single NIO thread for the server to receive client connections, there is a separate NIO thread pool. After an Acceptor receives a TCP connection request from a client and processes it (including access authentication), it registers the newly created SocketChannel to an I/O thread in the SUB REACTOR. It is responsible for the read, write, codec and decoding of socketchannels. The Acceptor thread pool is used only for client login, handshake, and security authentication. Once a link is established, it is registered with the I/O thread in the back-end subReactor thread pool, which is responsible for subsequent I/O operations.
This is one of the most commonly used threading models in game development and needs to be mastered, as illustrated in the chart below
3.2 EventLoopGroup
The NioEventLoopGroup core is essentially a thread pool that exists to process I/O events.
An EventLoopGroup contains one or more Eventloops; An EventLoop is bound to only one Thread during its lifetime; All I/O events handled by EnventLoop will be handled on its proprietary Thread; A Channel registers with only one EventLoop during its lifetime; Each EventLoop handles one or more channels;
When we implement the server, we typically initialize two thread groups:
BossGroup: receives connections.
WorkerGroup: handles specific processing and submits it to the corresponding Handler
BossEventLoop only handles connections with very little overhead. As soon as the connection arrives, the SocketChannel is forwarded to the WorkerEventLoopGroup, The WorkerEventLoopGroup selects one of the Eventloops by Next to register the SocketChannel with the Selector it maintains and process its subsequent I/O events.
Note: The default number of threads is the current number of cpus *2
public abstract class MultithreadEventLoopGroup extends MultithreadEventExecutorGroup implements EventLoopGroup {
private static final InternalLogger logger = InternalLoggerFactory.getInstance(MultithreadEventLoopGroup.class);
private static final int DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt("io.netty.eventLoopThreads", NettyRuntime.availableProcessors() * 2));
protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
}
Copy the code3.3 the Channel
A Channel indicates a connection established with a client, which is equivalent to a telephone connection. A Channel is a bidirectional Channel.
A Channel is bidirectional and can be read and written. Buffer is a top-level interface in Java NIO, and its common subclasses are:
FileChannel: Used to read and write files
DatagramChannel: used to send and receive UDP packets
ServerSocketChannel: Used to send and receive TCP packets on the server
SocketChannel: used to send and receive TCP packets on the client
Common channel types in games include the following:
NioSocketChannel: asynchronous non-blocking client TCP Socket connection.
NioServerSocketChannel: asynchronous, non-blocking, server-side TCP Socket connection.
These two channel types are commonly used because they are asynchronous and non-blocking. So it’s the first choice.
3.4 option () and childOption ()
First of all, the difference between the two.
Option () sets the connection that the server uses to receive it, namely the boosGroup thread.
ChildOption () is the connection that the parent pipe receives, that is, the workerGroup thread.
With that in mind, let’s take a look at some common Settings:
The SocketChannel argument, which is the usual argument for childOption() :
SO_RCVBUF Socket specifies the size of the TCP data receiving buffer. TCP_NODELAY TCP parameter: sends data immediately. The default value is true. The SO_KEEPALIVE Socket parameter is used to keep the connection alive. The default value is False. When this function is enabled, TCP proactively detects the validity of idle connections.
ServerSocketChannel option()
The SO_BACKLOG Socket parameter indicates the length of the queue for the server to accept connections. If the queue is full, the client connection will be rejected. The default value is 200 for Windows and 128 for others.
3.5 the inbound and outbound
Inbound indicates the path where the message enters the server, which can be regarded as the input
OutBound indicates the path where messages are sent to the client, which can be regarded as output
ChannelPipeline p = ... ; p.addLast("1".new InboundHandlerA());
p.addLast("2".new InboundHandlerB());
p.addLast("3".new OutboundHandlerA());
p.addLast("4".new OutboundHandlerB());
p.addLast("5".new InboundOutboundHandlerX());
Copy the codeWhen an input event comes in, the event handler is called in the order 1,2,5
When an output event comes in, the event handler processes it in sequence 5,4, and 3. (Note that the output event handlers take effect in the opposite order as defined.)
This can be interpreted as pushing the handler.
ChannelInboundHandlerAdapter processor common events are:
Register event fireChannelRegistered.
Connection establishment event fireChannelActive.
Read events and read completion events fireChannelRead and fireChannelReadComplete.
Abnormal notification event fireExceptionCaught.
User-defined event fireUserEventTriggered.
The Channel can be written fireChannelWritabilityChanged state change events.
Connection closing event fireChannelInactive.
ChannelOutboundHandler Common events of the processor are:
Bind the port.
Connect to server Connect.
Write event Write.
Flush time Flush.
Read event read.
Disconnect actively.
Close the channel event close.
A similar handler() is used to assemble parent channels, also known as bossGroup threads. In most cases, you don’t use this method.
3.6 ByteBuf
There are three types of ByteBuff:
Heap memory buffer (HeapByteBuf)
The data is stored in the heap, which you can think of as a common memory buffer
Direct Memory buffer
The data is stored in the kernel. Since the data itself is stored in the kernel, data can be transferred directly when using the network card, without the need for redundant copies. Therefore, this is also called zero copy.
Read data from the hard disk and send it to the network adapter. The general procedure is as follows:
Data is read from disk into the kernel’s Read Buffer data is copied from the kernel buffer to the user buffer Data is copied from the user buffer to the kernel’s socket buffer
Copying data from the kernel socket buffer to the nic interface buffer using the memory buffer only takes two steps:
By calling transferTo, the data is copied from the file by the DMA engine to the kernel read Buffer, which then DMA copies the data from the kernel read Buffer to the nic interface buffer
CompositeByteBuf
A composite buffer can combine multiple Bytebuffs
Note: that is, kernel function modules run in kernel space while applications run in user space
ByteBuf has two Pointers to read index and write writerIndex, which are used to mark “readable”, “writable”, and “discarded” bytes
When the write* method is called to write data, the write pointer is moved backwards
When the read* method is called to read the data, the read pointer is moved backwards
When writing or reading data, it checks whether there is enough space to write and whether there is data to read
Before writing data, the system checks the capacity. If the remaining write capacity is less than the required write capacity, expand the capacity
During capacity expansion, there is a threshold of 4MB. The capacity to be expanded is smaller than the threshold or larger than the threshold
The clear method only changes the value of the read/write pointer and does not affect the existing content in ByteBuf
Methods such as setZero to modify the byte value only modify the corresponding byte value, without affecting the read/write pointer value and byte read/write status
Netty also provides two utility classes, Pooled and Unpooled, to allocate Pooled and Unpooled bytebuFs, further simplifying the process of creating bytebuFs by calling static methods of these two utility classes.
3.7 Using the Netty Decoder
LineBasedFrameDecoder :
When sending packets, each packet is separated by a newline character. The LineBasedFrameDecoder works by iterating through the readable bytes of ByteBuf to determine if there is a newline character and intercepting it accordingly.
DelimiterBasedFrameDecoder :
Can custom delimiter decoder, LineBasedFrameDecoder is actually a kind of special DelimiterBasedFrameDecoder decoder.
FixedLengthFrameDecoder:
Fixed length decoder, which can unpack the message according to the specified length, each packet length is fixed.
LengthFieldBasedFrameDecoder:
This is the decoder that will be used by future servers. There will be an example next time
Netty version 3.8
The use of ForkJoinPool in Netty5 increased code complexity, but the performance improvement was not noticeable
Synchronizing code across multiple branches is a lot of work
The authors don’t think it’s time to release a new version
There are more issues to investigate before releasing the version, such as whether to scrap exceptionCaught, expose EventExecutorChooser, and so on.
The latest version is 4.1.68.final
4, Hello World
4.1 Official Demo
Download the official demo source code to see all the demos under Example,
GitHub address: github.com/netty/netty…
4.2 IDEA Establishes the Maven project
I used IDEA here, so the following screenshots also use IDEA.
4.2.1 File ->New The following page is displayed
Fill in your own information as shown below
4.2.3 Wait until the Maven loading project is complete, the structure is as follows
4.3 Server code
To show as much Netty code as possible, cut out the bells and whistles and just plain programming
package com.xiangcai;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioServerSocketChannel;
/** * server code *@authorParsley * /
public class GameServer {
/** * start */
public static void start(a) throws InterruptedException {
EventLoopGroup boss = new NioEventLoopGroup(1);
EventLoopGroup worker = new NioEventLoopGroup();
try {
ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap.group(boss, worker)
.channel(NioServerSocketChannel.class)
// The backlog parameter in TCP/IP LISTEN
.option(ChannelOption.SO_BACKLOG, 1024)
If there is no data communication within two hours,TCP automatically sends an active probe data packet
.childOption(ChannelOption.SO_KEEPALIVE, true)
// Small packets are packaged into larger frames for transmission to increase the network load, i.e., TCP latency
.childOption(ChannelOption.TCP_NODELAY, true)
.childHandler(new NettyServerHandlerInitializer());
ChannelFuture channelFuture = serverBootstrap.bind(8088).sync();
System.out.println("Server started");
channelFuture.channel().closeFuture().sync();
} finally {
// Close the threadboss.shutdownGracefully(); worker.shutdownGracefully(); }}public static void main(String[] args) throws InterruptedException { start(); }}Copy the codeLet’s look at the Channel initialization code:
Two decoders were used
package com.xiangcai;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
/** * server code *@authorParsley * /
public class NettyServerHandlerInitializer extends ChannelInitializer<Channel> {
@Override
protected void initChannel(Channel ch) throws Exception {
ch.pipeline()
// A newline based decoder
.addLast(new LineBasedFrameDecoder(1024))
// Strong the string
.addLast(new StringDecoder())
// Business processing
.addLast(newNettyServerHandler()); }}Copy the codeHere is the code for the business:
This shows a simple sending and receiving message
package com.xiangcai;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.handler.timeout.IdleState;
import io.netty.handler.timeout.IdleStateEvent;
/** * server code *@authorParsley * /
public class NettyServerHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
}
/** * Timeout processing If the client heartbeat is not received within 5 seconds, it is triggered; If more than two times, it is directly closed; * /
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object obj) throws Exception {
if (obj instanceof IdleStateEvent) {
IdleStateEvent event = (IdleStateEvent) obj;
if (IdleState.READER_IDLE.equals(event.state())) { // If the read channel is idle, no heartbeat command has been received
System.out.println("I have not received any message from the client for 5 seconds."); }}else {
super.userEventTriggered(ctx, obj); }}/** * business logic processing */
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println("The server receives the message:" + msg);
String respStr = "收到了 " + System.getProperty("line.separator");
ByteBuf resp = Unpooled.copiedBuffer(respStr.getBytes());
ctx.writeAndFlush(resp);
}
/** * exception handling */
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { cause.printStackTrace(); ctx.close(); }}Copy the code4.4 Client Code
Client startup code
package com.xiangcai;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import org.junit.Test;
/** * server code *@authorParsley * /
public class TestClient {
public static void clientStart(a) {
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap b = new Bootstrap();
b.group(group)
.channel(NioSocketChannel.class)
.option(ChannelOption.TCP_NODELAY, true)
.handler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new LineBasedFrameDecoder(1024));
ch.pipeline().addLast(new StringDecoder());
ch.pipeline().addLast(new ClientHandler());
};
});
// Initiate an asynchronous connection
ChannelFuture f = b.connect("127.0.0.1".8088).sync();
// Wait for the client connection to close
f.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
// Exit gracefully, freeing the NIO thread groupgroup.shutdownGracefully(); }}public static void main(String[] args) { clientStart(); }}Copy the codeClient business processing:
It is also simple to send and receive messages
package com.xiangcai;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import java.nio.charset.StandardCharsets;
/** * server code *@authorParsley * /
public class ClientHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) {
System.out.println("Establish a connection");
byte[] bytes = ("We're connected. Start talking." + System.getProperty("line.separator")).getBytes(StandardCharsets.UTF_8);
ByteBuf message = Unpooled.buffer(bytes.length);
message.writeBytes(bytes);
ctx.writeAndFlush(message);
}
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
String body = (String) msg;
System.out.println("Received message" + body);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
System.out.println("Abnormal"); ctx.close(); }}Copy the code5, summary
Now more and more game companies use Java for development, Netty is the foundation of the network, it is very important to understand Netty, Netty is also very simple, as long as remember the thread model, codec, the other are details, in the process of development to learn, I hope this article can help you understand. If you have any questions, you can leave a message to me and learn to communicate with me.
Complete the project source code download address: download.csdn.net/download/pe…
The last picture ends
Writing very tired, I hope the big guys can point a praise, point a look.