Original author Jiang Chengjun, original title “Still being abused by Java NIO? It’s time to try Netty.

1. Who to read

This article is suitable for beginners to Java NIO network programming who know nothing about Netty. In order to do so, the content from the most basic introduction to the configuration of the development environment, to the preparation of the first Demo code, all with detailed illustrations.

So this article is very helpful for novice drivers, but for veteran drivers, it is not necessary. Old driver please make a detour.

**PS: ** Yes, writing IM and push messages in Java is basically using Netty, so if you want to use Java for instant messaging and other systems, learn Netty.

2. The author of this article

** Jiang Chengjun: ** Ministry of Industry and Information Technology information System project manager, full stack engineer, years of rich experience in JavaWEB platform, PC software, mobile APP development and training, good at making complex things simple.

3. Basic knowledge

Before we get to Netty, it’s important to take a brief look at the basics of the Java Network programming model, namely BIO, NIO, and AIO.

**BIO, NIO, and AIO correspond to three communication models: ** blocking, non-blocking, and non-blocking asynchronous, which I won’t describe in detail here. There are a lot of blogs on the web that say Netty corresponds to NIO. To be exact, it can be either NIO or AIO, depending on how you implement it.

The differences between these three concepts are as follows:

• 1) BIO: One thread is connected. When the client has a connection request, the server needs to start a thread to process it, which is costly.
• 2) NIO: one request for one thread, the connection request sent by the client will be registered with the multiplexer, and the multiplexer will only start a thread to process when the multiplexer polls the connection for I/O requests;
• 3) AIO: each thread is effectively requested. The CLIENT’s I/O request is first completed by the OS and then notified the server application to start the thread for processing.

A simple summary is:

• 1) BIO is flow-oriented, NIO is buffer-oriented;
• 2) The BIO streams are blocking, while NIO is non-blocking;
• 3) BIO’s Stream is one-way, while NIO’s channel is bidirectional.

**NIO’s salient features: ** Event-driven model, single-thread processing multi-task, non-blocking I/O, I/O read and write no longer block, but return 0, block-based transmission is more efficient than stream-based transmission, more advanced IO function zero-copy, IO multiplexing greatly improves the scalability and usability of Java network applications. Based on the Reactor thread model.

Due to limited space, there is no way to expand the topic in depth here. If you want to know more about it, you can continue to read these articles:

• 1) “Java BIO and NIO are difficult to understand?” ;
• 2) The difference between Java NIO and Classic IO in one minute;
• 3) the greatest Introduction to Java NIO ever: For those worried about getting started and giving up, read this! (* Recommended reading).

4. Get to know Netty

4.1 Basic Introduction

Netty is an open source asynchronous event-driven network programming framework based on Java NIO technology for rapid development of maintainable high-performance protocol servers and clients.

In layman’s terms, Netty can be understood as an awesome framework that encapsulates Java NIO and makes it much easier to use and get started.

**PS: **Netty’s official website is netty. IO /, you can download the latest Netty source code, as well as various API documentation and development guide.

4.2 Technical Features

The advantages of Netty can be summarized as follows:

• 1) Simple to use;
• 2) Powerful function;
• 3) Strong performance.

Netty features:

• 1) High concurrency: based on NIO (Nonblocking IO) development, compared with BIO (Blocking I/O), its concurrency performance has been greatly improved;
• 2) Fast transmission: transmission relies on the zero-copy feature to minimize unnecessary memory copy and achieve more efficient transmission;
• 3) Encapsulation: Encapsulates many details of NIO operations and provides an easy-to-use call interface.

Netty’s advantages:

• 1) Simple to use: it encapsulates many details of NIO, making it easier to use;
• 2) Powerful function: preset a variety of codec functions, support a variety of mainstream protocols;
• 3) Strong scalability: the communication framework can be flexibly extended through ChannelHandler;
• 4) Excellent performance: Compared with other mainstream NIO frameworks in the industry, Netty has the best overall performance;
• 5) Stable operation: Netty fixes all NIO bugs found so developers can focus on the business itself;
• 6) Active community: Netty is an active open source project with short iteration cycle and fast bug repair.

What is Netty’s high performance?

• 1) IO thread model: synchronous non-blocking, doing more with the fewest resources;
• 2) Zero memory copy: reduce unnecessary memory copy to achieve more efficient transmission;
• 3) Memory pool design: The applied memory can be reused, mainly refers to the direct memory. The internal implementation uses a binary search tree to manage memory allocation.
• 4) Serial processing of read and write: avoid performance overhead caused by locking;
• 5) High-performance serialization protocol: support protobuf and other high-performance serialization protocols.

There is no room for detailed Netty features. If you are interested, read Getting Started: The most thorough Analysis of Netty high-performance Principles and Frameworks so far.

5. Author of Netty

Personally, when learning about a technology, I prefer to ask about its author. It is not gossip, but my personal habit. I hope to know more and more about the technology used.

5.1 Founder of Netty

The founder of Netty is South Korean Trustin Lee. He was born in 1980. Since the age of 8, he has written BASIC programs on MSX minicombter.

While studying computer science at Yonsei University in South Korea, he wrote high-performance network applications and a small number of Web programs for several companies. After graduation, he worked for Arreo Communications, one of the largest mobile SMS providers in South Korea.

He is now working for Line in South Korea (according to his blog, he left line at the end of August 2020). Mina, which was widely used in the early days, is also his work.

▲ Trustin Lee himself

Other information about Trustin Lee:

• His personal blog: t.mottd.kr /
• His Github: github.com/trustin

5.2 Netty Leader

Netty’s current project leader is German Norman Maurer (formerly of Redhat, working full-time on Netty), the author of Netty in Action, and currently a senior engineer at apple.

▲ Norman Maurer himself

Additional information about Norman Maurer:

• His personal blog is Normanmaurer. Me /
• His Github: github.com/normanmaure…

Finally, the block diagram of the two great gods is attached:

6. What can Netty do?

Learning skills is all about being able to apply them to the actual work. No one is learning for the sake of learning and playing. So what can Netty do?

There are two main aspects.

** On the one hand: ** Now the Internet of Things applications are everywhere, a large number of projects are involved in the application of sensors and server-side data communication, Netty as a basic communication component, can easily solve the previously high threshold of communication system development, you no longer have to figure out how to parse all kinds of simple or complex communication protocols. Programmers who have experience in this area will have a deeper, or more memorable, experience.

On the other hand: ** Today’s Internet system is all about high concurrency, distributed, micro-service, all kinds of messages flying everywhere (yes, IM system, message push system is typical), Netty applications in this kind of architecture can be said to be like a duck in water, if you are not comfortable with the current various application servers, Then we can realize HTTP server, FTP server, UDP server, RPC server, WebSocket server, Redis Proxy server, MySQL Proxy server and so on based on Netty.

7. What are the benefits of mastering Netty?

The immediate benefit of ** is that ** has the opportunity to enter a large factory and earn a high salary. Many well-known companies in the industry generally require proficiency in, or familiarity with Netty when recruiting senior/senior Java engineers.

The list could go on, on…

As a Java student, if you haven’t studied Netty, then your use and understanding of the Java language is only superficial. You can do SSH, write a few MVC’s, access databases and caches, and that’s just what a beginner or intermediate Java programmer does. Netty is definitely a must pass if you want to get to the next level of Java server knowledge.

** Indirect benefits: ** Many open source frameworks use Netty, and if you master Netty, you will have the foundation to analyze these open source frameworks and become a technology leader.

What are some of these open source frameworks?

A brief list of some typical, as follows:

• 1) Alibaba distributed service framework Dubbo’s RPC framework;
• 2) RocketMQ, taobao’s messaging middleware;
• 3) Avro’s RPC framework for Hadoop’s high-performance communication and serialization component;
• 4) Open source Cluster computing framework Spark;
• 5) Distributed computing framework Storm;
• 6) Concurrent application and distributed application Akka;
• 7) The list is still very, very long…

8. Theoretical knowledge preparation

The second half of this article, hand wing hand, takes you through a simple example of implementing a transfer string.

Before you get started, it’s important to know the basic concepts necessary, otherwise the code will be knocked down, the functionality will be implemented, but you will still have no idea about Netty, which is not the purpose of this article.

This example needs to use the basic knowledge mainly has the following aspects of the east, these knowledge points had better have a general understanding, otherwise, see the example will have certain difficulties.

• 1) Basic Knowledge of Java;
• 2) Master the basics of Maven;
• 3) Familiar with IntelliJ IDEA integrated development tool, which is referred to as IDEA;
• 4) Know the basic concepts of TCP and Socket.

In particular, TCP, Socket concept, the following several must read:

• 1) Network Programming lazy Introduction (I) : Quick Understanding of Network Communication Protocols (PART I)
• 2) “Network Programming lazy Introduction (II) : A Quick Understanding of Network Communication protocols (Part II)”
• 3) Network programming lazy introduction (3) : a quick understanding of TCP protocol is enough
• 4) Introduction to Brain-dead Network Programming (II) : What are We Reading and writing when We read and write sockets?

Get an overview of Netty’s main components and concepts:

• 1) I/O: Various streams (files, arrays, buffers, pipes…) Processing of (input/output);
• 2) Channel: Channel, representing a connection. Each Client corresponds to a specific Channel.
• 3) ChannelPipeline: responsibility chain, each Channel has and only one Channel pipeline corresponding to it, which is a variety of handlers;
• 4) Handler: used to process inbound and outbound messages and corresponding events to achieve our own business logic;
• 5) EventLoopGroup: an I/O thread pool that processes I/O events corresponding to a Channel;
• 6) ServerBootstrap: Start auxiliary objects on the server side;
• 7) Bootstrap: the client starts the auxiliary object;
• 8) ChannelInitializer: Channel initializer
• 9) ChannelFuture: represents the execution result of I/O operation, through the event mechanism, get the execution result, by adding listeners, perform the operation we want;
• 10) ByteBuf: sequence of bytes that operate on the underlying byte array and buffer through ByteBuf.

For an in-depth understanding of these Netty concepts, it is recommended that you read: Getting Started: The Most thorough Overview of Netty High-performance Principles and Frameworks so far.

For Netty development, API documentation and source code are the most commonly used resources. Here are the online reading links I compiled:

• 1) Netty – 4.1 x source address: docs.52im.net/extend/docs… (Available online)
• 2) Netty – 4.1 x API documentation: docs.52im.net/extend/docs… (Available online)

9. Preparation of development environment

There are three aspects to prepare the development environment: JDK installation and environment variable Settings, Maven installation and environment variable Settings, IDEA installation and basic Settings.

Please follow me one by one to do the foolproof configuration and operation.

9.1 JDK Installation and Environment Variable Settings

JDK download, can be from the official now, also can be searched on Baidu download link, I download is JDK8, should pay attention to a point is, now from the JDK official website Oracle download need account, no account can not be down, do not know what to do.

The download address of the official website is www.oracle.com, and the screenshots are as follows:

Download finished, all the way Next installed, when creating a Java environment variable is set, [computer] this right – > [properties] – > [advanced system Settings – > [environment variable] – > [] system variables.

Screenshot below:

** After the Java environment variable is created, run the command: **Java -version in the DOS window to test whether it is normal

9.2 Maven Installation and Setting Environment Variables

Maven is powerful, but don’t worry. In this example, we just take advantage of the convenience of JAR dependencies and JAR dependency passing, with virtually no learning cost.

Jar dependencies, jar dependencies, jar dependencies, jar dependencies, jar dependencies, jar dependencies

Maven is downloadable, unzipped, and configured with environment variables.

Download address: * * * * downloads.apache.org/maven/maven…

** Installation: ** Download the compressed package, decompress it, and copy the folder to the desired location (such as the root directory of drive C).

Create MAVEN_HOME, point to the Maven folder, and add it to your path.

The diagram below:

Since it is slow to automatically download jar packages directly from Maven’s central repository, it is common to add ali Cloud’s public repository configuration to Maven’s configuration file, which will significantly speed up the download of JAR packages.

As shown below:

** After setting the environment variables above, run the command ** MVN -version in the DOS window to check whether the environment variables are set successfully, as follows:

9.3 IDEA Installation and Basic Settings

The download and installation of IDEA are not mentioned. The versions are divided into the flagship version and the community version. The flagship version is charged, and the community version is free.

But for this example, the community version is enough, but if you don’t care about the money, you can consider the flagship version, in case we need to do WEB system development later.

After installing it, specify the location of JDK and Maven in its configuration, as shown in the following figure.

Maven specified: [File] – > [setting] – > [Build, the Excution, Deployment] – > [Build Tools] – > [Maven]

JDK: [File]–>[Project Structure]–>[Project Setting]–>[Project]

9.4 Creating a Maven Project in IDEA

New Projects:

Fill in the package name and project name:

Maven configuration:

Build project to automatically create Maven dependency files:

Configure Netty dependencies in pom.xml:

After the above steps, our Maven project has been created, now we can write Netty’s first program, this program is very simple, transfer a string, although the program is very simple, but has been able to reflect the overall flow of Netty communication program development.

10. Start coding

10.1 Basic Routines for Netty Development

The basic formula for Netty development is simple, both server-side and client-side.

The general routine is as follows:

The actual code process developed by Netty is really not complicated, as shown in the figure below, with client-side processes represented in green and server-side processes represented in blue. Note the red ones.

The actual code process looks like this:

10.2 Creating a Client Class

First, we create a Handler class that receives data sent from the server. This is a simplified class that only overwrites the message reading method channelRead0 and the exceptionCaught method.

The client Handler is SimpleChannelInboundHandler inheritance, the class has a lot of methods, heartbeat, timeout detection, connection status, etc.

The code is as follows:

import io.netty.buffer.ByteBuf; import io.netty.channel.ChannelHandler; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; /** * @Date: 2020/6/1 11:12 * @Description: General handler, Deal with an I/O * / @ ChannelHandler Sharable public class HandlerClientHello extends SimpleChannelInboundHandler < ByteBuf > { @Override protected void channelRead0(ChannelHandlerContext channelHandlerContext, ByteBuf ByteBuf) throws Exception {/** * @description Processes the received message **/ system.out.println (" Received message: "+byteBuf.toString(CharsetUtil.UTF_8)); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throwsException {/** * @description **/ cause. PrintStackTrace (); ctx.close(); }}Copy the code

Code description:

• 1) @channelHandler. Sharable: This annotation is for thread safety, if you don’t care about thread safety, don’t add it;
• 2) SimpleChannelInboundHandler: this type can be ByteBuf, also can be a String, can also be objects, according to the actual situation;
• 3) channelRead0: message reading method, note that there is a 0 in the name;
• 4) ChannelHandlerContext: Channel context;
• 5) ByteBuf: byte sequence, ByteBuf operates the basic byte array and buffer, because JDK native operation byte troublesome, low efficiency, so Netty encapsulated the byte operation, achieved exponential performance improvement, while using more convenient;
• 6) Charsetutil.utF_8: This is a JDK native method that specifies the encoding format for converting byte arrays to strings.

The client startup class establishes a connection according to the IP address and port of the server. After the connection is established, the bidirectional transmission of messages is realized.

The code is simple as follows:

import com.sun.org.apache.bcel.internal.generic.ATHROW; import io.netty.*; import java.net.InetSocketAddress; /*** @date: 2020/6/1 11:24* @description: Client startup class */public class AppClientHello{private Final String host; private fina lint port; public AppClientHello(String host, int port){this.host = host; this.port = port; }public void run() throws Exception{/*** @description Sets parameters. **/EventLoopGroup group = newNioEventLoopGroup(); //I/O thread pool try{Bootstrap bs = newBootstrap(); Channel (niosocketChannel.class)// Instantiate a channel.remoteAddress (newInetSocketAddress(host,port)).h Andler (newChannelInitializer<SocketChannel>()// Initialize the channel {@overrideprotected void initChannel(SocketChannel) socketChannel) throws Exception{socketChannel.pipeline().addLast(newHandlerClientHello()); // add our custom Handler}}); // Connect to the remote node; ChannelFuture future=bs.connect().sync(); // Send messages to the server in utF-8future.channel ().writeAndFlush(unpooled.copiedBuffer ("Hello World", charsetutil.utf_8)); CloseFuture () opens a listener for a channel(during which time the channel is working) until the link disconnects future.channel().closeFuture().sync(); } finally{group.shutdownGracefully().sync(); }}public static void main(String[] args) throws Exception{new AppClientHello("127.0.0.1",18080).run(); }}Copy the code

Since the code has been extensively commented, only a few examples are described here:

• 1) ChannelInitializer: initializes channels, such as adding multiple handlers.
• 2) bs.connect().sync() : sync() here indicates the synchronization method adopted, so that after the connection is successfully established, the execution can continue;
• 3) pipeline () : After the connection is established, a pipeline will be automatically created, which is also known as the chain of responsibility, to ensure the sequential execution, but also can flexibly configure all kinds of handlers, this is a very subtle design, not only reduce the resource overhead caused by thread switching, avoid a lot of trouble, and at the same time greatly enhance performance.

10.3 Creating a Server Class

As with the client, only the message read method channelRead(note this is not channelRead0) and the exceptionCaught method are overridden.

The other server-side Handler is inherited ChannelInboundHandlerAdapter, rather than SimpleChannelInboundHandler, as for the difference, don’t go into here, everyone should baidu.

The code is as follows:

import io.netty.*; /*** @Date: 2020/6/1 11:47* @Description: The server I/O processing class. * / @ ChannelHandler Sharablepublic class HandlerServerHello extends ChannelInboundHandlerAdapter {@ Overridepublic Void channelRead(ChannelHandlerContext CTX, Object MSG) throws Exception{// Process the received data, ByteBuf in = (ByteBuf) MSG; System.out.println(" receive message from client: "+ in.toString(charsetutil.utf_8)); // Write and send information to remote (client) ctx.writeAndFlush(Unpooled. CopiedBuffer (" Hello, I am the server, I have received the message you sent ", charsetutil.utf_8)); }@Overridepublic void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception{printStackTrace(); Throwable cause) throws Exception{printStackTrace(); ctx.close(); }}Copy the code

The above code is very succinct, notice the comparison with the client Handler class.

The server side startup class is a bit more complicated than the client side startup class. First post the following code:

import io.netty.*; import java.net.InetSocketAddress; /*** @date: 2020/6/1 11:51* @description: Public class AppServerHello{private int port; public AppServerHello(int port){this.port = port; }public void run() throws Exception{EventLoopGroup group = newNioEventLoopGroup(); //Netty's Reactor thread pool initializes a NioEventLoop array for I/O operations such as accepting new connections and reading/writing data. Try {ServerBootstrap b = newServerBootstrap(); / / used to launch the NIO service b.g roup (group). The channel (NioServerSocketChannel. Class) LocalAddress (newInetSocketAddress(port))// Set the listener port. ChildHandler (newChannelInitializer<SocketChann El >() {//ChannelInitializer is a special handler class, Overridepublic void initChannel(SocketChannel CH) throws Exception {//ChannelInitializer is a special handler class that helps users configure a new Channel. ch.pipeline().addLast(new HandlerServerHello()); // Configure childHandler to notify an instance of InfoServerHandler about message handling}}); ChannelFuture ChannelFuture = b.bind().sync(); ChannelFuture = b.bind().sync(); System.out.println(" enable listening on "+ channelfuture.channel ().localAddress()+"); // Block, closeFuture() opens a listener for a channel(during which the channel is working) until the link disconnects. } finally{group.shutdownGracefully().sync(); // Close EventLoopGroup and release all resources, Public static void main(String[] args) throws Exception{new AppServerHello(18080).run(); }}Copy the code

Code description:

• 1) EventLoopGroup: In the actual project, two instances of EventLoopGroup are created. One is responsible for receiving client connections and the other is responsible for processing message I/O.
• 2) NioServerSocketChannel: instantiate a channel through the factory through the factory method design pattern, which is not needed to go into the case of Netty project development.

Now that we have written both the server and the client, how to Run it? Right-click on the server startup class and click Run ‘AppServerHello.main()’ to Run it.

Then, do the same thing, run the client startup class, and you’ll see the effect.

11. Write at the end

That concludes this article, which hopefully gives you an overview of Netty and its development process.

Netty features a lot of, this article is just an introduction, if you are interested in Netty development, you can pay attention to me and give me a message, I will according to the attention and message situation, continue to write Netty actual combat development of the article.

Getting affirmation and positive feedback gives you the desire and motivation to keep writing, because it takes a lot of energy to write such a detailed article. (This article is simultaneously published at: www.52im.net/thread-3207…)