@[TOC]
introduce
- Netty is a Java open source framework provided by JBOSS and is now a Github independent project.
- Netty is an asynchronous, event-driven network application framework for rapid development of high performance, high availability network IO programs.
- Netty applies to TCP applications with high concurrency on the Client or in peer-to-peer scenarios where large amounts of data are continuously transmitted.
- Netty is essentially a NIO framework applicable to various application scenarios related to server communication
Netty is a NIO client server framework that makes it fast and easy to develop network applications, such as protocol servers and clients. It greatly simplifies and simplifies network programming such as TCP and UDP socket servers.
- Higher throughput, lower latency
- Reduce resource consumption
- Reduce unnecessary memory replication
Application scenarios
- RPC framework is essential for remote service invocation between each node in distributed system, so Netty is used by RPC framework as a basic communication component as an asynchronous high-performance communication framework
- Dubbo, the distributed framework of Ali, uses Netty as the basic communication component by default for Dubbo protocol, which is used to realize internal communication between process nodes
I/O Model (BIO, NIO, AIO)
-
BIO: Synchronous blocking (traditional blocking). The server implements a connection one thread at a time. When a client requests a connection, the server must start a thread to process it.
-
NIO: Synchronous non-blocking, server implementation mode is a thread to handle multiple requests, that is, the client sent connection requests are registered with the multiplexer, multiplexer polling connection to have I/O requests to process.
-
AIO: asynchronous non-blocking, AIO introduces the concept of asynchronous channel, uses the Proactor mode, simplifies the programming, effective request to start the thread, its characteristics is completed by the operating system before informing the server and the program start thread to deal with, generally applicable to the number of connections and connection time is long applications
I/O usage scenarios:
- The BIO approach is suitable for architectures with a small and fixed number of connections. It requires high server resources, and concurrency is limited to the application. The only option prior to JDK1.4 is simple and easy to understand.
- NIO mode is suitable for the architecture with a large number of connections and relatively short connections (light operation), such as chat server, bullet screen system, communication between servers, which is supported after JDK1.4
- AIP mode is suitable for architectures with a large number of connections and long connections (reoperation), such as album server. It fully invokes the OS to participate in concurrent operations, which is supported after JDK1.7.
BIO
- The server starts a ServerSocket
- The client starts the socket to communicate with the server. By default, the server needs to create a thread for each client to communicate with it
- When a client sends a request, it first consults the server to see if there is a thread response, and if not, it waits or is rejected
- If there is a response, the client thread waits for the request to end and then resumes execution
Case study:
- Write a server using the BIO model that listens for ports and starts a thread to communicate with a client when it connects
import java.io.IOException;
import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
/ * * *@author yxl
* @version 1.0
* @date2021/3/9 prepare * /
public class BIOServer {
public static void main(String[] args) throws IOException {
Create a thread pool
ExecutorService executorService = Executors.newCachedThreadPool();
ServerSocket serverSocket = new ServerSocket(6666);
System.out.println("Server started");
while(true){
Socket socket = serverSocket.accept();
System.out.println("Connect to a client");
// Create a thread pool
executorService.execute(new Runnable() {
@Override
public void run(a) { handler(socket); }}); }}public static void handler(Socket socket){
try {
System.out.println("Current thread ID"+ Thread.currentThread().getId() + "Name" + Thread.currentThread().getName());
byte[] bytes = new byte[1024];
InputStream inputStream = socket.getInputStream();
while (true) {int read = inputStream.read();
if(read ! = -1){
System.out.println("-");
System.out.println(new String(bytes,0,read));
}else{
break; }}}catch (Exception e) {
e.printStackTrace();
} finally {
System.out.println("Close client connection"); }}}Copy the codeUsing Telnet tests
Disadvantages:
- Each request needs to create a separate thread, with the corresponding client for data Read, business processing, data Write.
- When the number of concurrent connections is large, a large number of threads need to be created to process connections, and system resources are occupied.
- After the connection is established, if the current thread has no data to Read temporarily, the thread blocks on the Read operation, resulting in a waste of resources.
NIO
- Java NIO stands for Java non-blocking IO, which means the JDK provides a new API. Starting with JDK1.4, Java provides a series of improvements when input/output new features, collectively called NIO, are asynchronous and non-blocking
- Nio-related classes are placed in the java.nio package and subpackages, and many classes in the original java.io package are overwritten.
- NIO has three core parts: channels, buffers, and selectors.
- NIO is cache-oriented, or fast programming pair, where data is read into a pair that it later processes, moving back and forth in the cache as needed, which increases flexibility in class processing. It can be used to provide non-blocking highly scalable networks
Buffer:
├─ByteBuffer ├─IntBuffer ├─LongBuffer ├─ShortBuffer ├─DoubleBuffer ├─CharBuffer └Copy the code- A Bufer, essentially a block of memory that can read data, can be thought of as a container object (including an array) that provides a set of methods that make it easier to use a block of memory. A Bufer object has built-in mechanisms for tracking and logging changes in the state of the cache. Channel Provides the file network read Channel, but the data read or written must pass through buffer.
public class BasicBuffer {
public static void main(String[] args) {
// Create an IntBuffer of size 5
IntBuffer intBuffer = IntBuffer.allocate(5);
intBuffer.put(1);
intBuffer.put(2);
intBuffer.put(3);
intBuffer.put(4);
intBuffer.put(45);
// Read/write conversion
intBuffer.flip();
while(intBuffer.hasRemaining()){ System.out.println(intBuffer.get()); }}}Copy the code
All boffers inherit and have several important parameters
public abstract class IntBuffer
extends Buffer
implements Comparable<IntBuffer>
Copy the codepublic abstract class Buffer {
/** * The characteristics of Spliterators that traverse and split elements * maintained in Buffers. */
static final int SPLITERATOR_CHARACTERISTICS =
Spliterator.SIZED | Spliterator.SUBSIZED | Spliterator.ORDERED;
// Invariants: mark <= position <= limit <= capacity
private int mark = -1; //
private int position = 0; // When you write data to Buffer, position indicates the current position and the initial value is O
private int limit;
private int capacity;
}
Copy the code-
Mark: mark
-
Capacity: The maximum amount of data that can be held, which is set when the cache is created and cannot be changed.
-
Position: The index of the next element to be read or written. The value is changed each time data is read or written to the cache in preparation for the next read or write.
-
Limit: indicates the current end point of the cache. The value cannot be read or written to the location where the cache exceeds the limit. The value can be changed when the limit is reached.
Breakpoint:
Second, the Channel:
- Java channels are similar to streams, but with some differences. A channel can read and write simultaneously, while a stream can only read and write. It can also read and write data asynchronously, and it can also read and write data from the cache
- A Channel is a NIO interface
- Common Channel classes include FileChannel, DatagramChannrl, and ServerSocketChannel
FileChannel :
public class BasicFileChannel {
public static void main(String[] args) throws IOException {
String str = "Hello World";
// Create an output Channel
FileOutputStream fileOutputStream = new FileOutputStream("/Users/yanxiaolong/a.txt");
/ / get the FileChannel
FileChannel channel = fileOutputStream.getChannel();
// Create a Buffer
ByteBuffer allocate = ByteBuffer.allocate(1034);
allocate.put(str.getBytes(StandardCharsets.UTF_8));
// Flip ByteBuffer
allocate.flip();
// Write ByteBuffer to FileChannel
channel.write(allocate);
/ / close the flowfileOutputStream.close(); }}Copy the code
public class BasicFileChannel2 {
public static void main(String[] args) throws IOException {
// Read the file to create an input stream
File file = new File("/Users/yanxiaolong/a.txt");
FileInputStream fileInputStream = new FileInputStream(file);
FileChannel channel = fileInputStream.getChannel();
// Create a Buffer
ByteBuffer allocate = ByteBuffer.allocate((int)file.length());
channel.read(allocate);
System.out.println(newString(allocate.array())); fileInputStream.close(); }}Copy the code
MappedByteBuffer file Copy
public class BasicFileChannel3 {
public static void main(String[] args) throws IOException {
long start = System.currentTimeMillis();
FileChannel inChannel = FileChannel.open(Paths.get("/Users/yanxiaolong/a.txt"), StandardOpenOption.READ);
FileChannel outChannel = FileChannel.open(Paths.get("/Users/yanxiaolong/b.txt"), StandardOpenOption.WRITE,StandardOpenOption.READ,StandardOpenOption.CREATE_NEW);
// Memory mapped file
MappedByteBuffer inMappedBuf = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
MappedByteBuffer outMappedBuf = outChannel.map(FileChannel.MapMode.READ_WRITE, 0, inChannel.size());
// Write data directly to the buffer
byte[] dst = new byte[inMappedBuf.limit()];
inMappedBuf.get(dst);
outMappedBuf.put(dst);
inChannel.close();
outChannel.close();
long end = System.currentTimeMillis();
System.out.println("Time spent on memory mapped files:"+(end-start)); }}Copy the code
Three, the Selector:
- Java NIO, which is non-blocking iO, can handle multiple client connections with one thread, and it uses a selector, which is a multiplexer of SelecttableChannle objects
- Selector can detect if something is happening on multiple registered channels (note that multiple channels can be registered to the same Selector as an event), and if something is happening, fetch the event and then handle each event accordingly. This makes it possible to manage multiple channels with a single threaded area, that is, multiple connections and requests.
- Read/write only occurs when a connection actually has a read/write event, which greatly reduces overhead and eliminates the need to create a thread for each connection and maintain multiple threads
- Avoid the overhead of context switching between multiple threads
The service side
public class NIOServer {
public static void main(String[] args) throws IOException {
/ / create a ServerSocketChannel
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// Create a selector
Selector selector = Selector.open();
serverSocketChannel.bind(new InetSocketAddress(6666));
serverSocketChannel.configureBlocking(false);
serverSocketChannel.register(selector,SelectionKey.OP_ACCEPT);
// Loop waiting for the client to connect
while (true) {if(selector.select(1000) = =0){
System.out.println("Server waited one second without connection");
continue;
}
// If greater than 0, the associated selectedKeys connection has been obtained
//selector.selectedKeys(); Returns a collection of events
Set<SelectionKey> selectionKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()){
// Get the next element
SelectionKey next = iterator.next();
// If it is OP_ACCEPT, there is a new client connection
if(next.isAcceptable()){
The client generates a SocketChannel
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
System.out.println("Client connection successfully generated a socketChannel" + socketChannel.hashCode());
// Register and associate ByteBuffer
socketChannel.register(selector,SelectionKey.OP_READ, ByteBuffer.allocate(1024));
}
/ / send OP_READ
if(next.isReadable()){
SocketChannel socketChannel = (SocketChannel) next.channel();
ByteBuffer byteBuffer = (ByteBuffer) next.attachment();
socketChannel.read(byteBuffer);
System.out.println("From client:" + newString(byteBuffer.array())); } iterator.remove(); }}}}Copy the codeThe client
public class NIOClient {
public static void main(String[] args) throws IOException {
// Get the network channel
SocketChannel socketChannel = SocketChannel.open();
// Set non-blocking
socketChannel.configureBlocking(false);
// Socket connection address
InetSocketAddress inetSocketAddress = new InetSocketAddress("127.0.0.1".6666);
// Connect to the server
if(! socketChannel.connect(inetSocketAddress)){while(! socketChannel.finishConnect()){ System.out.println("Because the connection takes time, the client doesn't block and can do other work."); }}// Send data when the connection is successful
String str = "hello world";
ByteBuffer byteBuffer = ByteBuffer.wrap(str.getBytes(StandardCharsets.UTF_8));
// Write Buffer data to ChannelsocketChannel.write(byteBuffer); System.in.read(); }}Copy the code
Principle:
- When a client connects, it gets a SocketChannel through ServerSocketChannel
- Selector continues to listen, and the select method returns the number of channels on which events were sent
- Register socketchannels with a Selector register (Selector SEL,int OPS). A Selector can register multiple Socketchannels
- After registration, it returns a SelectionKey associated with the Selector
- The SocketChannel is obtained through SelectionKey. The method Channel() is used to write the service code
-
SelectionKey: represents the registration relationship between a SelectableChannel and a Selector. Each time a channel is registered with the Selector, an event (the SelectionKey) is selected. The selection key contains two sets of operations represented as integer values. Each bit of the action set represents a class of optional operations supported by the channel lock for that key
- SelectionKey.OP_READ
- SelectionKey.OP_WRITE
- SelectionKey.OP_CONNECT
- SelectionKey.OP_ACCEPT
NIO and zero copy
Introduction:
- Zero copy is the key to network programming and many performance optimizations are required
- In Java programs, zero copies are commonly used such as MMAP (memory map) and sendFile.
Traditional I/O:
- Traditional IO copy technology requires four copies (CPU copy and DMA copy) and four state transitions (user state and kernel state), resulting in low efficiency
Mmap optimization:
- Mmap uses memory mapping to match file maps to the kernel cache, while users see data that can be shared with the kernel. In this way, the kernel space to user software to copy times can be reduced during network transmission
SendFile:
- The sendFile function is provided in Linux version 2.1. The basic mechanism is as follows: data goes directly from the kernel Buffer to the Socket Buffer without passing through the user state at all. At the same time, it has nothing to do with the user state, thus eliminating a context switch.
Difference between mmap and sendFile
- Mmap is suitable for reading and writing small amounts of data, while sendFile is suitable for transferring large files
- Mmap requires 4 context switches and 3 data copies, while sendFile requires 3 context switches and at least 2 data copies
- SendFile can use DMA to reduce CPU copying, mmap (must be copied from kernel to Socket cache)
Case study:
public class NewIoServer {
public static void main(String[] args) throws Exception {
InetSocketAddress address = new InetSocketAddress(7001);
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
ServerSocket serverSocket = serverSocketChannel.socket();
serverSocket.bind(address);
/ / create a buffer
ByteBuffer byteBuffer = ByteBuffer.allocate(4096);
while (true) {
SocketChannel socketChannel = serverSocketChannel.accept();
int readcount = 0;
while (-1! = readcount) {try {
readcount = socketChannel.read(byteBuffer);
} catch (Exception ex) {
break;
}
// rewind position = 0 mark voidbyteBuffer.rewind(); }}}}Copy the codepublic class NewIoClient {
public static void main(String[] args) throws Exception {
SocketChannel socketChannel = SocketChannel.open();
socketChannel.connect(new InetSocketAddress("localhost".7001));
String filename = "Protoc 3.6.1 track - win32.zip";
// Get a file channel
FileChannel fileChannel = new FileInputStream(filename).getChannel();
// Ready to send
long startTime = System.currentTimeMillis();
// Under Linux a transferTo method can do the transfer
// Under Windows, a call to transferTo can only send 8m, so the file needs to be transferred in segments, and mainly
// The position of the transmission ="
//transferTo The underlying use to zero copy
long transferCount = fileChannel.transferTo(0, fileChannel.size(), socketChannel);
System.out.println("Total bytes sent =" + transferCount + "Take." + (System.currentTimeMillis() - startTime));
/ / closefileChannel.close(); }}Copy the codeAIO
- Asynchronous I/O Asynchronous I/O Asynchronous I/O Asynchronous I/O Asynchronous I/O Asynchronous I/O Asynchronous I/O
- AIO, or NIO mode 2.0, is called asynchronous non-blocking IO. AIO introduces the concept of asynchronous channel and adopts the Proactor mode to simplify the programming. Only valid requests can start the thread. It is characterized by notifting the server program to start to deal with the thread after the operating system completes it
- AIO is not yet widespread
NIO vs BIO
- BIO processes data as a stream, while NIO processes data as a block, and block I/O is much more efficient than stream I/O
- BIO is blocking, NIO is non-blocking
- BIO operates based on byte streams and character streams, while NIO operates based on channels and buffers. Data is always read from a Channel to a cache, or written from a cache to a Channel. A Selector listens for multiple Channel events (e.g. Connection requests, data arrivals, etc.), so multiple client channels can be listened on using a single thread
Netty threading model
- The original NIO exists: tedious API, need to master the three components, difficult development and maintenance, need to consider disconnection reconnection, network intermittent disconnection, half packet read and write, network congestion, etc
Basic introduction to thread model:
-
Different threading models have a significant impact on program performance
-
There are traditional blocking I/O service model and Reactor model
-
Depending on the number of reactors and the number of threads that process the resource pool, there are three typical implementations — single Reactor single thread, single Reactor multi-thread, and principal/slave Reactor multi-thread
-
Netty threading model (Netty is based on the master/slave Reactor multithreading model, which has multiple reactors)
Reactor mode:
- Based on the I/O multiplexing model, multiple connections share one blocking object, and applications only need to wait on one blocking object instead of waiting on all connections. When a connection has new data to process, the operating system informs the application, and the thread returns from the blocked state to start business processing
- Based on threads, thread resources are reused. Instead of creating threads for each connection, the business processing tasks after the connection is completed are assigned to threads for processing. One thread can process multiple services
Single-threaded model:
- Connection events are monitored internally by a Selector and dispatched via Dispatch. If a connection is an established event, it is processed by an Acceptor. An Acceptor accepts a connection and creates a Handler for subsequent connection events. If it is a read/write event, call the connection Handler directly to handle it
- Handler completes the read -> (decode -> compute -> encode) -> send business process
- The advantages of this model are simple, but the disadvantages are obvious. When a Handler blocks, it will cause the Handler and Accpetor of other clients to fail to execute, which cannot achieve high performance. It is only suitable for scenarios where business processing is very fast
The single-thread model is to specify only one thread to perform client connection and read/write operations, which is completed in a Reactor. The corresponding implementation in Netty is to set the number of NioEventLoopGroup threads to 1. The core code is as follows:
NioEventLoopGroup group = new NioEventLoopGroup(1);
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(group)
.channel(NioServerSocketChannel.class)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_BACKLOG, 1024)
.childHandler(new ServerHandlerInitializer());
Copy the codeMultithreaded model:
- In the main thread, the Reactor object monitors connection events using a Selector, dispatches the event through Dispatch, and if it is a connection establishment event, it is processed by an Acceptor, which accepts the connection and creates a Handler to handle subsequent events. The Handler is only responsible for responding to events, but does not carry out service operations, that is, only reads and writes data. The service processing is handed over to a thread pool for processing
- The thread pool allocates a thread to complete the actual service processing, and sends the response result to the main thread Handler, which sends the result to the client
Multithreading model is to process client connection in a single Reactor, and then transfer the business process to the thread pool. The core code is as follows:
NioEventLoopGroup eventGroup = new NioEventLoopGroup();
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(eventGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_BACKLOG, 1024)
.childHandler(new ServerHandlerInitializer());
Copy the codeMaster-slave multithreaded model
- There are multiple reactors, each with its own selector selector, thread, and dispatch
- The mainReactor in the main thread monitors connection establishment events through its selector, receives the event through Accpetor, and assigns a new connection to a child thread
- The subReactor in the child thread adds the connection assigned by the mainReactor to the connection queue to listen through its selector and creates a Handler to handle subsequent events
- Handler Completes the complete business process of read-> Service processing -> Send
The master-slave multithreading model has multiple reactors, that is, there are multiple selectors, so we define a bossGroup and a workGroup, and the core code is as follows:
NioEventLoopGroup bossGroup = new NioEventLoopGroup();
NioEventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_BACKLOG, 1024)
.childHandler(new ServerHandlerInitializer());
Copy the codeReactor model advantages:
- Fast response and does not have to be blocked by a single synchronization time, although the Reactor itself is still synchronous
- Complex multithreading and synchronization issues are minimized, and multithreading/process switching overhead is avoided
- It is easy to make full use of CPU resources by adding Reactor instances
- The Reactor model itself is independent of collective event processing logic and has high reusability
Personal blog address: blog.yanxiaolu.cn /