I/O INPUT OUTPUT, including file I/O and network I/O.
Speed disdain in the computer world:
- Memory read data: nanosecond level.
- Gigabit card read data: subtle level. 1 microsecond =1000 nanoseconds. The nic is a thousand times slower than the memory.
- Disk read data: millisecond level. One millisecond = 100,000 nanoseconds, and the hard drive is 100,000 times slower than the memory.
- CPU a clock cycle of 1 nanosecond or so, memory is relatively close to CPU, other can not wait.
The CPU can process data much faster than the I/O can prepare data.
Any programming language will encounter this mismatch between CPU processing speed and I/O speed!
How to optimize network I/O in network programming: how to efficiently use CPU for network data processing??
I. Related concepts
How to understand network I/O from the operating system level? The computer world has its own set of definitions. If you don’t understand these concepts, you can’t really understand the design and nature of technology. So in my opinion, these concepts are fundamental to understanding technology and the computer world.
1.1 Synchronous and asynchronous, blocking and non-blocking
Understand the unavoidable topics of network I/O: synchronous vs. asynchronous, blocking vs. non-blocking.
Take sanji for example (sanji acts like a user program and sanji acts like a system call provided by the kernel). These two sets of concepts can be translated into plain English.
- Synchronous/asynchronous is concerned with whether I need to handle the water when it comes to the boil.
- Blocking/non-blocking concerns whether something else has been done while the water is boiling.
1.1.1 Synchronization Blocking
After ignition, silly, etc., not to open the water resolutely do not do anything (blocking), water turned off the fire (synchronization).
1.1.2 Non-blocking Synchronization
After the ignition, go to watch TV (non-blocking), from time to time to see if the water is on, after the water is on, turn off the heat (synchronous).
1.1.3 Asynchronous Blocking
After pressing the switch, wait for the water to turn on (blocked), the water will automatically turn off (asynchronous).
A model that does not exist in network programming.
1.1.4 Asynchronous non-blocking
After pressing the switch, do what you need to do (non-blocking), automatic power off after water (asynchronous).
1.2 Kernel space and user space
- The kernel is responsible for reading and writing network and file data.
- User programs obtain network and file data through system calls.
1.2.1 Kernel Mode User mode
- The program has to make system calls to read and write data.
- Through the system call interface, threads switch from user mode to kernel mode, the kernel reads and writes data, and switches back again.
- Different spatial states of processes or threads.
1.2.2 Thread Switching
Switching between user mode and kernel mode takes time and costs resources (memory and CPU)
Optimization suggestions:
- Less switching.
- Shared space.
1.3 Socket – Socket
- With sockets, network programming is possible.
- The application establishes a connection, receives and sends data (I/O) through system calls to socket().
- SOCKET supports non-blocking, the application can be non-blocking call, support asynchronous, the application can be asynchronous call
1.4 File descriptor – FD handle
Network programming needs to know FD?? What is FD??
Linux: Everything is a file, FD is a reference to a file. Is everything like an object in JAVA? The program operates on references to objects. The number of objects created in JAVA is limited by memory, as is the number of FD’s.
Linux needs to turn FD on and off when working with files and network connections.
Each process will have a default FD:
- 0 The standard input is stdin
- 1 Standard output stdout
- 2 Error stderr output
1.5 Process of the server processing network requests
- After the connection is established.
- Wait for data to be ready (CPU idle).
- Copy data from kernel to process (CPU idle).
How do you optimize it?
For an I/O access (such as read), data is copied to the operating system kernel’s buffer before being copied from the operating system kernel’s buffer to the application’s address space.
So, when a read operation occurs, it goes through two phases:
- Waiting for the data to be ready.
- Copying the data from the kernel to the process.
Because of these two phases, the following three network-mode solutions emerged in the Linux upgrade iteration.
IO model introduction
2.1 Blocking I/ O-Blocking I/O
Introduction: The original network I/O model. The process blocks until the data copy is complete.
Disadvantages: High concurrency, peer connection between server and client, problems caused by multiple threads:
- CPU resource waste and context switching.
- Memory costs go up exponentially, with the JVM costing about 1MB per thread.
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket();
ss.bind(new InetSocketAddress(Constant.HOST, Constant.PORT));
int idx =0;
while (true) {
final Socket socket = ss.accept();// block method
new Thread(() -> {
handle(socket);
},"[" thread+idx+"]").start(); }}static void handle(Socket socket) {
byte[] bytes = new byte[1024];
try {
String serverMsg = "Server SSS [thread:"+ Thread.currentThread().getName() +"]";
socket.getOutputStream().write(serverMsg.getBytes());// block method
socket.getOutputStream().flush();
} catch(Exception e) { e.printStackTrace(); }}Copy the code2.2 Non-blocking I/O – Non-blocking IO
Summary: A process makes repeated system calls and returns results immediately.
Disadvantages: When the process has 1000FDS, on behalf of the user process polling system to call the kernel 1000 times, back and forth switching between the user state and the kernel state, the cost increases geometrically.
public static void main(String[] args) throws IOException {
ServerSocketChannel ss = ServerSocketChannel.open();
ss.bind(new InetSocketAddress(Constant.HOST, Constant.PORT));
System.out.println(" NIO server started ... ");
ss.configureBlocking(false);
int idx =0;
while (true) {
final SocketChannel socket = ss.accept();// block method
new Thread(() -> {
handle(socket);
},"[" thread+idx+"]").start(); }}static void handle(SocketChannel socket) {
try {
socket.configureBlocking(false);
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
socket.read(byteBuffer);
byteBuffer.flip();
System.out.println("Request:" + new String(byteBuffer.array()));
String resp = "Server Response";
byteBuffer.get(resp.getBytes());
socket.write(byteBuffer);
} catch(IOException e) { e.printStackTrace(); }}Copy the code2.3 I/O multiplexing – IO multiplexing
Summary: A single thread can handle multiple network connections simultaneously. The kernel polls all sockets and notifies user processes when data arrives on a socket. In the process of Linux kernel code iteration, multiplexing supports three kinds of call successively, namely, SELECT, POLL, EPOLL three multiplexing network I/O model. The following will be illustrated with Java code.
2.3.1 I/O multiplexing – select
Description: A connection request arrived and rechecked processing.
Disadvantages:
- Handle upper limit – There is a limit of 1024 FD open by default.
- Repeat initialization – each time select() is called, the fd collection needs to be copied from user state to kernel state, which is iterated by the kernel.
- It is not efficient to go through all FD states one by one.
The server select is like a block of sockets. The client connection connects to one of the sockets, creating a channel, and then registers read and write events on the channel. A ready, read, or write event must be deleted before it can be processed again.
public static void main(String[] args) throws IOException {
ServerSocketChannel ssc = ServerSocketChannel.open();// Pipe ServerSocket
ssc.socket().bind(new InetSocketAddress(Constant.HOST, Constant.PORT));
ssc.configureBlocking(false);// Set non-blocking
System.out.println(" NIO single server started, listening on :" + ssc.getLocalAddress());
Selector selector = Selector.open();
ssc.register(selector, SelectionKey.OP_ACCEPT);// On the established pipeline, the registered event is ready
while(true) {
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> it = keys.iterator();
while(it.hasNext()) {
SelectionKey key = it.next();
it.remove();// Handle events that must be deletedhandle(key); }}}private static void handle(SelectionKey key) throws IOException {
if(key.isAcceptable()) {
ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
SocketChannel sc = ssc.accept();
sc.configureBlocking(false);// Set non-blocking
sc.register(key.selector(), SelectionKey.OP_READ );// On established pipes, registered events of interest are readable
} else if (key.isReadable()) { //flip
SocketChannel sc = null;
sc = (SocketChannel)key.channel();
ByteBuffer buffer = ByteBuffer.allocate(512);
buffer.clear();
int len = sc.read(buffer);
if(len ! = -1) {
System.out.println("[" +Thread.currentThread().getName()+"] recv :"+ new String(buffer.array(), 0, len));
}
ByteBuffer bufferToWrite = ByteBuffer.wrap("HelloClient".getBytes()); sc.write(bufferToWrite); }}Copy the code2.3.2 I/O multiplexing – poll
Summary: Design new data structures (linked lists) to provide efficiency.
Poll is not much different in nature from SELECT, except that poll does not have a limit on the maximum number of file descriptors for select.
Disadvantages: It is inefficient to go through all FD states one by one.
2.3.3 I/O multiplexing – epoll
Description: There is no limit on the number of FDS. Copying from user mode to kernel mode only needs to be done once, triggered by event notification mechanism. A FD is registered with epoll_ctl, and once the FD is ready, the corresponding FD is activated through the callback mechanism for related I/O operations.
Disadvantages:
- Cross-platform, Linux support is best.
- The underlying implementation is complex.
- Synchronization.
public static void main(String[] args) throws Exception {
final AsynchronousServerSocketChannel serverChannel = AsynchronousServerSocketChannel.open()
.bind(new InetSocketAddress(Constant.HOST, Constant.PORT));
serverChannel.accept(null.new CompletionHandler<AsynchronousSocketChannel, Object>() {
@Override
public void completed(final AsynchronousSocketChannel client, Object attachment) {
serverChannel.accept(null.this);
ByteBuffer buffer = ByteBuffer.allocate(1024);
client.read(buffer, buffer, new CompletionHandler<Integer, ByteBuffer>() {
@Override
public void completed(Integer result, ByteBuffer attachment) {
attachment.flip();
client.write(ByteBuffer.wrap("HelloClient".getBytes()));// Business logic
}
@Override
public void failed(Throwable exc, ByteBuffer attachment) {
System.out.println(exc.getMessage());// Failed processing}}); }@Override
public void failed(Throwable exc, Object attachment) {
exc.printStackTrace();// Failed processing}});while (true) {
// The while true main method ends immediately}}Copy the codeOf course, compared with the above disadvantages, its advantages can be ignored. The JDK provides an asynchronous implementation, but in a real Linux environment the underlying layer is still epoll, but with a layer of looping, it’s not really asynchronous non-blocking. And like the code invocation in the figure above, the code that handles the network connection and the business code are not decoupled well enough. Netty provides a clean, decoupled, well-structured API.
public static void main(String[] args) {
new NettyServer().serverStart();
System.out.println("Netty server started !");
}
public void serverStart(a) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(newHandler()); }});try {
ChannelFuture f = b.localAddress(Constant.HOST, Constant.PORT).bind().sync();
f.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally{ workerGroup.shutdownGracefully(); bossGroup.shutdownGracefully(); }}}class Handler extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
ByteBuf buf = (ByteBuf) msg;
ctx.writeAndFlush(msg);
ctx.close();
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { cause.printStackTrace(); ctx.close(); }}Copy the codeBossGroup’s butlers handle network requests and, when the network connection is ready, hand them over to workGroup’s workers.
Third, summary
review
- Synchronous/asynchronous: When a connection is established and a user program reads or writes data, it is synchronous if the user program eventually calls the system read() to read data, and asynchronous if it does not. Windows implements true asynchrony, and the kernel code is complex but transparent to user programs.
- Blocking/non-blocking, after the connection is established, can the user program do something else while waiting to be read or written? If it can, it is non-blocking, and if it can, it is blocking. Most operating systems support it.
Redis, Nginx, Netty, Node. Js why so sweet?
These technologies have come along with system calls in Linux kernel iterations that provide efficient handling of network requests. Understanding the underlying knowledge of the computer can be more profound understanding of I/O, know why, but also know why. With you!
Article source: Technology Sharing of CreditEase Technology Institute & CreditEase Payment and Settlement Team No. 8 – Senior engineer of Payment R&D team of CreditEase Payment and Settlement Department zhou Shengshuai “Understanding Linux Network IO Model from the Operating System Level”
Share-sharing: Zhou Shengshuai, senior engineer of payment R&D team of CreditEase Payment and Settlement Department
First published in the payment and Settlement team technical public number “Wild Pointer”