SwiftNIO is a cross-platform asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients.

It’s like Netty, but written for Swift.

SwfitNIO is an event-driven, cross-platform web application development framework that aims to help developers quickly develop high performance and easy to maintain server and client application protocols.

For those of you who like to explore the origin, we can first understand some concepts of Netty.

This article is a whole rational article, mainly from:

  • Getting Started: by far the most thorough analysis of Netty’s high performance principles and frameworks
  • apple/swift-nio
  • Apple’s open source version of Netty: SwiftNIO
  • try! Swift Tokyo 2018 – Event driven networking for Swift

Netty high performance design

As an asynchronous event-driven network, Netty’s high performance mainly comes from its I/O model and thread processing model. The former decides how to send and receive data, and the latter decides how to process data.

I/O model

What channel is used to send data to each other? There are three popular I/O models in Java:

  • BIO: Synchronous and blocking, server implementation mode is one thread per connection, that is, when the client has a connection request, the server side needs to start a thread to process, if the connection does not do anything will cause unnecessary thread overhead, of course, can be improved through the thread pool mechanism. BIO mode is suitable for small and fixed number of connections. This mode has high requirements on server resources, and concurrency is limited to applications. The program is intuitive, simple and easy to understand.
  • NIO: Synchronous non-blocking, server implementation mode is one request one thread, that is, the client sent connection requests are registered to the multiplexer, the multiplexer polling for connection I/O requests to start a thread for processing. NIO mode is suitable for the architecture with a large number of short connections (light operation), such as chat server, concurrency is limited to the application, and programming is complicated.
  • AIO: Asynchronous, non-blocking, server implementation mode is a valid request for one thread, the CLIENT I/O request is completed by the OS to notify the server application to start the thread to process. The AIO approach is used for architectures with large number of connections and relatively long (heavy operation) connections, such as photo album servers, which fully invoke the OS to participate in concurrent operations and are more complex to program.

The key to Netty’s non-blocking I/O implementation is based on the I/O multiplexing model.

Netty’s IO thread, NioEventLoop, can process hundreds or thousands of client connections concurrently because it aggregates multiplexer selectors. When a thread reads and writes data from a client Socket channel, if no data is available, the thread can perform other tasks. Threads typically spend idle time for non-blocking IO operations on other channels, so a single thread can manage multiple input and output channels. Since both read and write operations are non-blocking, this improves the efficiency of THE I/O threads and prevents threads from being suspended due to frequent I/O blocking. A single I/O thread can process N client connections and read/write operations concurrently, which fundamentally solves the traditional synchronous blocking I/O connection-one-thread model and greatly improves the performance, resilience, and reliability of the architecture.

Based on the Buffer

Traditional I/O is byte – or character-stream-oriented, reading one or more bytes sequentially from a Stream in a streaming manner, so you cannot arbitrarily change the position of the read pointer.

In NIO, traditional I/O streams are abandoned and the concepts of channels and buffers are introduced. In NIO, data can only be read from a Channel to a Buffer or written from a Buffer to a Channel.

Unlike traditional SEQUENTIAL IO operations, buffer-based operations in NIO can read data anywhere at will.

Threading model

Event-driven model

There are two ways to design a program for an event processing model.

  1. Polling mode: the thread continuously polling to access the relevant event source has no event, event occurs on the call event processing logic;

  2. Event-driven mode: When an event occurs, the main thread puts the event into the event queue, and the other thread continuously consumes the event in the event list, and invokes the corresponding processing logic to process the event. Event-driven approach, also known as message notification approach, is actually the idea of observer pattern in design pattern.

    There are four basic components:

    1. Event Queue: An entry to receive events and store events to be processed.

    2. Event mediators: distribute different events to different business logic units.

    3. Event channel: the communication channel between the dispenser and the processor;

    4. Event processor: Implements business logic and sends an event to trigger the next operation.

    It can be seen that, compared with traditional polling mode, event-driven has the following advantages:

    1. Good scalability: distributed asynchronous architecture, high decoupling between event handlers, can easily extend the event processing logic;

    2. High performance: Queue-based staging of events facilitates parallel asynchronous processing of events.

Reactor thread model

Reactor stands for Reactor, and the Reactor model is an event-driven processing pattern of service requests that are simultaneously passed to a service processor by one or more inputs.

The server program processes incoming multiple requests and synchronously dispatches them to the corresponding processing thread. The Reactor mode is also called the Dispatcher mode, which means that I/O is overmultiplexed and uniformly listens for events, and dispatches the events after receiving them. It is one of the essential technologies for writing high-performance network servers.

The Reactor model has two key components:

  1. Reactor: The Reactor runs in a separate thread that listens for and distributes events to the appropriate handlers to react to IO events. It is like a company’s telephone operator, which takes calls from customers and transfers the line to the appropriate contact;

  2. Handlers: The actual events that the handler executes I/O events to accomplish, similar to the actual officials in the company that the customer wants to talk to. Reactor responds to I/O events by scheduling appropriate handlers that perform non-blocking operations.

There are three variations of the Reactor model, depending on the number of reactors and the number of Hanndler threads:

  1. Single Reactor single thread;

  2. Single Reactor multi-thread;

  3. Master-slave Reactor multithreading.

In this way, Reactor is an implementation

while (true) { selector.select(); ... }Copy the code

The circular thread, which produces a steady stream of new events, is aptly called a reactor.

Netty threading model

Netty has made some modifications based on the master-subordinate multi-threading model (as shown in the figure below), in which there are multiple Reactors in the master-subordinate multi-threading model:

  1. The MainReactor is responsible for the client connection requests and forwards the requests to the SubReactor.

  2. The SubReactor is responsible for IO read and write requests for the corresponding channel.

  3. Tasks that are not IO requests (specific logical processing) are written directly to the queue for worker Threads to process.

Asynchronous processing

The concept of asynchrony is the opposite of synchronization. When an asynchronous procedure call is issued, the caller does not immediately get the result. The widget that actually processes the call, when complete, notifies the caller through state, notification, and callback.

I/O operations in Netty are asynchronous. Operations such as Bind, Write, and Connect simply return a ChannelFuture.

The caller does not get the result immediately, but through the future-Listener mechanism, the user can easily obtain the IO operation result actively or through the notification mechanism.

When the Future object is first created, it is in an incomplete state. Callers can get the status of the operation execution by returning ChannelFuture, and register the listening function to perform the completed operation.

Common operations are as follows:

  1. Using isDone method to judge whether the current operation is complete;

  2. The isSuccess method is used to determine whether the current operation has completed successfully;

  3. GetCause () getCause () getCause () getCause () getCause

  4. The isCancelled method is used to determine whether the current operation has been cancelled.

  5. The addListener method registers the listener and notifies the specified listener when the operation has completed (the isDone method returns complete). If the Future object is complete, understand the listener specified by the notification.

Instead of blocking I/O, the thread is blocked after the I/O is executed until the operation is complete. The benefits of asynchronous processing are that it does not block the thread and that the thread can execute other programs during I/O operations, resulting in more stability and higher throughput in high concurrency situations.

Netty structure

The Server contains one Boss NioEventLoopGroup and one Worker NioEventLoopGroup.

NioEventLoopGroup is equivalent to an event loop group. This group contains multiple event loops called NioEventLoop. Each NioEventLoop contains a Selector and an event loop thread.

The task performed by each Boss NioEventLoop consists of three steps:

  1. Polling Accept events;
  2. Handle Accept I/O events, establish a connection with the Client, generate the NioSocketChannel, and register the NioSocketChannel with a Worker NioEventLoop Selector;
  3. Process tasks in the task queue, runAllTasks. Tasks in the task queue include tasks executed by the user calling EventLoop. Execute or schedule, or tasks submitted to the EventLoop by other threads.

The tasks performed by each Worker NioEventLoop loop consist of three steps:

  1. Polling Read and Write events.
  2. Handle I/O events, namely Read and Write events, when the NioSocketChannel readable and writable events occur.
  3. Process tasks in the task queue, runAllTasks.

SwiftNIO architecture

  • EventLoopGroup interface
  • EventLoop interface
  • The Channel interface
  • ChannelHandler interface
  • Bootstrap Multiple data structures
  • ByteBuffer structure
  • EventLoopFuture generic class
  • EventLoopPromise generic structure

EventLoops and EventLoopGroups

EventLoop is the most basic I/O primitive in SwfitNIO. It waits for an event to occur and fires some kind of callback operation when it does. In most SwfitNIO applications, the number of EventLoop objects is small, usually one or two per CPU core. In general, an EventLoop exists for infinite event distribution throughout the life cycle of an application.

Eventloops can be combined into Eventloopgroups, and EventLoopgroups provide a mechanism for distributing workloads among eventloops. For example, when a server is listening for external connections, the socket used to listen for connections is registered with an EventLoop. However, we don’t want one EventLoop to carry all the connection load, so we can spread the connection load among multiple EventLoops through EventLoopGroup.

At present, SwiftNIO provides a EventLoopGroup implementation (MultiThreadedEventLoopGroup) and two EventLoop implementation (SelectableEventLoop and EmbeddedEventLoop).

  • MultiThreadedEventLoopGroup creates multiple threads (using POSIX pthreads library), and assign a SelectableEventLoop object for each thread.
  • SelectableEventLoop uses selectors (based on kQueue or epoll) to manage IO events from files and the network.
  • The EmbeddedEventLoop is an empty EventLoop that does nothing, mainly for testing.

Channels, ChannelHandler, ChannelPipeline and ChannelHandlerContext

Despite the importance of EventLoop, most developers don’t interact with it much beyond creating EventLoopPromises and scheduling jobs. Developers often use channels and channelhandlers.

Each file descriptor corresponds to a Channel, which manages the lifecycle of the file descriptor and handles the events that occur on the file descriptor: Whenever EventLoop detects an event related to the corresponding file descriptor, the Channel is notified.

A ChannelPipeline consists of a series of ChannelHandlers who are responsible for processing events in a Channel in sequence. ChannelPipeline is like a data processing pipeline, hence the name.

ChannelHandler is either Inbound, Outbound, or both. The Inbound ChannelHandler handles Inbound events, such as reading data from the socket, closing the socket, or other remote initiated events. The Outbound ChannelHandler handles Outbound events, such as writing data, initiating connections, and closing the local socket.

The ChannelHandler processes events in a certain order, for example, read events are passed from the front to the back of the pipe, and write events are passed from the back to the front of the pipe. Each ChannelHandler will generate a new event for the next ChannelHandler after handling an event.

Channelhandlers are highly reusable components, so they are designed to be as lightweight as possible, with each ChannelHandler handling only one type of data transformation. This allows you to combine various ChannelHandlers flexibly, improving code reusability and encapsulation.

We can use ChannelHandlerContext to track the location of the ChannelHandler in ChannelPipeline. The ChannelHandlerContext contains references to the current ChannelHandler to the previous and next ChannelHandler, so that any time a ChannelHandler is still in the pipeline, a new event can be triggered.

SwiftNIO has several channelHandlers built in, including an HTTP parser. In addition, SwiftNIO provides several Channel implementations, Examples are ServerSocketChannel (for receiving connections), SocketChannel (for TCP connections), DatagramChannel (for UDP sockets), and EmbeddedChannel (for testing).

It is important to note that ChannelPipeline is thread safe, which means there is no separate synchronization. All Handlers in a ChannelPipeline are placed in the same thread and processed by EventLoop. This also means that all Handlers cannot be blocked or must be none blocking. If it blocks, the other Handlers in the pipeline wait until the current Handler finishes processing. Therefore, it is best to put the potentially blocking, or potentially highly concurrent processing into another child thread.

Bootstrap

SwiftNIO provides a number of Bootstrap objects to simplify Channel creation. Some Bootstrap objects provide additional functionality, such as support for Happy Eyeballs.

There are currently three types of Bootstrap available for SwiftNIO: ServerBootstrap (for listening channels), ClientBootstrap (for TCP channels), and DatagramBootstrap (for UDP channels).

ByteBuffer

SwiftNIO provides ByteBuffer, a fast copy-on-write ByteBuffer that is a key building block for most SwiftNIO applications.

The ByteBuffer provides many useful features, as well as “hooks” that let you use it in unsafe mode. This approach yields better performance at the expense of the potential memory problems for the application. In general, it is recommended to use byteBuffers in safe mode.

EventLoopPromise and EventLoopFuture

The main difference between concurrent and synchronous code is that not all actions can be completed immediately. For example, when writing data to a Channel, an EventLoop might not immediately flush the data onto the network. To this end, SwiftNIO provides EventLoopPromise and EventLoopFuture for managing asynchronous operations.

EventLoopFuture is actually a container for the return value of a function at some point in the future. Each EventLoopFuture object has a corresponding EventLoopPromise that holds the actual result. Once the EventLoopPromise executes successfully, an EventLoopFuture is complete.

Polling is a very inefficient way to check if an EventLoopFuture is complete, so EventLoopFutures are designed to receive callbacks. That is, the callback function is executed when it has a result.

EventLoopFuture handles the scheduling and ensures that the callback is executed on the EventLoop where the EventLoopPromise was originally created, so there is no need to do any more synchronization with the callback.