Serialization generally has two main purposes:

  • Network transmission
  • Object persistence

When evaluating the merits of a codec framework, the following factors will be considered:

  • Whether cross-language support is available;
  • The size of code stream after encoding;
  • Codec performance;
  • Class library is small, API is easy to use;
  • The amount of work and difficulty that users need to develop manually;
  • Open source and community activity of the class library;

In the same environment, the larger the number of encoded bytes is, the more space will be occupied during storage, the higher the hardware cost of storage, and the more bandwidth will be occupied during network transmission, resulting in reduced system throughput.

Mainstream codec framework

Google 的 Protobuf

Protobuf (Google Protocol Buffers), open-source by Google. It describes data structures as.proto files, and code generation tools can generate POJO objects and protobuf-related methods and properties for the corresponding data structures.

Its features:

  • Structured data storage format
  • Efficient codec performance
  • Language independent, platform independent, good scalability
  • A variety of languages are supported

In contrast to XML, although XML is very readable and extensible, and is well suited for describing data structures, the time cost of XML parsing and the space cost that XML sacrifices for readability are too high to be a high-performance communication protocol. Protobuf uses binary encoding, which gives it greater space and performance advantages.

Facebook 的 Thrift

Thrift is a cross-language RPC server framework developed by Facebook in 2007. It provides multi-language compilation and multiple server working modes. Users describe interface functions and data types through Thrift IDL (interface Definition Language), and then generate interface files of various language types through Thrift compilation environment. Users can develop client code and server code in different languages according to their own needs.

Actual combat Protobuf

The installation

You will need protobuf’s compiler to generate code for the specified language.

Hence the need to install the compiler, tutorial source: Apple/Swift-Protobuf

$brew install swift-protobuf $protoc --version libprotoc 3.14.0Copy the code

.proto

Create a.proto file and define messages in it, then generate Swift code via the Protobuf compiler.

For.proto files, we can edit them using any IDE or text editor. I recommend using VSCode with the vscode-proto3 plug-in.

The official documentation of Google Protocol Buffer: documentation

Let’s simply create a movie. Proto:

syntax = "proto3";

message Movie {
    enum Genre {
        COMEDY = 0;
        ACTION = 1;
        HORROR = 2;
        ROMANCE = 3;
        DRAMA = 4;
    }

    string title = 1;
    Genre genre = 2;
    int32 year = 3;
}
Copy the code

Then compile it:

$ protoc --swift_out=. movie.proto
Copy the code

Upon execution, the compiler generates the movies.pb.swift file.

The functional requirements

Create both MovieClient and MovieServer, and use SwiftNIO to send our Movie structure from the client to the server.

Both ends need to rely on Apple/Swift-NIO and Apple/Swift-Protobuf

/ / swift - tools - version: 5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "MovieClient",
    dependencies: [
        .package(url: "https://github.com/apple/swift-nio.git", from: "2.0.0"),
        .package(name: "SwiftProtobuf", url: "https://github.com/apple/swift-protobuf.git", from: "1.14.0")
    ],
    targets: [

        .target(
            name: "MovieClient",
            dependencies: [
                .product(name: "NIO", package: "swift-nio"),
                "SwiftProtobuf"
            ]),
        .testTarget(
            name: "MovieClientTests",
            dependencies: ["MovieClient"]),])Copy the code

Both client-side and server-side projects import movies.pb.swift.

MoviewClient project directory:

MovieServer’s project directory:

Handler of the codes at both ends is the core of their respective services. Other codes are common usage of SwiftNIO, so you can use them for reference.

Use 8030 as the port of the server.

If you’re not familiar with creating a Swift project, check out the previous articles on OldBirds

Implement MovieClient

The implementation of Movieclient.swift is as follows:

import Foundation
import NIO

final class MovieClient {
    private let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)

    private var host: String
    private var port: Int

    init(host: String.port: Int) {
        self.host = host
        self.port = port
    }

    func start(a) throws {
        do {
            let channel = try bootstrap.connect(host: host, port: port).wait()
            try channel.closeFuture.wait()
        } catch let error {
            throw error
        }
    }

    func stop(a) {
        do {
            try group.syncShutdownGracefully()
        } catch let error {
            print("Error shutting down \(error.localizedDescription)")
            exit(0)}print("Client connection closed")}private var bootstrap: ClientBootstrap {
        return ClientBootstrap(group: group)
            // Enable SO_REUSEADDR.
            .channelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
            .channelInitializer { channel in
                channel.pipeline.addHandler(MovieClientHandler()}}}Copy the code

MovieClientHandler. Swift

import Foundation
import NIO

class MovieClientHandler: ChannelInboundHandler {
    typealias InboundIn = ByteBuffer
    typealias OutboundOut = ByteBuffer

    func channelActive(context: ChannelHandlerContext) {
        var movie = Movie()
        movie.genre = .romance
        movie.title = "The girls we used to chase all those years."
        movie.year = 2011

        do {
            /// serialize objects
            let binaryData: Data = try movie.serializedData()

            / / create a buffer
            var buffer = context.channel.allocator.buffer(capacity: binaryData.count)

            // Write data to buffer
            buffer.writeBytes(binaryData)

            let promise: EventLoopPromise<Void> = context.eventLoop.makePromise()
            promise.futureResult.whenComplete { (_) in
                print("Sent data, closing the channel")
                context.close(promise: nil)}// write and flush the data
            context.writeAndFlush(wrapOutboundOut(buffer), promise: promise)
        } catch let error {
            print(error.localizedDescription)
        }
    }

    func errorCaught(context: ChannelHandlerContext.error: Error) {
        print(error.localizedDescription)
        context.close(promise: nil)}}Copy the code

Implementation of main.swift:

let client = MovieClient(host: "localhost", port: 8030)

do {
    try client.start()
} catch let error {
    print("Error: \(error.localizedDescription)")
    client.stop()
}
Copy the code

Implement MovieServer

Movieserver.swift implementation:


import Foundation
import NIO

final class MovieServer {

    private let group = MultiThreadedEventLoopGroup(numberOfThreads: System.coreCount)
    private var host: String
    private var port: Int

    init(host: String.port: Int) {
        self.host = host
        self.port = port
    }

    func start(a) throws {
        do {
            let channel = try serverBootstrap.bind(host: host, port: port).wait()
            print("Listening on \(String(describing: channel.localAddress))...")
            try channel.closeFuture.wait()
        } catch let error {
            throw error
        }
    }

    func stop(a) {
        do {
            try group.syncShutdownGracefully()
        } catch let error {
            print("Error shutting down \(error.localizedDescription)")
            exit(0)}print("Client connection closed")}private var serverBootstrap: ServerBootstrap {
        return ServerBootstrap(group: group)
            // Specify backlog and enable SO_REUSEADDR for the server itself
            .serverChannelOption(ChannelOptions.backlog, value: 256)
            .serverChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
            .childChannelInitializer { channel in
                // Ensure we don't read faster than we can write by adding the BackPressureHandler into the pipeline.
                channel.pipeline.addHandler(BackPressureHandler()).flatMap { v in
                    channel.pipeline.addHandler(MovieServerHandler())
                }
            }
            .childChannelOption(ChannelOptions.socket(IPPROTO_TCP.TCP_NODELAY), value: 1)
            // Enable SO_REUSEADDR for the accepted Channels
            .childChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
            .childChannelOption(ChannelOptions.maxMessagesPerRead, value: 16)
            .childChannelOption(ChannelOptions.recvAllocator, value: AdaptiveRecvByteBufferAllocator()}}Copy the code

Movieserverhandler. swift

import Foundation
import NIO

class MovieServerHandler: ChannelInboundHandler {
    typealias InboundIn = ByteBuffer
    typealias OutboundOut = ByteBuffer

    func channelRead(context: ChannelHandlerContext.data: NIOAny) {
        // Convert NIOAny to ByteBuffer
        var buffer = unwrapInboundIn(data)

        // Get the number of bytes readable
        let readableBytes = buffer.readableBytes

        // Read from buffer
        guard let received = buffer.readBytes(length: readableBytes) else {
            return
        }

        // Convert bytes to data
        let receivedData = Data(bytes: received, count: received.count)

        do {
            // deserialize
            let movie = try Movie(serializedData: receivedData)
            print("Received:\(movie)")
            // Do something else
        } catch let error {
            print("error: \(error.localizedDescription)")}}func channelReadComplete(context: ChannelHandlerContext) {
        context.flush()
    }

    func errorCaught(context: ChannelHandlerContext.error: Error) {
        print("error: \(error.localizedDescription)")
        context.close(promise: nil)}}Copy the code

Implementation of main.swift:

let server = MovieServer(host: "localhost", port: 8030)

do {
    try server.start()
} catch let error {
    print("Error: \(error.localizedDescription)")
    server.stop()
}
Copy the code

The results

Client output:

Sent data, closing the channel
Program ended with exit code: 0
Copy the code

Server-side output:

Listening on Optional([IPv4]127.0.0.1/127.0.0.1:8030).Received:MovieServer.Movie:
title: "The girls we used to chase all those years."
genre: ROMANCE
year: 2011
Copy the code

Fulfill functional requirements.

conclusion

The main purpose of this article is to introduce some basic knowledge of network codec, including Protobuf sample for simple explanation, so as to know the basic usage process of Protobuf in SwiftNIO, generally speaking, it is relatively simple:

  1. Proto is declared and swift code is generated using probuf’s compiler. This generated code is introduced into the client and server side.
  2. Create a Movie object in the client, serialize it into Data, and send it to the server via SwiftNIO;
  3. The server receives the Data from the client, reads the Data from ByteBuffer, converts it into Data, deserializes the Data into Movie objects through methods, and completes decoding.

Proto file to generate object code, easy to use; Serialization deserialization directly corresponds to the program in the data class, do not need to parse after mapping; Binary message, high performance/efficiency;

For me, maybe the cost is to write Proto and learn its syntax. At the client level, adoption is still not as widespread as JSON.

Refer to the

  • Serialization and deserialization
  • Protocol Buffers with SwiftNIO

To enter the world of SwiftNIO, please follow the official wechat account: OldBirds

More articles in series

  • IO model of SwiftNIO principle
  • Pseudo – one-step improvement of synchronous blocking IO model
  • SwiftNIO actual text modification server
  • SwiftNIO actual TCP sticky packet unpacking problem
  • SwiftNIO real serialization
  • Protobuf based RPC Framework for SwiftNIO