preface

Codable is a feature introduced after Swift 4.0 with the goal of replacing the NSCoding protocol.

Codable brings us a solution for JSON data parsing, but it’s not handy in many cases. So let’s explore Codable’s underlying implementation and how it can be improved.

Sil file ExplorationCodableimplementation

Let’s start with the simplest code:

struct Teacher: Codable {
    var name: String
    var age: Int
}

let jsonString = """ { "name": "Tom", "age": 23, } """

let jsonData = jsonString.data(using: .utf8)
let decoder = JSONDecoder(a)if let jsonData = jsonData,
   let result = try? decoder.decode(Teacher.self, from: jsonData) {
    print(result)
} else {
    print("Parsing failed")}Copy the code

Let’s look at the definition of Teacher in the SIL file

struct Teacher : Decodable & Encodable {
  @_hasStorage var name: String { get set }
  @_hasStorage var age: Int { get set }
  init(name: String.age: Int)
  enum CodingKeys : CodingKey {
    case name
    case age
    @_implements(Equatable.= =(_:_:)) static func __derived_enum_equals(_ a: Teacher.CodingKeys._ b: Teacher.CodingKeys) -> Bool
    var hashValue: Int { get }
    func hash(into hasher: inout Hasher)
    var stringValue: String { get }
    init?(stringValue: String)
    var intValue: Int? { get }
    init?(intValue: Int)
  }
  init(from decoder: Decoder) throws
  func encode(to encoder: Encoder) throws
}
Copy the code

We see that in addition to our original definition of Teacher, init(from decoder: decoder), func encode(to encoder: encoder, enum CodingKeys: CodingKey is the three things.

Init (from decoder: decoder) and func encode(to encoder: encoder: encoder) are mandatory for Codable, and enum CodingKeys: CodingKey is an enumeration for both methods. In other words, the original implementation of these three was required by us. However, we don’t have to do it ourselves, with the help of the compiler, which helps us improve the code.

But while we enjoy the convenience of the compiler, we also lose flexibility in Codable, because the code the compiler will help us complete is custom. Interpreting data can lead to keyword conflicts, data formatting errors, missing data, and more, so in order to make better use of Codable, we need to understand the underlying serialization and deserialization processes.

JSONDecoder

Above parse JSON data is called JSONDecoder decode method to achieve, we see from this place how to complete the data parsing, we first look at the source code:

    open func decode<T : Decodable> (_ type: T.Type.from data: Data) throws -> T {
        let topLevel: Any
        do {
            topLevel = try JSONSerialization.jsonObject(with: data, options: .allowFragments)
        } catch {
            throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: error))
        }

        let decoder = _JSONDecoder(referencing: topLevel, options: self.options)
        guard let value = try decoder.unbox(topLevel, as: type) else {
            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: [], debugDescription: "The given data did not contain a top-level value."))}return value
    }
Copy the code

The core consists of 3 steps:

1. To obtaintopLevel, callJSONSerialization.jsonObjectThis is very common. If you parse it correctly,topLevelIt’s a dictionary or an array.
2. To obtaindecoderBut thisdecoderIs notJSONDecoderType, but rather_JSONDecoderType,_JSONDecoderThe initialization method passes in two parameters, one of which is the first onetopLevelAnd there’s another oneJSONDecoderThe type ofoptionsLet’s seeoptionsHow it was acquired:

fileprivate var options: _Options {
        return _Options(dateDecodingStrategy: dateDecodingStrategy,
                        dataDecodingStrategy: dataDecodingStrategy,
                        nonConformingFloatDecodingStrategy: nonConformingFloatDecodingStrategy,
                        keyDecodingStrategy: keyDecodingStrategy,
                        userInfo: userInfo)
    }
Copy the code

Options is the strategy to parse JSON data, dateDecodingStrategy is to parse time format, dataDecodingStrategy is to parse data stream format, you can search for details, there should be a lot of data.

3. To obtaindecoderAfter the call is madeunboxMethod to get the finalvalue

So the core is the _JSONDecoder class and its unbox method

_JSONDecoder

Initialize _JSONDecoder

/// Initializes `self` with the given top-level container and options.
    fileprivate init(referencing container: Any.at codingPath: [CodingKey] =[].options: JSONDecoder._Options) {
        self.storage = _JSONDecodingStorage()
        self.storage.push(container: container)
        self.codingPath = codingPath
        self.options = options
    }
Copy the code

_JSONDecodingStorage = jsonDecodingStorage = jsonDecodingStorage = jsonDecodingStorage = jsonDecodingStorage = jsonDecodingStorage = jsonDecodingStorage = jsonDecodingStorage

Let’s look at the core unbox method:

fileprivate func unbox<T : Decodable> (_ value: Any.as type: T.Type) throws -> T? {
        return try unbox_(value, as: type) as? T
    }

    fileprivate func unbox_(_ value: Any.as type: Decodable.Type) throws -> Any? {
        if type = = Date.self {
            guard let date = try self.unbox(value, as: Date.self) else { return nil }
            return date
        } else if type = = Data.self {
            guard let data = try self.unbox(value, as: Data.self) else { return nil }
            return data
        } else if type = = URL.self {
            guard let urlString = try self.unbox(value, as: String.self) else {
                return nil
            }

            guard let url = URL(string: urlString) else {
                throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath,
                                                                        debugDescription: "Invalid URL string."))}return url
        } else if type = = Decimal.self {
            guard let decimal = try self.unbox(value, as: Decimal.self) else { return nil }
            return decimal
        } else if let stringKeyedDictType = type as? _JSONStringDictionaryDecodableMarker.Type {
            return try self.unbox(value, as: stringKeyedDictType)
        } else {
            self.storage.push(container: value)
            defer { self.storage.popContainer() }
            return try type.init(from: self)}}Copy the code

Here I went to a macro judgment code block, about NSDate and other bridging implementations, but it doesn’t affect the overall understanding, mainly because the code is too long.

When we see things like Date, Data, URL, etc., we call separate unbox methods because of the parsing strategy that is involved, and when we declare our own class or structure, we end up with type.init(from: self), which is also called init(from decoder: Decoder) method.

init(from decoder: Decoder)

Let’s take a look at Decoder. Decoder is a protocol:

    var codingPath: [CodingKey] { get }
    var userInfo: [CodingUserInfoKey : Any] { get }
    func container<Key> (keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> where Key : CodingKey
    func unkeyedContainer(a) throws -> UnkeyedDecodingContainer
    func singleValueContainer(a) throws -> SingleValueDecodingContainer
Copy the code

At first glance, I don’t know what it is. Shall we go firstSil fileLet’s see, how does the compiler implement thatinit(from decoder: Decoder):We see the formationUnkeyedDecodingContainerBecause it is too long, I give up the displaySil fileGo straight to the complete Swift implementation.

    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try container.decode(String.self, forKey: .name)
        age = try container.decode(Int.self, forKey: .age)
    }
Copy the code

Container = KeyedDecodingContainer = KeyedDecodingContainer = KeyedDecodingContainer

public struct KeyedDecodingContainer<K> : KeyedDecodingContainerProtocol where K : CodingKey {

    public typealias Key = K
    public init<Container> (_ container: Container) where K = = Container.Key.Container : KeyedDecodingContainerProtocol
    public var codingPath: [CodingKey] { get }
    public var allKeys: [KeyedDecodingContainer<K>.Key] { get }
    public func contains(_ key: KeyedDecodingContainer<K>.Key) -> Bool
    
    public func decodeNil(forKey key: KeyedDecodingContainer<K>.Key) throws -> Bool
    public func decode(_ type: Bool.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Bool
    public func decode(_ type: String.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> String
    public func decode(_ type: Double.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Double
    public func decode(_ type: Float.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Float
    public func decode(_ type: Int.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Int
    .
    public func decode<T> (_ type: T.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> T where T : Decodable

    public func decodeIfPresent(_ type: Bool.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Bool?
    public func decodeIfPresent(_ type: String.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> String?
    public func decodeIfPresent(_ type: Double.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Double?
    public func decodeIfPresent(_ type: Float.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Float?
    public func decodeIfPresent(_ type: Int.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Int?
    .
    public func decodeIfPresent<T> (_ type: T.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> T? where T : Decodable
    .
}
Copy the code

KeyedDecodingContainer defines a number of decode and decodeIfPresent parsing methods, where decodeIfPresent is used for optional values. Let’s take a look at one of these:

public func decode(_ type: Int.Type.forKey key: Key) throws -> Int {
        guard let entry = self.container[key.stringValue] else {
            throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: self.decoder.codingPath, debugDescription: "No value associated with key \(_errorDescription(of: key))."))}self.decoder.codingPath.append(key)
        defer { self.decoder.codingPath.removeLast() }

        guard let value = try self.decoder.unbox(entry, as: Int.self) else {
            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: self.decoder.codingPath, debugDescription: "Expected \(type) value but found null instead."))}return value
    }
Copy the code

We see that the core method is self.decoder.unbox, and self.decoder is the _JSONDecoder that we passed in above, and we go around and call unbox of _JSONDecoder

KeyedDecodingContainer,UnkeyedDecodingContainer,SingleValueDecodingContainerThe difference between

The difference between the three is in the contents of the container. Take KeyedDecodingContainer for example, as we saw in decode

let entry = self.container[key.stringValue]
Copy the code

The value is in dictionary style, indicating that the dictionary is stored in container.

Let’s take a look at decode of UnkeyedDecodingContainer

public mutating func decode(_ type: Int.Type) throws -> Int {
        guard !self.isAtEnd else {
            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: self.decoder.codingPath + [_JSONKey(index: self.currentIndex)], debugDescription: "Unkeyed container is at end."))}self.decoder.codingPath.append(_JSONKey(index: self.currentIndex))
        defer { self.decoder.codingPath.removeLast() }

        guard let decoded = try self.decoder.unbox(self.container[self.currentIndex], as: Int.self) else {
            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: self.decoder.codingPath + [_JSONKey(index: self.currentIndex)], debugDescription: "Expected \(type) but found null instead."))}self.currentIndex + = 1
        return decoded
    }
Copy the code

We extract the value from the container:

let value = self.container[self.currentIndex]
self.currentIndex + = 1
Copy the code

The container is an array, and the subscript value is incremented each time the container is decode. The next time the container is decode, the subscript value is automatically removed.

Finally SingleValueDecodingContainer under study, the more special, we see how _JSONDecoder generate SingleValueDecodingContainer:

public func singleValueContainer(a) throws -> SingleValueDecodingContainer {
        return self
}
Copy the code

We see SingleValueDecodingContainer return is _JSONDecoder itself, so SingleValueDecodingContainer decode method is realized in the classification of _JSONDecoder implemented:

public func decode(_ type: Int.Type) throws -> Int {
        try expectNonNull(Int.self)
        return try self.unbox(self.storage.topContainer, as: Int.self)!
    }
Copy the code

So the container put SingleValueDecodingContainer is a single value, the analytical time is as a whole, can I put my values of the three process of abstracting together under the contrast:

// KeyedDecodingContainer, using the dictionary key
let value = container[key]

/ / UnkeyedDecodingContainer, through the array subscript values
let value = container[index]

/ / SingleValueDecodingContainer, the value is the container itself
let value = container
Copy the code

So in SingleValueDecodingContainer, the container is generally put the String or Int, of course, also can put an array of the dictionary, but no longer take the elements inside, but as a whole is resolved.

Although 3 values are different, but finally get value, call or _JSONDecoder unbox method.

We finished talking about the difference between the three, but maybe some friends still do not have a specific concept, I take some practical examples to explain, should be able to understand.

KeyedDecodingContainerThe sample application

KeyedDecodingContainer should be used most often. Whenever JSON data can be serialized into a dictionary, use KeyedDecodingContainer to decode it.

struct Teacher: Codable {
    var name: String
    var age: Int
    
    enum CodingKeys: CodingKey {
        case name
        case age
    }
    
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try container.decode(String.self, forKey: .name)
        age = try container.decode(Int.self, forKey: .age)
    }
    
    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(name, forKey: .name)
        try container.encode(age, forKey: .age)
    }
}

let jsonString = """ { "name": "Tom", "age": 23, } """

let jsonData = jsonString.data(using: .utf8)
let decoder = JSONDecoder(a)if let jsonData = jsonData,
   let result = try? decoder.decode(Teacher.self, from: jsonData) {
    print(result)
} else {
    print("Parsing failed")}Copy the code

JsonString can be serialized as a dictionary, so select the KeyedDecodingContainer initialization method when initializing the container.

UnkeyedDecodingContainerThe sample application

Here is a scenario where you design a coordinate structure with attributes x-coordinate and y-coordinate:

struct Location: Codable {
    var x: Double
    var y: Double
}
Copy the code

But the data given by the service does not look like this:

let jsonString = """ { "x": 10, "y": 20, } """
Copy the code

It goes like this:

let jsonString = "[10, 20]."
Copy the code

Instead of using KeyedDecodingContainer, use UnkeyedDecodingContainer, which can be written like this:

struct Location: Codable {
    var x: Double
    var y: Double
    
    init(from decoder: Decoder) throws {
        var container = try decoder.unkeyedContainer()
        x = try container.decode(Double.self)
        y = try container.decode(Double.self)}func encode(to encoder: Encoder) throws {
        var container = encoder.unkeyedContainer()
        try container.encode(x)
        try container.encode(y)
    }
}
Copy the code

If your JSON data can be serialized into an array, use UnkeyedDecodingContainer to parse it.

UnkeyedDecodingContainer = UnkeyedDecodingContainer = UnkeyedDecodingContainer = UnkeyedDecodingContainer = UnkeyedDecodingContainer

SingleValueDecodingContainerThe sample application

Suppose the server returns a field, which could be a string or an integer.

We can customize a type:

struct TStrInt: Codable {
    
    var int: Int
    var string: String

    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()

        if let stringValue = try? container.decode(String.self) {
            string = stringValue
            int = Int(stringValue) ?? 0
        } else if let intValue = try? container.decode(Int.self) {
            int = intValue
            string = String(intValue);
        } else {
            int = 0
            string = ""}}}Copy the code

TStrInt (Int, String, Int); decoder (Int, String, Int); Decoder) can use singleValueContainer method to load data directly, and then try to use Int, String type to parse, so you can complete the requirements of the scene.

Special scenarios encountered during development

In our development, although in most cases, parsing data directly maps keywords in JSON data, we often encounter some special cases, such as above, the coordinate is returned by array, the field type may be string or integer, and so on.

The best thing to do in these cases is to implement the Codable protocol on your own, but that’s way too cumbersome for Codable, and not the way we programmers like it. So we explored how to make minimal changes to Codable for our special needs.

Keyword conflicts with system

The most common type of field conflict is id. This solution is relatively simple and can be found in many tutorials:

    enum CodingKeys: String.CodingKey {
        case kid = "id"
    }
Copy the code

We call the property kid and set the original value of the enumeration to ID to map the ID field in the JSON data to the kid property.

The default value is used when the value is missing

In a service scenario, one field stores a Bool. If the field is missing, the value is true by default. For example:

struct Person: Codable {
    var isMan: Bool
}
Copy the code

Our first reaction is to make isMan optional:

struct Person: Codable {
    var isMan: Bool?
}
Copy the code

However, it is not so convenient to use, and you have to make two judgments. If you use it too much in the project, you will feel very sick. An experienced programmer would encapsulate a method to retrieve true Bool values. In Swift, we can use computed properties and hide the properties:

struct Person: Codable {
    private var isMan: Bool?
    var is_Man: Bool {
        guard let isMan = isMan else {
            return true
        }
        return isMan
    }
}
Copy the code

Unexpected change or missing of server data type

In fact, all the special cases can be solved as long as they can be predicted in advance, but clearly agreed with the server data type, the result is the return of other types of data, or the server has guaranteed that the value must have a value, but the result is no value on the line.

One of the most annoying things about Codable is that when a value resolution fails, the program throws an error outward into the function block. You may be able to retrieve the error area, but you may not be able to retrieve the correctly resolved value. For example, in a list data similar to the moments of friends, the URL of one avatar is missing, resulting in the parsing error of the whole list. When displaying UI clearly, you just need to replace the wrong avatar with the default one, and the whole list will not be displayed.

Therefore, we need to understand that the server is not trusted. So what to do? You might think of making all model attributes optional:

struct Teacher: Codable {
    var name: String?
    var age: Int?
}
Copy the code

While this ensures that the parse function is properly parsed to the end, it’s annoying to unpack each of these model attributes when you use them, although you can do this:

extension Optional where Wrapped= =String {
    var value: String {
        switch self {
        case .some:
            return self!
        case .none:
            return ""}}}Copy the code

This is a little bit better than unpacking each time, but is there a better way to call one more property than usual?

This will probably use the @propertyWrapper property, which is a bit like a Python decorator, if you’re not familiar with it.

Let’s define a protocol DefaultValue:

protocol DefaultValue {
    static var defaultValue: Self { get}}extension String: DefaultValue {
    static let defaultValue = ""
}

extension Int: DefaultValue {
    static let defaultValue = 0
}
Copy the code

Let’s define what the default assignment for each type should be if something unexpected happens during parsing.

Then we implement the property wrapper:

typealias DefaultCodable = DefaultValue & Codable

@propertyWrapper
struct Default<T: DefaultCodable> {
    var wrappedValue: T
}

extension Default: Codable {
    
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        wrappedValue = (try? container.decode(T.self)) ?? T.defaultValue
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        try container.encode(wrappedValue)
    }
}
Copy the code

The type T wrapped in an attribute should itself satisfy Codable and DefaultValue so that it can be set to a DefaultValue if parsing fails. Our wrapper, Default, also implements the Codable protocol. Even though we use a property wrapper, which is type T, we are actually of type Default and set the Default value for resolution failure in the protocol method.

Next we need to implement the KeyedDecodingContainer and UnkeyedDecodingContainer extensions:

extension KeyedDecodingContainer {
    func decode<T> (_ type: Default<T>.Type.forKey key: KeyedDecodingContainer<K>.Key) throws -> Default<T> where T : DefaultCodable {
        try decodeIfPresent(type, forKey: key) ?? Default(wrappedValue: T.defaultValue)
    }
}

extension UnkeyedDecodingContainer {
    mutating func decode<T> (_ type: Default<T>.Type) throws -> Default<T> where T : DefaultCodable {
        try decodeIfPresent(type) ?? Default(wrappedValue: T.defaultValue)
    }
}
Copy the code

The extension is written because the original decode method throws an error directly out of the function when the value is nil from the original container. The unbox method of _JSONDecoder is not called at all, and the Codable protocol method of the Default type is not called.

Here, our property wrapper, Default, is complete, and we just need to wrap the property wrapper around each property of the model:

struct Teacher: Codable {
    @Default var name: String
    @Default var age: Int
}
Copy the code

Let’s try missing value resolution:

let jsonString = """ { "name": "Tom", } """


let jsonData = jsonString.data(using: .utf8)
let decoder = JSONDecoder(a)if let jsonData = jsonData,
   let result = try? decoder.decode(Teacher.self, from: jsonData) {
    print(result)
    print("name: \(result.name)")
    print("age: \(result.age)")}else {
    print("Parsing failed")}// Print the result
Teacher(_name: SwiftSIL.Default<Swift.String>(wrappedValue: "Tom"), _age: SwiftSIL.Default<Swift.Int>(wrappedValue: 0))
name: Tom
age: 0
Program ended with exit code: 0
Copy the code

We see that although the age value is missing, the parse does not fail, but instead assigns a default value of 0 to age. Name is a Default

and age is a Default

.

conclusion

Codable may not be as useful as YYModel or HandyJson yet, but Codable is Swift’s baby boy. There will be more features coming in Codable, like @propertyWrapper in Swift5.1. Before Swift5, Apple worked on ABI stability, and after ABI stability, Apple will work on Swift syntax, features, etc. All in all, Swift will grow rapidly and Swift will become a more and more complex language. Come on, guys!

reference

• Set defaults for Codable decoding using Property Wrapper