By Soroush Khanlou, translator: Zheng Yi; Proofreading: NUMbbbbb, PMST; Finalized: Forelax

The URL class in the Foundation framework provides very comprehensive functionality, and the URLSession class has since been added to iOS 7. However, multipart file uploading is still missing from the base library.

What is a multipart request?

Multipart coding is actually a way of uploading large files over the network. In the browser, sometimes you will select a file as part of a form submission. This file is uploaded as a multipart request.

At first glance, a multipart request looks like a normal request. The difference is that the multipart request additionally specifies a unique encoding for the HTTP request body. Multipart encoding does something slightly different than JSON encoding ({“key”: “value”}) or URL character encoding (key=value). Because the multipart request body is really just a string of byte streams, the receiving entity needs to know the boundaries between the parts of the byte stream when parsing the data. So multipart requests need to use “boundaries” to resolve this problem. In the content-Type at the request header, we can define boundary:

Accept: application/json
Content-Type: multipart/form-data; boundary=khanlou.comNczcJGcxe
Copy the code

The specific content of Boundary is not important, the only thing that needs to be noted is that it should not be repeated in the request body (so as to reflect the role of Boundary). You can use the UUID as a boundary.

Each part of the request can be plain data (such as an image) or metadata (usually text, corresponding to a name, forming a key-value pair). If the data were pictures, it would look something like this:

--<boundary>
Content-Disposition: form-data; name=<name>; filename=<filename.jpg>
Content-Type: image/jpeg

<image data>
Copy the code

If it is plain text, it looks like this:

--<boundary>
Content-Disposition: form-data; name=<name>
Content-Type: text/plain

<some text>
Copy the code

The request will end with a boundary with two hyphens, —

–. (Note here that all new lines must be carriage return newlines.)

That’s all there is to say about multipart requests, and it’s not particularly complicated. In fact, when writing the first client implementation of multipart encoding, I somewhat resisted reading the RFC for Multipart /form-data. But as I began to read it, I understood the agreement better. The entire document is very readable and easily accessible to the source of knowledge.

I implemented the above functionality in the open source Backchannel SDK. BAKUploadAttachmentRequest and BAKMultipartRequestBuilder class contains a method of dealing with the mulitipart. In this project, only processing of a single file is covered, and metadata is not included. But as an example, it’s still a good example of how mulitiPart requests are constructed. Additional implementation code can be added to support metadata and multi-file capabilities.

There is a problem with multiple file uploads using one request or one file for each request. The problem is that if you try to upload many files at once, the app will flash back. This is because with this version of the code, the loaded data will be directly into the memory, in the case of memory explosion, even using the most powerful flagship phone will have a flash back.

Read data from a hard disk as a stream

The most common solution is to stream data from a hard disk. The idea is that bytes of file data remain on the hard disk until they are read and sent to the network. Only a small amount of mirror data is retained in memory.

At present, I can think of two ways to solve this problem. The first method writes all the data in the multipart request body to a new file on the hard disk and uses the URLSession uploadTask(with Request: URLRequest, fromFile fileURL: The URL) method converts a file to a stream. This works, but I don’t want to create a new file for every request and save it to hard drive. This means that the file needs to be deleted after the request is made.

The second method is to merge the data from the memory and the hard disk together and output the data to the network through a unified interface.

If you think the second method sounds like a class cluster, congratulations, you’re exactly right. Many common Cocoa classes allow you to create subclasses and implement superclass methods that behave the same as the parent class. Recall the -count property and the -objectatIndex: method of NSArray. Because all other methods of NSArray are implemented based on the -count attribute and the -objectatIndex: method, you can easily create optimized versions of NSArray subclasses.

You can create an NSData subclass that doesn’t actually read from the hard disk, but just creates a pointer that points directly to the data on the hard disk. This has the advantage of eliminating the need to load data into memory for reading. This approach is called memory mapping and is based on the Unix method MMAP. You can use this NSData feature with.mappedifsafe or alwaysMapped. Because NSData is a class cluster, we’ll create a ConcatenatedData subclass (just as a flattening collection works in Swift) that treats multiple NSData objects as one continuous NSData. Once the creation is complete, we are ready to solve this problem.

By looking at all the native methods of NSData, you can see that -count and -bytes are the ones that need to be implemented. It’s not hard to implement -count, we can add up the sizes of all NSData objects; There is a problem with implementing -bytes. -bytes returns a pointer to a contiguous buffer, which we do not currently have.

In the base library, the NSInputStream class is provided to handle discontinuous data. Luckily, NSInputStream is also a class cluster. We can create a subclass that merges multiple streams. When using a subclass, it feels like a stream. By using the +inputStreamWithData: and +inputStreamWithURL: methods, you can easily create an input stream that represents the files on your hard disk and the data in memory (e.g. boundaries).

By reading the source code for the best third-party networking libraries, you’ll find that AFNetworking takes this approach. (Alamofire, the Swift version of AFNetworking, takes the first approach, loading all the data into memory, but writing it to a file on the hard drive if it gets too big.)

Put all the pieces together

You can see my implementation of the serial input stream (in Objective-C, and I’ll probably write a Swift version later) here.

Streams can be grouped together through the SKSerialInputStream class. The prefix and suffix attributes are shown below:

extension MultipartComponent {
    var prefixData: Data {
        let string = """
        \(self.boundary)
        Content-Disposition: form-data; name="\(self.name); filename="\(self.filename)"
        """
        return string.data(using: .utf8)
    }
    
    var postfixData: Data {
        return "\r\n".data(using: .utf8)
    }
}
Copy the code

Combining the metadata with the dataStream of the file yields an input stream:

extension MultipartComponent {
    var inputStream: NSInputStream {
        
        let streams = [
            NSInputStream(data: prefixData),
            self.fileDataStream,
            NSInputStream(data: postfixData),
        ]
    
        return SKSerialInputStream(inputStreams: streams)
    }
}
Copy the code

Once you have created each part of the input stream, you can combine all the streams together to create a complete input stream. Add a boundary at the end of the request:

extension RequestBuilder {
    var bodyInputStream: NSInputStream {
        let stream = parts
            .map({$0.inputStream })
            + [NSInputStream(data: "--\ [self.boundary)-".data(using: .utf8))]
    
        return SKSerialInputStream(inputStreams: streams)
    }
}
Copy the code

Finally, assign bodyInputStream to the httpBodyStream property of the URL request:

let urlRequest = URLRequest(url: url)

urlRequest.httpBodyStream = requestBuilder.bodyInputStream;
Copy the code

Note that the httpBodyStream and httpBody properties are mutually exclusive — both properties do not work at the same time. Setting httpBodyStream invalidates the Data version of httpBody and vice versa.

The key to streaming file uploads is the ability to combine multiple input streams into a single stream. The SKSerialInputStream class does the job. Although there are some difficulties in subclassing NSInputStream, once we solve this problem, we are close to success.

Issues to be aware of during subclassing

The process of subclassing NS putStream is not easy, even difficult. You have to implement 9 methods. For seven of these methods, the parent class has only a trivial default implementation. Only 3 of the 9 methods are mentioned in the documentation, so you still have to implement 6 NSStream (parent of NSInputStream) methods, 2 of which are run loop methods, and allow null implementations. Before that, you need to implement three additional private methods, but that’s not necessary now. In addition, you need to define three read-only properties: streamStatus, streamError, and Delegate.

After working through the subclassing-related details above, the next challenge is to create an NSInputStream subclass that behaves as expected by API users. However, the heavy coupling of this class state is not easy to spot.

There are some states that require consistent behavior. For example, hasBytesAvailable is different from other states, but there is a subtle connection. In a recent bug I found, the hasBytesAvailable attribute returns self.currentIndex! = self.inputStream.count, but this causes a bug that keeps the stream open and eventually causes requests to time out. The way to fix this bug is to return YES instead, but I never found the root cause of the bug.

Another state, streamStatus, has many possible values, the two most important of which are NSStreamStatusOpen and NSStreamStatusClosed.

The last interesting state is the number of bytes, which is returned from the read method. In addition to returning a positive integer, this property returns -1, which means an error occurred and requires further examination of the non-empty property streamError for more information. The number of bytes can also return 0, which is another way to indicate the end of a stream, according to the documentation.

The documentation doesn’t tell you which combinations of states are meaningful. For example, if a stream produces a streamError and the state is NSStreamStatusClosed instead of NSStreamStatusError, is there a problem? It’s very difficult to manage all the states, but eventually it can be done.

I’m not so confident that the SKSerialStream class will work in all cases. However, it seems that SKSerialStream does a good job of supporting uploading multipart data by using URLSession. If you find any problems with this code, please feel free to contact me and we can work together to refine this class.

This article is translated by SwiftGG translation team and has been authorized to be translated by the authors. Please visit swift.gg for the latest articles.