preface
Simple demo records
This article thought
The ideas of this paper are as follows:
- Large file slicing.
- Example Set the maximum number of concurrent uploads.
- According to the
YYcacheBreakpoint continuation of cached data. - Sets a property to handle notification when all uploads are complete.
implementation
The code is as follows:
- To import the
Pod 'AFNetworking', '~ > 4.0'.YYModel
//
// ViewController.m
// Large file upload
//
// Created by Jg on 7/9/2021
//
#import "ViewController.h"
#import <AFNetworking/AFNetworking.h>
#import "YYCache.h"
@interface ViewController (a)
@property (nonatomic, assign) NSInteger pageCount; // Number of slices
@end
@implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];
// Do any additional setup after loading the view.
[self upload];
}
- (void)upload {
AFHTTPSessionManager *manager = [AFHTTPSessionManager manager];
/ / the serialization
manager.requestSerializer = [AFJSONRequestSerializer serializer];
manager.responseSerializer = [AFJSONResponseSerializer serializer];
// Request header Settings
[manager.requestSerializer setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
NSString *url = @"url";
NSDictionary *parameter = @{};
NSString *filePath = @"File path";
unsigned long long pageSize = 1 << 20;//1 MB, fragment size
unsigned long long size = fileSizeAtPath(filePath); // File size
int count = ceil(size/pageSize * 1.0); // round up
// That data is in the cache
NSMutableDictionary *uploadDic = @{}.mutableCopy;
YYCache *cache = [[YYCache alloc] initWithName:@"netUpload"];
if ([cache objectForKey:@"netData"]) {
// Get data from the cache
uploadDic = (NSMutableDictionary *)[cache objectForKey:@"netData"];
// Set the number of slices
self.pageCount = uploadDic.allKeys.count;
} else {
@autoreleasepool {
for (int i = 0; i < count; i++) {
NSString *key = [NSString stringWithFormat:@"%d", i]; [uploadDic setObject:key forKey:key]; }}// Set the number of slices
self.pageCount = count;
[cache setObject:uploadDic forKey:@"netData"];
}
dispatch_queue_t queue = dispatch_queue_create(0, DISPATCH_QUEUE_CONCURRENT);
dispatch_semaphore_t semp = dispatch_semaphore_create(5);
dispatch_async(queue, ^{
NSMutableArray *indexArr = uploadDic.allKeys.mutableCopy;
for (int i = 0; i < indexArr.count; i++) {
dispatch_semaphore_wait(semp, DISPATCH_TIME_FOREVER);
[manager POST:url parameters:parameter headers:nil constructingBodyWithBlock:^(id<AFMultipartFormData> _Nonnull formData) {
// Get the file offset
int index = [indexArr[i] intValue];
// File fragmentation
NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:filePath];
// Fetch data according to offset address
//1. Set the offset address
[handle seekToFileOffset:(pageSize * index)];
//2. Set the read size and get the slice data
NSData *blockData = [handle readDataOfLength:pageSize];
//3. Upload fragmented data
[formData appendPartWithFileData:blockData name:@"block" fileName:filePath.lastPathComponent mimeType:@"video/mp4"];
} progress:^(NSProgress * _Nonnull uploadProgress) {
} success:^(NSURLSessionDataTask * _Nonnull task, id _Nullable responseObject) {
// Upload succeeded
// 1. Process data
[uploadDic removeObjectForKey:indexArr[i]];
@synchronized (self) {
self.pageCount -= 1;
}
dispatch_async(dispatch_get_main_queue(), ^ {// 2. Main thread updates UI progress
});
dispatch_semaphore_signal(semp);
} failure:^(NSURLSessionDataTask * _Nullable task, NSError * _Nonnull error) {
// Upload failed
// Set up your own failure mechanism, handle failure messages, see whether to re-upload or do other user prompts.
// According to the data corresponding to the cache netData, you can determine the transmission problem.
dispatch_semaphore_signal(semp); }]; }}); } - (void)setPageCount:(NSInteger)pageCount {
_pageCount = pageCount;
if (pageCount == 0) {
// Handle all loading completed operations}}long long fileSizeAtPath(NSString* filePath){
NSFileManager* manager = [NSFileManager defaultManager];
if ([manager fileExistsAtPath:filePath]){
return [[manager attributesOfItemAtPath:filePath error:nil] fileSize];
}
return 0;
}
@end
Copy the codeconclusion
If it is a small file, it can be converted into NSData and operated in memory. If it is a large file of several gigabytes, it can only slice the file bit by bit by offset address using NSFileHandle.
Other ideas
Here’s another idea:
-
The data to be uploaded will create a new folder in Document with the file hash name.
-
Then use NSFileHandle to slice the file and create a new file for each slice in the folder. Then the data is stored in the file and the name can be named according to the hash value. However, the server needs to be informed of the corresponding sequence of concatenation during upload.
-
When you upload, you just read the folder and get the array.
- Upload the file and delete it after success
-
Finally, according to whether the folder is empty, you can judge whether the transmission is complete.
In this way, the program interrupt or other next startup directly find the corresponding folder, do not need to do other processing. However, parsing files into corresponding folders and then sharding them is also very expensive I/O.
conclusion
Issues needing attention:
- Large files cannot be directly loaded into the memory. Otherwise, the memory will overflow.
- through
NSFileHandleClass to handle
- through
- There are two things to tell the server when uploading asynchronously:
- Splicing mode of file slices
- Notification of completion of file transfer.
- Child thread uploads, main thread refreshes UI.
- Control concurrency.