SGDownload explore

preface

SGDownload is a file download, very suitable for video download, support background, lock screen download. Simultaneously supports iOS, macOS, tvOS three platforms — source address

Use as follows

// Add the following code to the AppDelegate.
- (void)application:(UIApplication *)application 
    handleEventsForBackgroundURLSession:(NSString *)identifier 
    completionHandler: (void (^)())completionHandler
{
    [SGDownload handleEventsForBackgroundURLSession:identifier 
        completionHandler:completionHandler];
}

// Start the download
self.download = [SGDownload download];
[self.download run];
SGDownloadTask * task = [self.download taskWithContentURL:contentURL];
if(! Task) {task = [SGDownloadTask taskWithTitle:@ "title"contentURL:contentURL
                                 fileURL:fileURL];
}
[self.download addDownloadTask:task]
Copy the code

May read the above code, may be confused, how to run up? In addition, how to download the task directly addDownloadTask to go, do not actively call download?

The actual author uses NSCondition to implement this function in a producer-consumer pattern

It can be seen that the author has a very deep understanding of locks, which is worth our learning

Let’s explore this framework with questions

Introduction to the

SGDownLoad has seven main feature classes, , respectively, SGDownloadImp, SGDownloadTask, SGDownloadTaskQueue, SGDownloadTuple, SGDownloadTupleQueue, SGDownloadTaskCorrect, SGDownloadTo ols

SGDownloadImp: this mainly contains the SGDownload class, which coordinates several other classes and starts a resident thread (blocked when idle) to coordinate task execution, cancellation, etc

SGDownloadTask, SGDownloadTaskQueue: SGDownloadTask is the description of the request file class, including the request URL, title, file address and other information, support archiving, unarchiving, used for resumable breakpoint function, and SGDownloadTaskQueue is the management class control SGDownloadTask. NSCondition controls the timing of task execution (producer-consumer mode). In addition to controlling the addition and deletion of SGDownloadTask, SGDownloadTask can also be used to read and delete files cached by SGDownloadTask

SGDownloadTuple, SGDownloadTupleQueue: SGDownloadTuple is the combination of SGDownloadTask and NSURLSessionDownloadTask, mainly used to coordinate file description and request task information, to ensure that the description file can be directly linked to the network request. SGDownloadTupleQueue is a management class that manages SGDownloadTuple. It can be added, deleted, or cancelled

SGDownloadTaskCorrect: mainly used to correct the problem that some versions generate tasks

SGDownloadTools: File manipulation class

The general structure is shown below

The source code to explore

SGDownloadTask

This contains the necessary information for the download, as well as the resumeInfoData information in the event of a download failure to resume the download

This class implements NSObject archiving protocol, mainly used for task reply, has failed files

You can use this information to cancel downloaded tasks and delete downloaded files (both completed and unfinished).

@property (nonatomic, assign, readonly) SGDownloadTaskState state;

@property (nonatomic, copy, readonly) NSURL * contentURL;
@property (nonatomic, copy, readonly) NSString * title;
@property (nonatomic, copy, readonly) NSURL * fileURL;

@property (nonatomic, assign, readonly) BOOL fileDidRemoved;
@property (nonatomic, assign, readonly) BOOL fileIsValid;

@property (nonatomic, assign) BOOL replaceHomeDirectoryIfNeed;      // default is YES;

@property (nonatomic, assign, readonly) float progress;
@property (nonatomic, assign, readonly) int64_t bytesWritten;
@property (nonatomic, assign, readonly) int64_t totalBytesWritten;
@property (nonatomic, assign, readonly) int64_t totalBytesExpectedToWrite;

// about resume
@property (nonatomic, strong, readonly) NSData * resumeInfoData;
@property (nonatomic, assign, readonly) int64_t resumeFileOffset;
@property (nonatomic, assign, readonly) int64_t resumeExpectedTotalBytes;
Copy the code

Note also the state parameter of this class, which indicates the state of the download and can be used according to its state

typedef NS_ENUM(NSUInteger, SGDownloadTaskState)
{
    SGDownloadTaskStateNone, // No operation is performed by default
    SGDownloadTaskStateWaiting, // Tasks are queued for execution
    SGDownloadTaskStateRunning, // He is blind
    SGDownloadTaskStateSuspend, // The task was started, but then suspended
    SGDownloadTaskStateFinished, // The task is complete
    SGDownloadTaskStateCanceled, // The task was cancelled
    SGDownloadTaskStateFailured, // Task failed
};
Copy the code

SGDownloadTaskQueue

This class is used to manage SGDownloadTask objects, including adding and deleting, archiving and unarchiving

During initialization, unarchiving is performed to recover unfinished tasks. You can delete unfinished tasks if you do not want to

+ (instancetype)queueWithDownload:(SGDownload *)download
{
    return [[self alloc] initWithDownload:download];
}

- (instancetype)initWithDownload:(SGDownload *)download
{
    if (self = [super init]) {
        self->_download = download;
        self->_archiverPath = [SGDownloadTools
            archiverFilePathWithIdentifier:download.identifier];
        // Get the task set
        self->_tasks = [NSKeyedUnarchiver unarchiveObjectWithFile:self.archiverPath];
        if(! self->_tasks) { self->_tasks = [NSMutableArray array]; } self.condition = [[NSCondition alloc] init]; [self resetQueue]; }return self;
}
Copy the code

Here are the core functions of taskQueue, the add and remove functions, which add and execute tasks

Add any and execute tasks set to producer-consumer mode

When executing a task: Iterate over the task set to find the default state or waiting task, and continue to execute the task. Otherwise, NSCondition’s wait function blocks the current thread

When adding a task: If the task is in the pending, cancelled, default, or failed state, the state is changed to wait for execution. Then, NSCondition’s wait function is used to wake up the current thread. In an SGDownload custom thread)

Producer-consumer mode, can refer to the ios common several locks, which are described in the inside

In short, the consumer goes through a loop, gets the execution of the task, blocks when there is no task, and when the producer adds the task, wakes up the thread of the consumer to continue the previous loop and continue the execution of the task

The code looks like this:

/ / consumer
- (SGDownloadTask *)downloadTaskSync
{
    if (self.closed) returnnil; Condition lock [self.condition lock]; SGDownloadTask * task;// Start consumer mode, a running loop that blocks the queue until the producer adds a new task and wakes up the current thread
    // Stop running if task is found
    do {
        for (SGDownloadTask * obj in self.tasks) {
            if (self.closed) {
                [self.condition unlock];
                return nil;
            }
            switch (obj.state) {
                case SGDownloadTaskStateNone:
                case SGDownloadTaskStateWaiting:
                    task = obj;
                    break;
                default:
                    break;
            }
            if (task) break;
        }
        If it is empty, the queue has no task, and the current thread is blocked, waiting for the task to wake up, and then continue to fetch
        if(! task) {// It is blocked[self.condition wait]; }}while(! task); [self.condition unlock];return task;
}
Copy the code

/ / producer
- (void)addDownloadTasks:(NSArray <SGDownloadTask *> *)tasks
{
    if (self.closed) return;
    if (tasks.count <= 0) return;
    [self.condition lock];
    BOOL needSignal = NO;
    for (SGDownloadTask * obj in tasks) {
        if(! [self.tasks containsObject:obj]) { obj.download = self.download; [self.tasks addObject:obj]; }switch (obj.state) {
            case SGDownloadTaskStateNone:
            case SGDownloadTaskStateSuspend:
            case SGDownloadTaskStateCanceled:
            case SGDownloadTaskStateFailured:
                obj.state = SGDownloadTaskStateWaiting;
                needSignal = YES;
                break;
            default:
                break; }}// If the task queue already has a task, use signal to wake up the consumer's thread and continue executing
    if (needSignal) {
        // Send a signal to wake up the current
        [self.condition signal];
    }
    [self.condition unlock];
    [self tryArchive];
}
Copy the code

The above two steps will be followed in SGDownload to learn more about the delivery process

The following setTaskState method is called when the task state changes, handles the state, and re-archives the task to ensure that the download progress is saved properly and not lost

- (void)setTaskState:(SGDownloadTask *)task state:(SGDownloadTaskState)state
{
    if(! task)return;
    if (task.state == state) return;
    Lock / / conditions
    [self.condition lock];
    task.state = state;
    [self.condition unlock];
    // Archive once
    [self tryArchive];
}
Copy the code

SGDownloadTuple

A class that associates SGDownloadTask with NSURLSessionDownloadTask. This class has both task description information and download task information, reducing code coupling in the SGDownload class and enabling simultaneous operation of two related functions

@property (nonatomic, strong) SGDownloadTask * downloadTask;
@property (nonatomic, strong) NSURLSessionDownloadTask * sessionTask;
Copy the code

SGDownloadTupleQueue

It is mainly used to add SGDownloadTuple, delete, etc., including the cancellation of tasks, etc., because it is relatively simple, I will not introduce more

- (void)addTuple:(SGDownloadTuple *)tuple;
- (void)removeTupleWithSesstionTask:(NSURLSessionTask *)sessionTask;
- (void)removeTuple:(SGDownloadTuple *)tuple;
- (void)removeTuples:(NSArray <SGDownloadTuple *> *)tuples;

- (void)cancelDownloadTask:(SGDownloadTask *)downloadTask resume:(BOOL)resume completionHandler:(void(^)(SGDownloadTuple * tuple))completionHandler;
- (void)cancelDownloadTasks:(NSArray <SGDownloadTask *> *)downloadTasks resume:(BOOL)resume completionHandler:(void(^)(NSArray <SGDownloadTuple *> * tuples))completionHandler;

- (void)cancelAllTupleResume:(BOOL)resume completionHandler:(void(^)(NSArray <SGDownloadTuple *> * tuples))completionHandler;
- (void)cancelTuple:(SGDownloadTuple *)tuple resume:(BOOL)resume completionHandler:(void(^)(SGDownloadTuple * tuple))completionHandler;
- (void)cancelTuples:(NSArray <SGDownloadTuple *> *)tuples resume:(BOOL)resume completionHandler:(void(^)(NSArray <SGDownloadTuple *> * tuples))completionHandler;
Copy the code

SGDownload

The core class of this functionality coordinates the processing of SGDownloadTupleQueue, SGDownloadTaskQueue, and download callback results

Note that this class has all the required functions, and it basically has all the functions in the coordinated class, as far as possible to avoid the tedious call process, as shown below

- (nullable SGDownloadTask *)taskForContentURL:(NSURL *)contentURL;
- (nullable NSArray <SGDownloadTask *> *)tasksForAll;
- (nullable NSArray <SGDownloadTask *> *)tasksForRunningOrWatting;
- (nullable NSArray <SGDownloadTask *> *)tasksForState:(SGDownloadTaskState)state;


- (void)addDownloadTask:(SGDownloadTask *)task;
- (void)addDownloadTasks:(NSArray <SGDownloadTask *> *)tasks;

- (void)addSuppendTask:(SGDownloadTask *)task;
- (void)addSuppendTasks:(NSArray <SGDownloadTask *> *)tasks;

- (void)resumeAllTasks;
- (void)resumeTask:(SGDownloadTask *)task;
- (void)resumeTasks:(NSArray <SGDownloadTask *> *)tasks;

- (void)suspendAllTasks;
- (void)suspendTask:(SGDownloadTask *)task;
- (void)suspendTasks:(NSArray <SGDownloadTask *> *)tasks;

- (void)cancelAllTasks;
- (void)cancelTask:(SGDownloadTask *)task;
- (void)cancelTasks:(NSArray <SGDownloadTask *> *)tasks;

- (void)cancelAllTasksAndDeleteFiles;
- (void)cancelTaskAndDeleteFile:(SGDownloadTask *)task;
- (void)cancelTasksAndDeleteFiles:(NSArray <SGDownloadTask *> *)tasks;
Copy the code

When SGDownload is initialized, it is threaded in the background to ensure that users can continue downloading while the screen is locked and in the background

- (instancetype)initWithIdentifier:(NSString *)identifier
{
    if (self = [super init]) {
        / / logo
        self->_identifier = identifier;
        // Start background threads
        self->_sessionConfiguration = [NSURLSessionConfiguration 
            backgroundSessionConfigurationWithIdentifier:identifier];
        self.maxConcurrentOperationCount = 1;
        // Task queue
        self.taskQueue = [SGDownloadTaskQueue queueWithDownload:self];
        // Tuple queue
        self.taskTupleQueue = [[SGDownloadTupleQueue alloc] init];
    }
    return self;
}
Copy the code

When the code is called, it is found that the run function is called first, and then the automatic execution of the task is added, with questions, to refer to the following code

- (void)run
{
    if (!self.running) {
        self.running = YES;
        [self setupOperation];
    }
}
- (void)setupOperation
{
    if (self.maxConcurrentOperationCount <= 0) {
        self.maxConcurrentOperationCount = 1;
    }
    // Initialize the concurrent conditional lock
    self.concurrentCondition = [[NSCondition alloc] init];
    self.lastResumeLock = [[NSLock alloc] init];
    
    // Callback queue, asynchronous serial queue, proxy callback
    self.sessionDelegateQueue = [[NSOperationQueue alloc] init];
    self.sessionDelegateQueue.maxConcurrentOperationCount = 1;
    // Highest priority, mainly used to provide interactive UI operations, such as handling click events and drawing images to the screen
    self.sessionDelegateQueue.qualityOfService = NSQualityOfServiceUserInteractive;
    self.sessionDelegateQueue.suspended = YES;
    
    [self.lastResumeLock lock];
    self.session = [NSURLSession sessionWithConfiguration:self.sessionConfiguration
                                                 delegate:self
                                            delegateQueue:self.sessionDelegateQueue];
    Ivar ivar = class_getInstanceVariable(NSClassFromString(@"__NSURLBackgroundSession"), "_tasks");
    if (ivar) {
        NSDictionary <NSNumber *, NSURLSessionDownloadTask *> * lastTasks =
            object_getIvar(self.session, ivar);
        if (lastTasks && lastTasks.count > 0) {
            for (NSNumber * key in lastTasks) {
                //NSURLSession Download task
                NSURLSessionDownloadTask * obj = [lastTasks objectForKey:key];
                // Create downloadTask for caching
                SGDownloadTask * downloadTask = [self.taskQueue 
                    taskForContentURL:[self getURLFromSessionTask:obj]];
                
                if (downloadTask) {
                    // Change the state
                    [self.taskQueue setTaskState:downloadTask state:SGDownloadTaskStateRunning];
                    SGDownloadTuple * tuple = [SGDownloadTuple 
                        tupleWithDownloadTask:downloadTask sessionTask:obj];
                    [self.taskTupleQueue addTuple:tuple];
                }
            }
        }
    }
    [self.lastResumeLock unlock];
    self.sessionDelegateQueue.suspended = NO;
    // Create a download task NSInvocationOperation
    self.downloadOperation = [[NSInvocationOperation alloc] 
        initWithTarget:self selector:@selector(downloadOperationHandler) object:nil];
    // Download the queue
    self.downloadOperationQueue = [[NSOperationQueue alloc] init];
    // Serial queue
    self.downloadOperationQueue.maxConcurrentOperationCount = 1;
    self.downloadOperationQueue.qualityOfService = NSQualityOfServiceUserInteractive;
   [self.downloadOperationQueue addOperation:self.downloadOperation];
}
Copy the code

When running the task, will perform the thread below, the following look closely, there will be a lot of details, first by resident loop setting thread (resident everybody thought thread need through the runloop at ordinary times, this is one-sided, child threads after open, will take the initiative to perform a task, if the task is done, did not take the initiative to open the child thread runloop, The thread will be automatically destroyed after the task is executed. After the runloop is started, the task will not be destroyed because the task cannot be completed. If the task is a loop itself, the task will not be destroyed, which means the loop is a runloop.

The following is a custom run loop that uses the consumer mode to get the task specification, as shown below

- (void)downloadOperationHandler
{
    // Permanent thread, block, execute, wait (self.concurrentCondition, conditional lock check)
    // Consumer role -- downLoadTask
    while (YES) {
        @autoreleasepool
        {
            if (self.closed) {
                break;
            }
            NSLog(@"current Thread -- %@",[NSThread currentThread]);
            // Condition lock
            [self.concurrentCondition lock];
            // Maximum number of concurrent requests
            while (self.taskTupleQueue.tuples.count >= self.maxConcurrentOperationCount) {
                NSLog(@"current wait Thread -- %@",[NSThread currentThread]);
                [self.concurrentCondition wait];
            }
            [self.concurrentCondition unlock];
            // Get tasks from taskQueue (downloadTaskSync is the consumer of taskQueue)
            The taskQueue will block in the downloadTaskSync method until the taskQueue gets the task
            // When the producer function in taskQueue is called through the SGDownload profile, it is unblocked and continues
            SGDownloadTask * downloadTask = [self.taskQueue downloadTaskSync];
            // Avoid the user forcing the loop to end
            if(! downloadTask) {break;
            }
            // Change the download state to download state
            [self.taskQueue setTaskState:downloadTask state:SGDownloadTaskStateRunning];
            
            / / create NSURLSessionDownloadTask
            NSURLSessionDownloadTask * sessionTask = nil;
            if (downloadTask.resumeInfoData.length > 0) {
                // Restore download progress according to reusmeInfoData (when the status changes, the task is archived to disk)
                sessionTask = [SGDownloadTaskCorrect 
                    downloadTaskWithSession:self.session resumeData:downloadTask.resumeInfoData];
            } else {
                sessionTask = [self.session downloadTaskWithURL:downloadTask.contentURL];
            }
            // Generate a new SGDownloadTuple and place it in the tupleQueue
            SGDownloadTuple * tuple = [SGDownloadTuple 
                tupleWithDownloadTask:downloadTask sessionTask:sessionTask]; [self.taskTupleQueue addTuple:tuple]; [sessionTask resume]; }}}Copy the code

A callback after a single file has been downloaded

- (void)URLSession:(NSURLSession *)session 
    task:(NSURLSessionTask *)sessionTask 
    didCompleteWithError:(NSError *)error
{
    // This lock ensures that several download modules, callbacks specifying the cause of the problem at the same time, or actively secure data downloaded by multiple threads at the same time
    [self.lastResumeLock lock];
    // Conditional lock to ensure data security when processing tasks such as taskTupleQueue
    [self.concurrentCondition lock];
    // Get the task, then update the status of tupleQueue, taskQueue, etc
    SGDownloadTask * downloadTask = [self.taskQueue taskForContentURL:
        [self getURLFromSessionTask:sessionTask]];
    SGDownloadTuple * tuple = [self.taskTupleQueue tupleWithDownloadTask:downloadTask 
        sessionTask:(NSURLSessionDownloadTask *)sessionTask];
    if(! tuple) { [self.taskTupleQueue removeTupleWithSesstionTask:sessionTask];//signal -- wakes up the current thread, and the currently waiting task continues
        [self.concurrentCondition signal];
        [self.concurrentCondition unlock];
        [self.lastResumeLock unlock];
        return;
    }
    
    // updates the status of the downloadTask. If the downloadTask succeeds, it has no value. If the downloadTask fails, resumeInfoData needs to be updated
    // To resume the download
    SGDownloadTaskState state;
    if (error) {
        NSData * resumeData = [error.userInfo objectForKey:NSURLSessionDownloadTaskResumeData];
        if (resumeData) {
            tuple.downloadTask.resumeInfoData = resumeData;
        }
        if (error.code == NSURLErrorCancelled) {
            state = SGDownloadTaskStateSuspend;
        } else{ tuple.downloadTask.error = error; state = SGDownloadTaskStateFailured; }}else {
        if(! [[NSFileManager defaultManager] fileExistsAtPath:tuple.downloadTask.fileURL.path]) { tuple.downloadTask.error = [NSErrorerrorWithDomain: @"download file is deleted" code:-1 userInfo:nil];
            state = SGDownloadTaskStateFailured;
        } else{ state = SGDownloadTaskStateFinished; }}// Updates the status of tupl. downloadTask to notify archived data
    [self.taskQueue setTaskState:tuple.downloadTask state:state];
    [self.taskTupleQueue removeTuple:tuple];
    if ([self.taskQueue tasksForRunningOrWatting].count <= 0 &&
        self.taskTupleQueue.tuples.count <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            if ([self.delegate 
                respondsToSelector:@selector(downloadDidCompleteAllRunningTasks:)]) { [self.delegate downloadDidCompleteAllRunningTasks:self]; }}); } [self.concurrentCondition signal]; [self.concurrentCondition unlock]; [self.lastResumeLock unlock]; }Copy the code

A callback during download to update download information in a timely manner

- (void)URLSession:(NSURLSession *)session 
    downloadTask:(NSURLSessionDownloadTask *)sessionTask 
    didWriteData:(int64_t)bytesWritten 
    totalBytesWritten:(int64_t)totalBytesWritten 
    totalBytesExpectedToWrite:(int64_t)totalBytesExpectedToWrite
{
    // Object lock ensures secure access to the downloadTask
    [self.lastResumeLock lock];
    // The user updates the download task information
    SGDownloadTask * downloadTask = [self.taskQueue 
        taskForContentURL:[self getURLFromSessionTask:sessionTask]];
    SGDownloadTuple * tuple = [self.taskTupleQueue 
        tupleWithDownloadTask:downloadTask sessionTask:(NSURLSessionDownloadTask *)sessionTask];
    if(! tuple) { [self.lastResumeLock unlock];return; } update download task information [tuple downloadTask setBytesWritten: bytesWrittentotalBytesWritten:totalBytesWritten
              totalBytesExpectedToWrite:totalBytesExpectedToWrite];
     // This step updates the status of the task and performs an archive to ensure that the data can be successfully recovered even if the data fails
    if(tuple.downloadTask.state ! = SGDownloadTaskStateSuspend) { [self.taskQueue setTaskState:tuple.downloadTask state:SGDownloadTaskStateRunning]; } [self.lastResumeLock unlock]; }Copy the code

The last

SGDownLoad makes full use of locks and semaphores (semaphores are also included in common locks) to take advantage of the producer-consumer model and implement the function of adding tasks. The author’s understanding of locks is well worth learning

This pattern is also covered in the chapter on database transactions and other operations, so it is not just used at the front end

SGDownload is introduced here, want to study carefully can see the source code to continue, address has been given, welcome to discuss