If there is no basic playback control function during the video playing, it is very bad that the video can only play normally from the beginning to the end. This section addresses this seemingly simple but very important problem.

✅ section 1 – Hello FFmpeg ✅ section 2 – Soft unclip video stream, render RGB24 ✅ section 3 – understand YUV ✅ section 4 – hard decoding,OpenGL render YUV Disk section 5 – Metal render YUV disk section 6 – Decode audio, play with AudioQueue ✅ section 7 – audio and video synchronization 🔔 Section 8 – Perfect playback control 📗 Section 9 – double speed play 📗 section 10 – increase video filtering effect section 11 – audio changes

The Demo address of this section is github.com/czqasngit/f… The example code provides both Objective-C and Swift implementations. This article cites Objective-C code for ease of illustration, since Swift code Pointers don’t look neat. The final effect of this section is shown below:

The target

  • Implement play and pause
  • Achieve forward and backward
  • Drag the slider to change the playback time

Implement play and pause

Audio playback and image rendering are separated, and it is necessary to control audio playback and image rendering separately. Audio uses AudioQueu, so its pause and play control code is as follows:

- (void)pause { AudioQueuePause(audioQueue); NSLog(@" pause "); } - (void)resume { AudioQueueStart(audioQueue, NULL); NSLog(@" restore "); }Copy the code

Video is controlled by timer, so its pause and play control code is as follows:

- (void)startVideoRender {
    if(self->video_render_timer) {
        dispatch_source_cancel(self->video_render_timer);
    }
    self->video_render_timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, video_render_dispatch_queue);
    float duration = 1.0 / self.fps * NSEC_PER_SEC;
    dispatch_source_set_timer(self->video_render_timer, DISPATCH_TIME_NOW, duration, duration);
    dispatch_source_set_event_handler(self->video_render_timer, ^{
        [self playNextVideoFrame];
    });
    dispatch_resume(self->video_render_timer);
}

- (void)stopVideoRender {
    if(self->video_render_timer) dispatch_cancel(self->video_render_timer);
    self->video_render_timer = NULL;
}
#pragma mark - Public
- (void)start {
    [self startVideoRender];
}
- (void)stop {
    [self stopVideoRender];
}
- (void)pause {
    [self stop];
}
- (void)resume {
    [self start];
}
Copy the code

Pause and play are relatively simple to implement

Achieve forward and backward

Before realizing forward and backward, we need to realize the play seek function, that is, set the playing time of the video. Fortunately, ffMPEG already has this functionality built in, but there are a few details that require special attention and attention

Achieve seek to a specific point in time function

Ffmpeg provides the functionality of seek, which is defined as follows:

int av_seek_frame(AVFormatContext *s, 
                  int stream_index, 
                  int64_t timestamp,
                  int flags)
Copy the code

Stream_index: the target AVStream to operate on. If set to -1, ffmPEG automatically selects a default object timestamp: Time_base flags sets the time point to seek in avstream. time_base flags: the enumeration is defined as follows

#define AVSEEK_FLAG_BACKWARD 1 ///< look for keyframes forward at the set time point as the final set target time point #define AVSEEK_FLAG_BYTE 2 //< set based on bytes #define AVSEEK_FLAG_ANY 4 ///< set to any frame, if the frame is not a key frame will be discarded backtracking keyframe #define AVSEEK_FLAG_FRAME 8 //< set to any frame, if the frame is not a key frame will be discarded backtracking keyframeCopy the code

Example code for setting if:

av_seek_frame(self->formatContext, -1, time * AV_TIME_BASE, AVSEEK_FLAG_BACKWARD);
Copy the code

The complete seek process looks like this:

  • 1. Pause the player
  • 2. Clear the buffered data in the audio and image playback queue
  • 3. Clear the cached data in the audio player AudioQueue
  • 4. Clear the decoded buffer data in FFMPEG
  • 5. Call av_seek_frame
  • 6. Wake up the decoding thread to continue decoding
  • 7. Resume the player

The complete example code is as follows:

- (void)seekTo:(float)time {
    [self pause];
    [self.audioFrameCacheQueue clean];
    [self.videoFrameCacheQueue clean];
    pthread_mutex_lock(&(self->mutex));
    avcodec_flush_buffers([_mediaVideoContext codecContext]);
    avcodec_flush_buffers([_mediaAudioContext codecContext]);
    [self.audioPlayer cleanQueueCacheData];
    av_seek_frame(self->formatContext, -1, time * AV_TIME_BASE, AVSEEK_FLAG_BACKWARD);
    _NotifyWaitThreadWakeUp(self.decodeCondition);
    pthread_mutex_unlock(&(self->mutex));
    [self resume];
}
Copy the code

Achieve forward and backward

It’s easy to move forward and backward in Seek’s functionality:

/// Back [self seekTo: self.currentTime-3]; [self seekTo:self.currentTime + 3];Copy the code

Drag the slider to change the playback time

Drag is also based on the seek function, which needs to be noted here is the click to slide when the playback needs to pause, in AppKit using NSSlider to achieve this function needs to be implemented separately

@interface _NSSlider : NSSlider @property (nonatomic, strong)void(^mouseDownBlock)(void); @end@implementation _NSSlider - (void)mouseDown:(NSEvent *)event {NSLog(@" slide: mouseDown "); self.mouseDownBlock(); [super mouseDown:event]; } @endCopy the code

_NSSlider continuous value set to false to disable continuous event callback during dragging except for the last callback after the last mouse bounce. Set maxValue and minValue for _NSSlider in the playerReadyToPlay callback. By setting the action and target of _NSSlider, we can get the specific time point we need to seek when we call back.

 [self seekTo:sender.floatValue];
Copy the code

At this point, the basic video control function has been implemented, and the specific implementation code can be found in the corresponding tag of git repo. It’s much easier to implement it on the iOS UI. AppKit’s interactive implementation is not very friendly.

Conclusion:

  • Realize play and pause, through the audio play and image rendering respectively pause and play control.
  • Realize forward and backward, easy to realize forward and fast backward on the basis of seek.
  • Drag slider to change the playback time point, in the implementation of seek based on the easy implementation of drag function, but need special attentionNSSliderThe implementation of the.