- Custom edit mode
- Template edit mode
Custom edit mode, which contains a lot of features, most of the implementation can find a single example, the difficulty is how to put all these features together, how to edit, how to preview.
- Video stitching
- Video clipping
- Video sequence
- Video segmentation
- Video rotation
- Video transmission
- Delete the video
- Adjust the format
- Adjust the volume
- Picture in picture
- A filter
- Video transitions
- Video fall seeding
- The background music
- The music
- subtitles
- Save drafts
- Redo function
- Export video
Template editing mode, that is, to provide users with “template video”, users only need to select a video or picture, then can create the same video with the same editing effects as the “template video”, to achieve “one-click editing”.
1. Technology selection
On iOS, there are a number of technologies that enable the above custom editing capabilities. Such as AVFoundation, GPUImage, FFMPEG, this article is mainly based on AVFoundation to achieve.
Key points to consider:
1. Use AVFoundation for easy preview, and use AVPlayer for edit preview
2. The performance of AVFoundation is guaranteed, the follow-up optimization is easy, the interface is perfect, and the expansion is convenient
Basic editing functions such as concatenation, sorting, clipping, and so on are easily implemented using AVMutableComposition. How to achieve transitions, filters, paintings in pictures? Using AVMutableVideoComposition can specify video rendering parameters such as size, scaling and frame rate, animated transitions can be realized by adding AVVideoCompositionLayerInstruction instructions. The filter has to be done on a per-frame basis, so I used AVVideoCompositing for that. We can get every frame, and if we have multiple tracks, we can get frames from multiple tracks at the same time, and we can do transitions, filters, picture-in-picture.
2. Frame design
Video editing is mainly divided into the following modules
- Resource cache (cache current video editing resources, including video paths, pictures, music, recordings, video frames)
- Edit description (description of the video editing operation, which can be exported as JSON file or restored from JSON file to edit state)
- Preview builds (build video previews, process and edit descriptions to form playable video resources)
- Video Preview (Play preview build resources)
- Export build (build video export, process edit description to form video resources that can be exported)
- UI
The picture below shows the overall process
In video editing, each edit is a modification to the edit description module, and all subsequent previews and exports depend on parsing the edit description. From the video editing description we can generate avmutlecomposition for playback and export, so how do we describe video editing?
First, let’s look at what the resource mix looks like after the video is edited:
- Timeline: First, there is the concept of a timeline. All of our resources are located in a period of time. Every resource is located in the timeline, and all operations are located in a period of time. The length of the timeline also represents the length of the final video composition.
If you are looking for a new job or are looking for a new job, add our communication group 101 295 1431 to get a detailed interview data for your job hopping to add a guarantee.
- Track: The container with the time line as the coordinate system. The contents and editing functions required by each time point are stored in the container, which can be divided into video track, audio track, picture-in-picture track and text track
Then we describe the editing model of the above figure
-
We create a 100s video with five tracks. The video track contains the description of the transition and the description of the filter
-
The video track is divided into two parts, one is basic video and the other is picture-in-picture video
- Basic video: contains 5 video segments, respectively located in two video tracks, type is VIDEO, respectively in [0–30], [42–55], [75–100], [25–45], [60–78]
- Picture-in-picture: Contains only one video segment, type IS PIP, location is [8–92]
-
Image track: Contains two images, type image, positions: [28–45], [62–90]
-
Audio track: The example contains only one video track and one video segment, and the position is [0-100]. In practice, there can be multiple tracks and multiple video segments just like the video
-
Description of transitions and filters: Here you can also think of transitions as full tracks, with type and location descriptions, just like a video clip
With the above description, we can translate the description into the corresponding code:
Time line:
Let’s create a timeline description class, FXTimelineDescribe, which will have the following properties for each track description file
CMTime duration; NSMutableArray<FXVideoDescribe *> *videoArray; NSMutableArray<FXPIPVideoDescribe *> *pipVideoArray; NSMutableArray<FXAudioDescribe *> *audioArray; NSMutableArray<FXTransitionDescribe *> *transitionArray; NSMutableArray<FXMusicDescribe *>*musicArray; // Music description NSMutableArray *filterArray; // Filter description NSMutableArray *titleArray; // Subtitle description NSMutableArray *overlayArray; // Watermark descriptionCopy the codeOrbit description:
Depending on the type, we create different types of track description:
Typedef NS_ENUM(NSUInteger, FXDescribeType) {FXDescribeTypeNone, FXDescribeTypeVideo, FXDescribeTypeAudio, / / audio FXDescribeTypeTransition, / / ferry FXDescribeTypeTitle, / / subtitle FXDescribeTypePip, / / picture in picture FXDescribeTypeMusic, // describe the description of music fxdescribbetyperecord // dub};Copy the codeThen pull out the same part of the track description as the base class for the track content description: FXDescribe
@interface FXDescribe : NSObject @property (nonatomic, assign) CMTime startTime; @property (nonatomic, assign) CMTime duration; @property (nonatomic, assign) CMTimeRange sourceRange; @property (nonatomic, assign) CGFloat Scale; @property (nonatomic, assign) FXDescribeType desType; // Type - (NSDictionary *)objectDictionary; - (void)setObjectWithDic:(NSDictionary *)dic; @endCopy the codeA sourceRange property is added that describes the current location of the resource in the source resource, such as a video clip that is part of a video.
There are two methods that need to be overridden by subclasses
1, (NSDictionary *) objectDictionary;
2, (void) setObjectWithDic (dic NSDictionary *);
1, (NSDictionary *) objectDictionary;
To convert the description to a dictionary, and then combine it to a description JSON, so subclasses need to override this method, and then add the properties that the subclasses have added so if you’re jumping ship or you’re about to jump ship you can do that, Add our communication group 101 295 1431 to get a detailed interview information for your job-hopping to add a guarantee.
2, (void) setObjectWithDic: (dic NSDictionary *)
This part is equivalent to digital-to-analog conversion. Automatic digital-to-analog conversion (such as YYModel) is not used here, because some special data need to be processed and some resources need to be searched.
Let’s take a look at how the video of the video track describes it:
We created FXVideoDescribe, inherited from: FXDescribe, and we needed to add some description properties specific to the video
@property (nonatomic, assign) NSInteger videoIndex; @property (nonatomic, assign) BOOL reverse; @property (nonatomic, readonly) rotate FXRotation; @property (nonatomic, strong) FXVideoItem *videoItem; // Attribute (nonatomic, assign) BOOL mute; // Attribute (nonatomic, assign) BOOL mute; // Whether to muteCopy the codeSimilar to video descriptions, other types of descriptions add attributes that they need.
Finally convert the timeline into a dictionary :(we add two videos, add a transition effect, and then split the first video into two) let’s look at the generated description dictionary
{ audioTrack = ( ); defaultNaturalSizeHeight = 1080; defaultNaturalSizeWidth = 608; Duration = "30.182"; lengthTimeScale = 30; mainVideoVolume = 100; pipVideoVolume = 100; transitionTrack = ( { backVideoIndex = 1; desType = 3; duration = 2; preVideoIndex = 0; scale = 1; sourceRangeDuration = nan; sourceRangeStart = nan; startTime = 0; transType = 1; }); videoTrack = ( { desType = 1; Duration = "5.471666666666667"; filterType = 2; mute = 0; reverse = 0; rotate = 0; scale = 1; SourceRangeDuration = "5.471666666666667"; sourceRangeStart = 0; startTime = 0; videoIndex = 0; videoItem = "file:///var/mobile/Media/PhotoData/CPLAssets/group115/480C781E-4B41-48A3-B367-484F5C693464.MP4"; }, { desType = 1; Duration = "8.389333333333333"; filterType = 2; mute = 0; reverse = 0; rotate = 0; scale = 1; SourceRangeDuration = "8.389333333333333"; SourceRangeStart = "5.471666666666667"; StartTime = "3.471666666666667"; videoIndex = 1; videoItem = "file:///var/mobile/Media/PhotoData/CPLAssets/group115/480C781E-4B41-48A3-B367-484F5C693464.MP4"; }, { desType = 1; Duration = "18.321"; filterType = 2; mute = 0; reverse = 0; rotate = 0; scale = 1; SourceRangeDuration = "18.321"; sourceRangeStart = 0; StartTime = "11.861"; videoIndex = 2; videoItem = "file:///var/mobile/Media/PhotoData/CPLAssets/group259/98D7CEA4-69EA-4EB8-924C-FB99DCDBBDD7.MP4"; }); }Copy the codeConvert to a JSON string:
{"pipVideoVolume":"100","mainVideoVolume":"100","transitionTrack":[{"scale":1,"desType":3,"backVideoIndex":"1","sourceRa ngeDuration":"nan","duration":"2","preVideoIndex":"0","startTime":"0","sourceRangeStart":"nan","transType":"1"}],"defaul TNaturalSizeWidth ":" 608 ", "duration" : "30.182", "audioTrack lengthTimeScale" : [], "" :" ", "30 videoTrack" : [{" scale ": 1," reverse ":" 0 "and" rotate ", "0", "desType sourceRangeStart" : 1, "" :" 0 "and" sourceRangeDuration ":" 5.471666666666667 ", "filterType" : "2", "videoInde x":"0","videoItem":"file:\/\/\/var\/mobile\/Media\/PhotoData\/CPLAssets\/group115\/480C781E-4B41-48A3-B367-484F5C693464. MP4, "" duration" : "5.471666666666667", ".mute ":" 0 ", "startTime" : "0"}, {" scale ": 1," reverse ":" 0 "and" rotate ":" 0 "and" desType ": 1," sourc SourceRangeDuration eRangeStart ":" 5.471666666666667 ", "" :" 8.389333333333333 ", "filterType" : "2", "videoIndex" : "1", "videoItem" :"file:\/\/\/var\/mobile\/Media\/PhotoData\/CPLAssets\/group115\/480C781E-4B41-48A3-B367-484F5C693464.MP4","duration":"8 .mute. 389333333333333 ", "" :" 0 ", "startTime" : "3.471666666666667"}, {" scale ": 1," reverse ":" 0 "and" rotate ":" 0 "and" desType ": 1," sourceR SourceRangeDuration angeStart ":" 0 ", "" :" 18.321 ", "filterType" : "2", "videoIndex" : "2", "videoItem" : "file: / / / / / / var/mobile / / / Med Ia \ / PhotoData \ / CPLAssets \ / group259 \ / 98 eb8 ea d7cea4-69-4-924 - c - FB99DCDBBDD7. MP4, "" duration" : "18.321", ".mute ":" 0 ", "startTime" : "11.861"}], "defaultNaturalSizeHeight" : "1080"}Copy the codeAt this point, the description part of our video editing is complete, and we can reverse the corresponding description model according to the description file. Therefore, we can store the description JSON file after each modification as a state of the modification, which can be used for undo and redo operations (you can also use NSUndoManager to do undo redo, but it’s a bit more complicated).
Once we have the edit description, we can build the UI and build the play and export modules based on the edit description. The final structure is shown below
