In order to cope with the complex interaction of video editing tools, Dukakios uses the design idea of Flux architecture mode for reference, refers to the topology concept of directed acyclic graph, centralized management of events, and from the development experience to achieve a comfortable and cool, easy to control “one-way flow” mode; In this scheduling mode, event changes and tracking become clear and predictable, and significantly increase business scalability.

The full text is 6882 words, and the expected reading time is 18 minutes.

1. Architectural background

Video editing tool applications often interact with each other in complex ways. Most operations are carried out on the same main interface, which also has many view areas (preview area, axis area, undo redo area, operation panel, etc.). Each area not only receives user gestures, but also updates the status in linkage with user actions. At the same time, in addition to supporting the main scene editing function, but also support other features, such as universal editing, quick editing, theme templates, etc., all need to use the preview and editing function; Therefore, the scalability and reusable capabilities of the architecture naturally have high requirements.

After investigation, Duka iOS finally drew on the design idea of Flux architecture mode, referred to the topology concept of directed acyclic graph, centralized management of events, and realized a comfortable, refreshing and easy to control “one-way flow” mode from the development experience. In this scheduling mode, event changes and tracking become clear and predictable, and significantly increase business scalability.

Second, playback preview reuse

Doka general editing, as well as many of its derivatives, relies on basic capabilities such as preview and material editing.

For example, the following functions rely on the same set of preview playback logic and need to be abstracted as a base controller.

BaseVC structure is:

Three, function module reuse

Preview playback reuse problem solved, how to add a variety of material editing functions in this logic, such as stickers, text, filter and other functions, and make these functions and VC decoupled, finally achieve the purpose of reuse?

We ended up using the concept of plug and pull design, abstracting each sub-function as a plugin, By directly calling the dependency layer, controller, View, timeline, streamingContext and liveWindow, which are used in 90% of scenarios, are directly assigned to plugin through weak.

Protocol BDTZEditPlugin: NSObjectProtocol {var editViewController: BDTZEditViewController? Var mainView: BDTZEditLevelView? Var mainView: BDTZEditLevelView? {get set} // Edit the timeline entity of the scene. It consists of tracks, which can have multiple video tracks and audio tracks. The length is determined by the video track. Var streamingContext: streamingContext? StreamingContext: streamingContext? {get set} var liveWindow: liveWindow? Func pluginDidLoad() {pluginDidUnload()}Copy the code

Once the protocol is implemented and plugin is added by calling baseVC’s add: method, the corresponding plugin will be called with the corresponding properties, avoiding singletons or layer upon layer callbacks to VC.

func addPlugin(_ plugin: BDTZEditPlugin) { plugin.pluginWillLoad() plugin.editViewController = self plugin.mainView = self.view plugin.liveWindow  = liveWindow plugin.streamingContext = streamingContext plugin.timeline = timeline if plugin.conforms(to: BDTZEditViewControllerDelegate.self) { pluginDispatcher.add(subscriber: plugin as! BDTZEditViewControllerDelegate) } plugin.pluginDidLoad() } func removePugin(_ plugin: BDTZEditPlugin) { plugin.pluginWillUnload() plugin.editViewController = nil plugin.mainView = nil plugin.liveWindow = nil plugin.streamingContext = nil plugin.timeline = nil if plugin.conforms(to: BDTZEditViewControllerDelegate.self) { pluginDispatcher.remove(subscriber: plugin as! BDTZEditViewControllerDelegate) } plugin.pluginDidUnload() }Copy the code

Plugin is an intermediate layer between specific functions and VC. It can accept VC life cycle events, preview playback events, get key objects in VC, and call VC’s internal public interface capabilities. As an independent sub-functional unit inserted in VC, it has editing ability, material ability, network UI interaction and other abilities.

Plugin is divided into service layer and UI layer. At the beginning of design, plugin based on this architecture can not only be used in Duka APP, but also other apps in the factory can access plugin immediately with little work.

All functions can be dispersed into plug-ins, assembled and reused on demand.

Not only a single plugin but also a combination of plugins can be exported simultaneously. Take the cover function as an example. Cover editing is a controller organized by coverVC, which contains multiple plugins, such as the existing text plugin and sticker plugin. In addition to being a standalone feature, coverVC is packaged as a cover Plugin with a small amount of data docking code (the generic clip data docking Plugin shown above) that can be integrated into the generic clip VC and assembled like lego blocks.

Iv. Event status management

Editing tool APP is highly dependent on status update due to the complexity of interaction. Generally speaking, in iOS development, the following ways are generally adopted to notify objects of state changes:

  • Delegate

  • KVO

  • NotificationCenter

  • Block

All four methods manage state changes, but there are some problems. Delegates and blocks, which tend to create strong dependencies between components; KVO and Notifications can create invisible dependencies that can be difficult to detect if important messages are removed or changed, reducing application stability.

Even Apple’s MVC pattern only advocates the separation of the data layer and its presentation layer without providing any tool code or guidance architecture.

4.1 Why is the Flux Architecture Mode selected

So we borrowed the idea of the Flux architecture pattern. Flux is a very lightweight architectural pattern that Facebook uses for its client-side Web applications to bypass MVC and support one-way data flow (also listed below is the front-end MVC data flow diagram). The core idea is centralized control, which allows all requests and changes to be sent only through action and distributed by dispatcher. The advantage is that the View can be kept highly concise, and it doesn’t need to care much about logic, just the incoming data. Centralization also controls all data so that problems can be easily queried and located.

  • Dispatcher: Handles event distribution and maintains dependencies between stores

  • Store: Responsible for storing data and processing data-related logic

  • Action: Triggers Dispatcher

  • View: the View that displays the user interface

As can be seen from the figure above, Flux is characterized by one-way data flow:

  1. The user initiates an Action object to D ispatcher in the View layer

  2. The Dispatcher receives the Action and asks the Store to make changes accordingly

  3. Store makes the corresponding update and then emits a changeEvent

  4. When the View receives a changeEvent, it updates the page

  • Basic MVC data flow

  • Complex MVC data

  • Simple Flux data flow

  • Complex Flux data flows

Compared with MVC mode, Flux has more arrows and ICONS, but there is a key difference: all the arrows point in the same direction, forming an event transmission chain in the whole system.

4.2 Apply Flux ideas to achieve state management

There are two kinds of states:

  • Organize events emitted by the controller to generate state changes, such as controller lifecycle ViewDidLoad(), etc., callbacks to basic edit preview capabilities, such as seek, Progress, playState changes, etc

  • The plugin protocol in the figure below abstracts the Store function in the figure above

The controller holds an EventDispatch capable object Dispatcher and passes events through this Dispatcher.

Dispatcher

class WeakProxy: Equatable { weak var value: AnyObject? init(value: AnyObject) { self.value = value } static func == (lhs: WeakProxy, rhs: WeakProxy) -> Bool { return lhs.value === rhs.value } } open class BDTZActionDispatcher<T>: NSObject { fileprivate var subscribers = [WeakProxy]() public func add(subscriber: T) { guard ! subscribers.contains(WeakProxy(value: subscriber as AnyObject)) else { return } subscribers.append(WeakProxy(value: subscriber as AnyObject)) } public func remove(subscriber: T) { let weak = WeakProxy(value: subscriber as AnyObject) if let index = subscribers.firstIndex(of: weak) { subscribers.remove(at: index) } } public func contains(subscriber: T) -> Bool { var res: Bool = false res = subscribers.contains(WeakProxy(value: subscriber as AnyObject)) return res } public func dispatch(_ invocation: @escaping(T) -> ()) { clearNil() subscribers.forEach { if let subscriber = $0.value as? T { invocation(subscriber) } } } private func clearNil() { subscribers = subscribers.filter({ $0.value ! = nil}) } }Copy the code

Subscribers can be given events by generic multiple proxies (addPlugin: Internal Add Subscribers in Block above), or by registering blocks.

The Dispatcher instance

Declares a protocol inherited capability to distribute

@objc protocol BDTZEditViewControllerDelegate: BDTZEditViewLifeCycleDelegate, StreamingContextDelegate, BDTZEditActionSubscriber {/ / / / BDTZEditViewLifeCycleDelegate controller statement cycle StreamingContextDelegate preview editing skills / / callback Communication protocol between BDTZEditActionSubscriber Plugin}Copy the code

Controller Event Distribution

public class BDTZEditViewController: UIViewController {/ / instantiate BDTZEditViewControllerDelegate var pluginDispatcher = BDTZEditViewControllerDelegateImp () public override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) pluginDispatcher.dispatch { subscriber in subscriber.editViewControllerViewDidAppear? }} public override func viewDidLoad() {super.viewdidLoad () {super.viewdidload () pluginDispatcher.dispatch { subscriber in subscriber.editViewControllerViewDidLoad? ()}} / * * *... * * / / / / the seek progress callback func didSeekingTimelinePosition (_ timeline: timeline!!! , position: Int64) { pluginDispatcher.dispatch { subscriber in subscriber.didSeekingTimelinePosition? (timeline, position: position) } } /***... / * *}Copy the code

Event passing between plugins

The BDTZEditActionSubscriber protocol is used for event transfer between plugins.

@objc protocol BDTZEditAction {
}
@objc protocol BDTZEditActionSubscriber {
    @objc optional func update(action: BDTZEditAction)
}
Copy the code

BDTZEditAction is an empty protocol that can be inherited by any class to describe any information it wants to pass. Given the nature of the editing tool (the interaction is complex but the material types and operations are limited), only a few actions are required to describe all the states. Currently, we use selected Action, various material actions, panel up and down Action, forward and backward action, and so on to describe adding, deleting, moving, clipping, and saving the draft. Let’s take the example of selecting an action(an event that selects a fragment) :

When APlugin issues a selected event, BPlugin, CPlugin, and so on receive the event and make the corresponding state change.

//APlugin func sendAction(model: Any?) { let action = BDTZClipSeleteAction.init(event: .selected, type: .sticker, actionTarget: model) editViewController? .pluginDispatcher.dispatch({ subscriber in subscriber.update? (action: action) }) }Copy the code
//BPlugin
extension BDTZTrackPlugin: BDTZEditActionSubscriber {
    func update(action: BDTZEditAction) {
        if let action = action as? BDTZClipSeleteAction {
            handleSelectActionDoSomething()
        }
    }
}
Copy the code

When the stickers in the preview area are selected, the axis area is also selected, and the bottom area is switched to the three-level menu. ** When an action is dispatched, all plugins receive it and any plugin that is interested in the action changes its status accordingly. **

Five, the summary

IOS also has a ReSwift framework designed with flux in mind, but the disadvantages of using pure Flux mode are also obvious:

  1. With too many layers, it’s easy to create a lot of redundant code.

  2. Porting old code is a lot of work.

It is more important for us to adopt the Flux pattern design concept than a specific implementation framework. According to the characteristics of the Doka business, we just take the idea and use a single layer structure to manage the relationship between ViewController and Plugin abstraction and event delivery, without adding views to the hierarchy. Plugin can use any architecture such as MVC and MVVM internally, as long as the communication mode is unified.

The above is just a simple example to introduce the application of the editing tool on Flux idea. However, in practical use, we should also consider:

  1. Ui-level masking: a View in the plugin needs to be added to the controller View, causing control-level masking. The BDTZEditLevelView in the code above is designed to solve this problem.

  2. Multithreading problem: in the development of a large number of asynchronous processing tasks, we must specify the plug-in communication between the threads, the Dispatcher should also have thread management code.

  3. Plugin dependency problem: Dispatcher also needs to maintain the dependency relationship between plugins. For example, an action should be processed by APlugin after modifying some data or state, and then processed by BPlugin.

  4. Action bloat problem: Listening to an action requires less code than calling it directly from an API, but it tends to result in an infinite number of actions, so consider extensibility and structuring when defining an action.

Reference links:

[1] reswift. Making. IO/reswift/mas…

[2]facebook.github.io/flux/

[3]redux.js.org

[4] blog. Benjamin – encz. DE/post/real – w…

Recommended reading:

IOS crash log online symbolization practice

Baidu commercial hosting page system high availability construction method and practice

AI in the field of video application – bullets through people

———- END ———-

Baidu said Geek

Baidu official technology public number online!

Technical dry goods, industry information, online salon, industry conference

Recruitment information · Internal push information · technical books · Baidu surrounding

Welcome to your attention