Author: Conco Man – Hercules

Project background

In the static class design, linglong platform has made corresponding achievements. On this basis, combined with the current environment, we think we can do some dynamic class design, and transform animation and sound into data that can be stored, portable and reusable. In this way, users can use these high-quality animations and effects in a relatively simple way when creating.

What problem does the video editor solve?

The main purpose of the video editor is that the user can manipulate the static PSD to achieve the desired dynamic design effect. Compared with AE and other complex video editing software, the learning cost is greatly reduced, and the reusability and portability of dynamic effects also reduce the workload of users.

The following is the design effect:

Development ‘

How to get your static PSD to “move”?

Refer to the process of AE animation production, first of all, the script and storyboard will be preset, and then plan how the lens movement in the storyboard, how the role movement, as well as processing and planning materials. We can extract several key points: multiple scenes, camera movement (that is, the overall dynamic effect of the scene), planning materials (the time and length of layer content appearing can be flexibly controlled). The main function points of video editor operation are as follows:

  • The fusion of multi-scene switching and transition effect makes the video effect more vivid and flexible;
  • The setting of scene motion effect and motion effect parameters reduces the development of motion effect of the same type (such as the combination of bit-motion effect into one), opens the imagination of designers to use motion effect, and gains additional video effect;
  • Layer operation, adjust the appearance time and duration;

The editor interface is as follows:

State management

The implementation of the video editor is mainly divided into five parts: video preview area, dynamic effect adding area, parameter editing area, layer operation area and scene operation area. As shown in the following figure, every operation of other parts will be mapped to the video preview area, and data of each part will be shared. In addition, every step of the editor’s action needs to be “memorized,” so that the editor can go back and undo it.

The analysis will involve the following scenarios, such as:

  • The state of components in the preview area needs to be shared
  • Changes to other action areas change the state of components in the preview area
  • Component states all need to be revocable/recoverable

We can use Redux to centralize state management to reduce data flow between components; As for the undo restore function, we can use redux-undo to enhance the existing undo restore function according to the existing reducer and configuration object.

import ReduxUndo from 'redux-undo'
Define the original reducer
const editReducer = (state = null, action) = > {
  switch (action.type) {
    case VIDEO_INIT: {
       const { templates } = action.payload
       return { templates }
    }
    case VIDEO_TPL_CLEAR: {
      return{}}}// Enhance the revocation function of reducer with ReduxUndo
export const undoEditReducer = ReduxUndo(editReducer, {
  initTypes: [VIDEO_TPL_CLEAR],
  filter: function filterActions (action, currentState, previousHistory) {
    const { isUndoIgnore = false } = action
    return! isUndoIgnore },groupBy: groupByActionTypes([SOME_ACTION]),
  /* custom groupBy:(action, currentState, previousHistory) => {}, */
})
Copy the code

Parameters that

  • InitTypes: History will be set (reset) according to the type of initialization operation
  • Filter: filters that help filter out actions that you don’t want to include in undo/redo history
  • GroupBy: Actions can be combined into a single undo/redo step using the default groupByActionTypes method. You can also implement custom grouping behavior so that if the return value is not NULL, the new state is grouped by that return value. If the next state is grouped with the previous state, the two states are grouped together in one step; If the return value is null, redux-undo does not group the next state with the previous state.

Use store.dispatch() and Undo/Redo Actions to perform Undo/Redo operations on your state

import { ActionCreators } from 'redux-undo'
export const undo = () = > (dispatch, getState) = > {
  dispatch(ActionCreators.undo())
}
export const redo = () = > (dispatch, getState) = > {
  dispatch(ActionCreators.redo())
}
export const recovery = () = > (dispatch, getState) = > {
  dispatch(ActionCreators.jumpToPast(0))
  dispatch(ActionCreators.clearHistory())
}
Copy the code

conclusion

For state management, we can first consider whether redux, MOBx and other tools should be introduced from the following points:

  • Whether state is shared by multiple components or across pages;
  • Component state needs to span the lifecycle;
  • State requires operations such as persistence, recoverable/undo, etc.

When using Redux to manage state, avoid pulling all state away from the Redux store, such as

  • The private state of the component;
  • Fewer layers of component state transfer;
  • Data that can be destroyed when a component is unmounted, etc

The principle is to put it inside the component if you can. Secondly, in order to make the state readable and operable, before designing the state structure, it is necessary to clarify the relationship between various data objects, balance the complexity of data acquisition and operation, and recommend flat data structure to reduce nesting and data redundancy.

The layer interaction

In the process of using the editor, layer interaction is the most frequent. We refer to the interaction of AE and Final Cut, commonly used client video editing software, and try our best to provide users with convenience of operation and layer visualization on the Web. The specific effects are as follows:

Layer combing operation requirements mainly include:

  • Layer tracks need to scale (adjust layer duration)
  • The motion track on the layer can be scaled separately (adjust motion duration)
  • The layer tracks need to move left and right, and the dynamic tracks follow (adjust the time of occurrence).
  • Dynamic effect track can move side to side alone (adjust the time of dynamic effect)
  • Different layer tracks can be adjusted up and down, and the dynamic tracks move with the layer tracks (adjust the layer order).
  • Drag to display different looks

At the beginning, we first considered the need to move the layer order. We implemented the basic layer order dragging and moving based on react-sortable-hoc. However, the layer stretching and dragging need to be processed by custom mouse events, and the layer movement needs to be controlled by custom calculation. And we didn’t initially consider that the appearance of the drag source would need to be adjusted during the drag process, and eventually we abandoned this implementation. We needed a more customizable drag component. After some comparison, we settled on the React-DND drag component.

Helps you build complex drag-and-drop interfaces while keeping components separate; It is ideal for transferring data between different parts of an application while dragging, and even better, components can change their appearance and application state in response to drag-and-drop events.

React-dnd is built on the HTML5 drag-and-drop API. It can take screenshots of dragged DOM nodes and use them as “drag previews”, simplifying the operation of drawing when the cursor moves. However, the HTML5 drag-and-drop API has some drawbacks. It doesn’t work on touch screens and offers fewer opportunities for customization on IE than other browsers. That’s why there’s pluggable HTML5 drag-and-drop support in React-Dnd, or you can write your own implementations based on touch events, mouse events, etc., without it.

Next, we introduce the basic implementation from the outside in.

Scene: the level of

Importing required components

import { DndProvider } from 'react-dnd'
import HTML5Backend from 'react-dnd-html5-backend'
Copy the code

Put DndProvider in the outer layer of the scenario and set Backend to HTML5Backend

<DndProvider backend={HTML5Backend}> 
  <TemplateViewer   // -----Display components in a single scenariotemplate={tpl}
    handleLayerSort={handleLayerSort}
    onLayerDrop={onLayerDrop}
    onLayerStretch={onLayerStretch}
  />
  <CustomDragLayer />// -- Custom drag preview layer</DndProvider>

Copy the code

The TemplateViewer contains different types of layer components. Each layer component provides a pure render component method called renderLayerContent, which looks like this:

export function renderLayerContent (layer) {
  return <div style={{... }}>.</div>
}

export default function XxxxLayerComponent (layer) {...return <div>{renderLayerContent(layer)}</div>
}
Copy the code

In the CustomDragLayer, renderLayerContent is called according to the component type of the currently dragged object to draw the visual content, so that the view before and after the drag is consistent.

The layer level

The layer can be dragged up or down, or left or right, meaning that it is both the source of the drag and the target of the placement.

To distinguish drag-and-drop purposes, we define two drag-and-drop sources

  const [{ isHorizontalDragging }, horizontalDrag, preview] = useDrag({
    item: {
      type: DragTypes.Horizontal,
    },
    collect: monitor= > ({
      isHorizontalDragging: monitor.isDragging(),
    }),
  })
  const [{ isVerticalDragging }, verticalDrag, verticalPreview] = useDrag({
    item: {
      type: DragTypes.Vertical,
    },
    collect: monitor= > ({
      isVerticalDragging: monitor.isDragging(),
    }),
  })
Copy the code

In the placement process, drag type is used to determine the processing

  const [, drop] = useDrop({
    accept: [DragTypes.Horizontal, DragTypes.Vertical],
    drop (item, monitor) {
      // Handle drag left and right
    },
    hover: throttle(item= > {
      // Handle sorting up and down
    }, 300})),Copy the code

Associate the DOM with the defined drag source and drop target. The outermost DIV is the layer’s draggable area, that is, the place target, then the horizontal drag layer, then the vertical drag layer

<div ref={drop}> // -- place the target DOM
  <div ref={verticalPreview}>

    <div ref={horizontalDrag}>// -- drag the DOM horizontally<div ref={verticalDrag}>// -- drag the DOM vertically<Icon type='drag'/>
      </div>/* Layer content display */<div>{renderLayerContent(layer)}</div>
    </div>
  </div>
</div>
Copy the code

The general framework for dragging layers up and down, left and right has been implemented.

You can set the transparency of the drag source to 0 according to the current drag status to keep the generated screen snapshot without displaying the drag source

<div ref={drop}>// -- place the target DOM<div
    ref={verticalPreview}
    style={{ opacity: isVerticalDragging ? 0 : 1 }}
  >.</div>
</div>

Copy the code

When dragging horizontally, set the drag source translucency, the same as when dragging up and down.

In the layer

The layer has two areas, the lower area can be stretched by the left and right action points, the upper area can be stretched by the left and right action points within the width of the lower area and also by the left and right action points. The implementation method of movement has been introduced before and will not be repeated. For the operation of stretching, we package a Stretch class to deal with it uniformly

function Stretch ({ children, left, width, onStretchEnd, onStretchMove, }) {
  function handleMouseDown (align) {
    // Calculate offset
  }

  return (
    <div>
      {children}
      <div
        className={classnames(styles.stretch, styles.stretchHead)}
        onMouseDown={handleMouseDown('head')} / >
      <div
        className={classnames(styles.stretch, styles.stretchEnd)}
        onMouseDown={handleMouseDown('end')} / >
    </div>)}Copy the code

Pass in the area that needs to be supported as a children with a Stretch

<div>
  <div>
    {motions.map((motion, i) => <Stretch key={i}>{/* an area above */}</Stretch>)}
  </div>
  <div>
    <Stretch>{/* Lower area */}</Stretch>
  </div>
</div>
Copy the code

Experience optimization

Add shortcut keys

The whole editor content is relatively much, for frequent operations, we can keep the operation habit of common shortcut keys. React-hot-keys can be used to perform functions such as space playback and delete.

You first introduce the shortcut key library, then specify the bound shortcut key and add event handling.

import Hotkeys from 'react-hot-keys'

<Hotkeys
  keyName='space'
  onKeyDown={(keyName, e) = > {
    e.preventDefault()
    play()
  }}
/>
Copy the code

Text to SVG

In addition, there is a small technique for layer content display. The product needs Chinese layer tiling display. Pity that I originally calculated the number of times the text was displayed by the length of the text and the length of the track, and then pushed it into the node. You can convert text to SVG and display it as a background image. How sweet!

<div
  className={styles.contentText}
  style={{
    backgroundImage: `url("data:image/svg+xml;utf8,
      <svg xmlns='http://www.w3.org/2000/svg' version='1.1'     width='${size(layer.text) * 12 + 15}px' height='35px'>
      <text x='10' y='22' fill='black' font-size='12'>
      ${layer.text}
      </text>
      </svg>") `,}} / >Copy the code

Effect:

Project summary

This article describes the processing of the main module of the operation area in the video editor. In terms of state management, we mainly need to clarify whether it is necessary to introduce management tools and whether all states must be moved to the Store after using state management tools, etc. In addition, for complex layer drag-and-drop functions, like peeling an onion, you should first remove layers to improve their structure. For the project, after receiving the requirements, we analyze them from whole to part, prioritize the overall framework and the realization of core functions, and then consider how to improve user experience. Requirements are prioritized so that we can prioritize development and improve efficiency.

The resources

[1] the react – DND: react – DND. Making. IO/react – DND/a…

Welcome to the bump Lab blog: AOtu.io

Or pay attention to the bump Laboratory public account (AOTULabs), push the article from time to time: