Introduction:

Narrative visualization has been a hot topic in academic research in recent years. This paper is about the extraction and sorting of some papers (see references at the end of this paper), aiming to draw inspiration from others. At the end of the paper, a research and development idea of a data video generation tool for ordinary users is proposed, welcome to discuss.

1. Narrative Visualization

Narrative visualization simply means telling a story in the form of visualization. Common forms are infographics, data cartoons, data videos and so on.

1.1 Visual Narrative Grammar

Visual Narrative Grammar (VNG) proposes a sequential image understanding model that integrates linguistic theories and psychological experiments. VNG is designed to describe the narrative structure of continuous images.

There are four components of narrative structure: Establisher(E), Initial(I), Peak(P), and Release(R).

  • Establisher (E): A sequence of actions or events that provides reference information but does not participate in the narrative

  • Initial (I): sequence of “start actions or events”

  • Peak (P): The climax of an event

  • Release (R): Sequence after climax

VNG is actually developed in the theory of traditional drama creation, and we usually talk about writing techniques are similar.

As shown above, in a simple cartoon, the Establisher stage constructs the scene, stating the time and place of the character; In the second picture events begin — the hero can’t sleep and starts thinking. In Peak, the character starts to struggle with whether or not to get out of bed and take a shower, which is a repetitive process. Finally, the protagonist decides to sleep in the Release phase.

To show a more complex narrative structure, the four basic components are often constructed in a nested tree structure. This is a bit like a novel, in which small events will be divided within the overall framework, so as to achieve the effect of ups and downs and climaxes. Let’s analyze a few narrative modes.

1.2 Typical narrative mode

[Phase X (Establisher) — (Initial) — Peak — (Release)]

In a typical narrative model, both the overall narrative structure and the local sub-flow, only Peak is required and the other components are optional.

The cartoon above is divided into three parts as a whole:

Initial: Describes two boxers fighting in the ring

-Blair: At the Peak, a beautiful shot from the boxer on the left, knocking out his opponent

Release: the ending

This is a tree nested structure where each stage can extend its own subprocesses. The final realization of an overall narrative structure is: I-P-E-i-ref-p-R

Conjunction schema [Phase X X1 – X2 -… Xn]

In the whole narrative process, we often need to use different ways to show local information and decompose the overall content to be expressed, which can better highlight the key points and express the intention.

If several consecutive images play the same grammatical role in the component of the stage, we can use different combinations to present the final overall concept. In the image below, these successive images on the left create the conceptual equivalent of the panel on the right.

In the figure above, the centralized combination is often seen in cartoons, which is described grammatically as follows:

(a) Action or event (A-conjunction) (b) role in the scene (e-conjunction) (c) part of a single role (N-conjunction) (d) different semantically related elements (S-conjunction)

2. Data video

Data video is a common form of visual narrative, which narrates the story in data through customized graphics and graphics animation combined with audio effects. Data video is a very attractive way of data narrative, which can provide diversified information to the audience in a short time.

For example, below is a video showing China’s economic development trend since the reform and opening up through the changes in GDP data of China, Japan and South Korea, accompanied by patriotic music to highlight the patriotic theme.

www.bilibili.com/video/BV14J…

In order to better understand the content and structure of data videos, relevant papers collected 50 typical data videos for qualitative analysis.

2.1 Narrative structure of data video

It is observed that the data video is also layered and can be further decomposed into subunits. The following figure shows the statistical results, using regular expressions to represent narrative patterns.

EI+PR+

Access the link: www.shutterstock.com/blog/an-inf…

You can watch this 2.07 minute data video. The video follows the (EI+PR+) narrative structure model and uses a combination of voice narration, video footage, data visualization and attentional cues to tell a powerful story about the evolution of the relationship between humans and media.The Establisher section shows the theme, indicating that the video is about the “power of video”, quickly transitioning to the Initial stage, which consists of 4 basic units. The first three units describe a basic fact, statistics on the rise of video usage. At the beginning of unit 4, the narrator asks, “Why video? “This is followed by fragments of everyday life, reflecting on the information that has been presented, and answering questions in one sentencePeakStage builds tension. Peak uses rousing music and animation to grab the audience’s attention. The video then goes on to present more facts that support the answers given in Peak. This is done through four release units, all of which include data visualization. Dynamic charts, highlighting individual bars in a bar chart, and continuous changes in a year range are all common attentional cues in data visualization.

E+I+PR+

The most common narrative mode was E+I+PR+, used in 34% of videos.

E+I+P

These structures contain a climax unit (P) and no ending unit. After presenting some facts in the data, the data video with this narrative structure leaves the audience with a “problem” or “something to think about”, thus completing the narrative.

“EI +” and “ER +”

The flat presentation of data often relies on the charm of data itself to attract the audience.

2.2 Composition Analysis

If the time distribution of these clips is counted, it can be found that Initial takes up the longest proportion, accounting for more than 60% of the video’s time on average. However, such Initial tends to be relatively short in traditional videos.This shows that the structure of data video is different from that of traditional video.Researchers also made statistics on the proportion of data visualization and attention cues in each structural component (figure below), and found that 60% of the Initial video clips contained data visualization elements, which indicates that visualization is often used to pave the way for events, rather than only for the presentation of event results. In addition, the distribution of attention cues is basically similar to that of data visualization, indicating that they are directly related, and data visualization often needs these attention cues to enhance their expressiveness. About 20 percent of the videos begin with a question, and 30 percent end with additional information.

2.3 Use of visual elements of data video

Display type

In data video, visualization of data is the main means of storytelling. While the average duration of data videos was 3 minutes (1 to 7.5 minutes), the average number of data visualizations presented was 6 (1 to 19). On average, 48% of the total time of data video is devoted to data visualization. 72% of the data videos rely on an average of just five different types of visualizations (Figure 2- left). More than half of the data video duration consists of data visualizations, but they are limited to just three representations (scatter plots, bar plots, and maps).

Attention Cues

We identified nine main types of visual and auditory effects designed to capture the viewer’s attention.

The top three most commonly used effects are animation, appear/disappear, and highlight.

3. How to make data video

From the perspective of tools and ecology, there is no independent tool for the production of data video at present, and it has not been subdivided from the big branch of video creation.

Currently commonly used methods

  1. Chart generation tools + video editing tools

Some chart libraries provide simple dynamic chart making tools, but do not effectively fulfill the requirements of visual narrative. Relevant recommendations can reference on zhihu: www.zhihu.com/question/29…

Currently, the types and forms of charting tools available out of the box are very limited, which is one of the reasons that most of the user data videos we see are bar charts and line charts.

  1. Image editing tools + Video creation tools

Almost all well-made data videos are drawn by professional designers and produced in combination with video editing.

Special tools

In some papers, we can see the direction of research, such as DataClips (fereshtehamini. Making. IO/assets/data…).

But there are no publicly available tools, and there is a gap in the market.

4. The idea of data video production and publishing tools for C-end crowd (content creators)

(The following software interface is only an auxiliary example, not a screenshot of the actual data video production tool)

We can see that the frequency of video content creation and distribution, among a significant number of people, has surpassed the creation of written content. Data video has prominent themes, diversified forms of presentation and simple and lively narrative structure, which has natural creative advantages in the field of short video release.

It’s essentially a story design tool

Video is just an end product (or one of the products), and the core function is how to design and choreograph a story. This point is consistent with PPT in concept. We design how a story is told, presented and interacted with each other. Finally, “Export video” is only a functional feature.

Data visualization is the core presentation

This tool perfectly combines the core functions of traditional visual presentation tools and data analysis tools with story creation. Visualization plays several roles here:

  1. Show diversified forms. Can be statistical charts, graphs, Gis and other forms of display into the, get rid of the PPT tool to focus on the use of text, pictures, graphics display form.
  2. A whole new form of narrative. With the help of visual presentation and camera voice, we can have a new multi-dimensional narrative. For example, the whole canvas narrative mode below breaks the point-to-line display narrative mode of PPT tools and expands the “surface”.

Of course we can go further and tell the story in 3D.

Intelligentization is the source power that continuously satisfies user demand

Only smart, evolving tools can continue to meet the needs of users. Intelligentization is reflected in the following aspects:

  1. Intelligent data analysis capability
  2. Intelligent text analysis capabilities
  3. Visual automatic generation capability
  4. Automatic story orchestration
  5. Automatic material recommendation ability

summary

Research on the application of narrative visualization in the context of video content is of great significance not only to the improvement of narrative ability of data visualization itself but also to the improvement of the direction of video content expression. The semantic expression of data visualization and video can learn from each other. In this direction, both content output and academic research are at an initial stage. I hope this article can give you some inspiration.

References:

Fereshtehamini. Making. IO/assets/unde… Mucollective.northwestern.edu/files/2011-… Idl.cs.washington.edu/files/2010-… www.visuallanguagelab.com/P/VNG_Tutor… Fereshtehamini. Making. IO/assets/data… www.zhihu.com/question/29… zhuanlan.zhihu.com/p/73530983