✊ Give Me the Font, Back to You the Animation

A list,

The goal of the stroke sequence background is to generate the corresponding stroke sequence animation data for a given font file (WOFF, OTF, TTF) and the required glyphs (Chinese characters, letters or other languages). Is an exercise in the open source project Make Me Han zi.

Second, the effect demonstration

Display effect

Display effect of hardware side energetically

Background data resources

Json files of stroke sequence animation are generated in the background and distributed through CDN resources. In the case of font determination, a glyph corresponds to a unique data resource (the glyph is encoded by encodeURI with the “%” removed, i.e. “I” -> “E68891”). The business side can directly obtain the static resources of the stroke sequence by concatenating URL.

Highlight Function 1

| stroke and dismantling

  • Prevent the number of strokes generated by the algorithm is wrong, provide manual intervention ability
  • On the left, the same color represents the same stroke
  • By adding or subtracting the red line on the right, you can disassemble or merge the glyphs

Highlight Function 2

Adjusted | stroke order

  • Prevent algorithm generated stroke order error, provide manual intervention ability
  • You can adjust the order of strokes, or the direction of the red

Highlight Function 3

| scale & translation function

  • Provides manual modification when the position or size of the rendered glyph does not meet requirements
  • The size of the font and its position in the field grid have been adjusted at the time the data is generated
  • Drag the red dot in the upper right corner to scale
  • Drag glyph to shift position

Three, animation implementation introduction

Here is mainly to explain how to use the data of background production of pen shun

*/ {" STROKES ":["M 350 571 Q 380 593 449 614 Q 465 615 468 623 Q 471 633 458 643 Q 439 656 396 668 Q 381 674 370 672 Q 363 668 363 657 Q 364 621 200 527 Q 196 518 201 516 Q 213 516 290 546 Q 303 550 316 556 L 350 571 Z","M 584 466 Q 666 485 734 497 Q 746 496 754 511 Q 755 524 729 533 Q 693 554 622 527 Q 598 520 575 511 L 537 499 Q 518 495 500 488 Q 442 472 386 457 L 337 446 Q 327 446 179 416 Q 148 409 173 392 Q 212 365 241 376 Q 287 389 339 404 L 387 416 Q 460 438 545 457 L 584 466 Z","M 386 457 Q 387 493 398 517 Q 405 535 390 548 Q 371 564 350 571 C 323 583 303 583 316 556 Q 315 556 316 555 Q 338 519 337 478 Q 337 462 337 446 L 339 404 Q 340 343 339 289 L 338 241 Q 337 180 334 133 Q 333 115 323 109 Q 317 105 250 119 Q 238 122 239 114 Q 240 108 249 100 Q 309 42 328 6 Q 341 -10 357 3 Q 390 36 390 126 Q 387 169 387 265 L 387 306 Q 387 355 387 416 L 386 457 Z","M 339 289 Q 254 261 161 229 Q 139 222 101 221 Q 86 220 85 207 Q 84 192  94 184 Q 119 166 157 147 Q 169 144 182 154 Q 239 199 338 241 L 387 265 Q 477 314 484 318 Q 499 327 498 337 Q 492 343 479 340 Q 434 324 387 306 L 339 289 Z","M 635 195 Q 690 75 797 -14 Q 876 -62 898 -47 Q 920 -37 914 3 Q 905 34 899 152 Q 900 174 894 178 Q 890 179 884 160 Q 857 75 838 60 Q 823 56 785 88 Q 710 155 670 226 L 644 279 Q 599 381 584 466 L 575 511 Q 547 659 576 752 Q 586 779 543 805 Q 509 827 489 825 Q 470 824 479 795 Q 503 752 507 707 Q 517 601 537 499 L 545 457 Q 573 334 612 245 L 635 195 Z","M 612 245 Q 558 197 452 138 Q 442 132 448 128 Q 455 124 468 126 Q 523 135 574 160 Q 608 175 635 195 L 670 226 Q 706 260 747 317 Q 762 336 778 354 Q 788 361 785 374 Q 781 386 753 410 Q 734 428 723 428 Q 708 427 707 411 Q 701 354 644 279 L 612 245 Z","M 687 669 Q 718 648 754 623 Q 770 613 786 615 Q 798 618 801 632 Q 802 648 789 678 Q 780 697 746 708 Q 665 726 651 715 Q 647 711 651 697 Q 655 687 687 669 [[Z], "" medians" : [458627], [392631], [336588], [274552], [258550], [253542], [220530], [212532], [203522]], [[174404], [2 15398], [241402], [672514], [742512]], [[323556], [351542], [365522], [361116], [340 thirdly], [246113]], [[100206], [124195] [163189], [492334]], [[492807], [537760], [538627], [569435], [612299], [676170], [717112], [779] 13, 817, 22], [[859]. ,78 [880], [891140], [886147], [894173]], [[723412], [737365], [664259], [594198], [489142], [454132]], [[657710], [750,66 8], [781634]]], "strokeInfos" : [{" strokeMode ": 29," strokeName ":" prime "}, {" strokeMode ": 27," strokeName ":" horizontal "}, {" strokeMode ": 40," st RokeName ":" vertical hook "}, {" strokeMode ": 1," strokeName ":" "}, {" strokeMode ": 4," strokeName ":" inclined hook "}, {" strokeMode ": 29," strokeName ":" prime "}, {" strokeMode ": 31," strokeName ":" point "}]}Copy the code

How to Render glyphs

Strokes outline data of each stroke in the corresponding strokes in the original data

< SVG version="1.1" viewBox="0 0 1024"> <g key="wordBg" stroke="var(--color-text-4)" StrokeWidth strokeDasharray = "1" = "1" transform = "scale (4, 4)" > <line x1="0" y1="0" x2="256" y2="0"></line> <line x1="0" y1="0" x2="0" y2="256"></line> <line x1="256" y1="0" x2="256" y2="256"></line> <line x1="0" y1="256" x2="256" y2="256"></line> <line x1="0" y1="0" x2="256" y2="256"></line> <line x1="256" y1="0" x2="0" y2="256"></line> <line x1="128" y1="0" x2="128" y2="256"></line> <line x1="0" y1="128" X2 = "256" y2 = "128" > < / line > > < / g {text / * * / SVG path} < g transform = "scale (1, 1) translate (0, -900)"> {strokes.map((strokePath, idx) => ( <path key={strokePath} d={strokePath} /> ))} </g> </svg>Copy the code
  • Set up thesvgtheviewBoxIs “0 0 1024 1024”. Because, in the acquisitionTTFFor the command data of font font, we will unify the data and convert the font unit length to 1024 unit length to ensure that the output animation data does not need to be adapted when it is used.
  • When drawing text paths, note that a transformation is requiredtransform="scale(1, -1) translate(0, -900)"; Because heresvgThe orientation of the coordinate system is different from that of the font font.
    • I’m going to put one intransformThe effect of

  • transform="scale(1, -1)"Later, will begTo move the glyph to the middle of the grid, you need to move the glyph down

  • *transform="scale(1, -1) translate(0, -900)"* after

Why didn’t I move 1024 here? Because there is a concept of baseline in the TTF font specification; In the current coordinate system, the red line is the font reference line; YMax = 900, yMin=-124. Therefore, you need to move the glyph down to the baseline. As can be seen from the coordinate system in the figure (the origin is at the intersection of the left boundary of the baseline, with the positive Y-axis up), it is different from the original coordinate system of SVG (the origin is at the upper left corner, and the positive Y-axis is down), so the transform is needed to align the coordinate system of the standard font we selected at the beginning.

How to animate

  • throughstrokesNow that I can draw the outline of the glyph, how do I add the redaction?
    • In this case, you need to use the original datamediansField corresponding to the data.mediansThe corresponding data is an array of median lines, and median lines are arrays of midpoints. The following figure

  • How to get themediansHow about converting data to animated data?
  • Calculate the length of each median line
const lengths = medians

    .map((x) => getMedianLength(x))

    .map(Math.round);
Copy the code
  • Calculates the animation of the median line for each strokeduration&delay
let totalDuration = 0;

for (let i = 0; i < medians.length; i++) {

    const offset = lengths[i] + kWidth;

    const duration = (delay + offset) / speed / 60;

    const fraction = Math.round((100 * offset) / (delay + offset));

    animations.push({

      animationId: `animation-${i}`,

      clipId: `clip-${i}`,

      keyframeId: `keyframes${i}`,

      path: paths[i],

      delay: totalDuration,

      duration,

      fraction,

      length: lengths[i],

      offset,

      spacing: 2 * lengths[i],

      stroke: strokes[i],

      width: kWidth,

    });

    totalDuration += duration;

}
Copy the code

  • Animation using stroke-Dashoffset, stroke-Dashari & KeyFrame & Clip-path

  • Stroke, stroke – dashoffset – dasharray parsing

  • MDN clip – path analysis

const animationStyle = `@keyframes ${keyframeId} {

        0% {

            stroke: blue;

            stroke-dashoffset: ${animation.offset};

            stroke-width: ${animation.width};

        }

        ${animation.fraction}% {

            /* animation-timing-function: step-end; */

            stroke: blue;

            stroke-dashoffset: 0;

            stroke-width: ${animation.width};

        }

        100% {

            stroke: var(--color-text-1);

            stroke-width: ${STANDARD_LEN};

        }

    }

    #${animationId} {

        animation: ${keyframeId} ${duration}s linear ${delay}s both;

    }

`;
Copy the code
<g key={`${animationId}${playCount}`}>

    <style>{animationStyle}</style>

    <clipPath key={clipId} id={clipId}>

        <path d={stroke} />

    </clipPath>

    <path

        id={animationId}

        clipPath={`url(#${clipId})`}

        d={path}

        fill="none"

        strokeDasharray={`${length} ${spacing}`}

        strokeLinecap="round"

    />

</g>
Copy the code
  • stroke-dashoffset.stroke-dasharray&keyframeAnimation effect; It’s like you take a big brush and you brush it in the right direction.

  • To optimize the animation effect, just use the outline effect corresponding to the glyphclip-pathKeep only animations within the outline of the glyph

4. Data production principle

Font point information acquisition

TTF font file specification

  • Official – Font configuration rules
  • TTFMain process of font production (from original design draft to digitized glyphs, and then to digitized Outlines in font files)

  • Each font has oneEMBase font box (virtual), this oneemThe box is generally a square of equal length and width; Among themAsecentandDescentEach represents a distance between glyph and baseline

  • And there’s going to be one hereFUnit, for example, 512,1024,2048emThe relative size of the box. Grid of two EM squares: left side eachemIt contains 8 units per right-hand sideemIt contains 16 units. When the unit number is larger, the font resolution is higher and less prone to distortion

  • TTFThe glyphs consist of one or more contours (contour(For the word “I”, there are twocontours: Green + blue

  • I’m going to place all of these points atFUnitIt’s positioned in the coordinate system. Finally, it is converted into a series of drawing instructions in the corresponding coordinate system

Leverage open source toolsopentype.jsParse TTF font files

  • Get the coordinate system information of the required font (ascender: Uppermost distancebaselineThe distance;descender: Distance from baseline, usually negative;unitsPerEm:FUnitThe number of units of phi, you can also think of it as phiTTFFont size in coordinate system)

  • Obtain the point information of all contours and the connection between the points (TTF join point common command: MLQZ)

  • Convert to our standard coordinate system (1024 * 1024, baseline to upper and lower distance 900, 124 respectively)

Stroke break up

As mentioned earlier, TTF glyphs will only contain multiple Outlines and will not be aware of the specific stroke segmentation of the current glyphs. The following figure illustrates which contour points will be connected to a path

Therefore, here we want to cross the path at the stroke junction, so we need other methods to separate the Chinese strokes we need. The key to disassembling strokes is to identify the common boundaries of strokes.

Extract the corner point

  • By comparing the tangent Angle difference across the current point as the end point and the starting point in the forward and backward paths (as shown in the figure)r1) if the Angle difference is greater than18 °, this point is judged to be an inflection point (corner), indicating that the outline of the font has a large amplitude turning point here, which may be the junction of multiple pens.

Deep learning acquisitioncornersThe degree of match between

  • So how do you judge thiscornerIs the point the junction of multiple pens? This is the time to compare allcornerTo see if there is a correlation between them. Want to getcornersThe relationship between point and point needs the help of neural network (Model download address)convnetjsDeep learning, acquisitioncornersThe degree of match between
  • getcornersCharacteristic information from point to point
const getFeatures = (ins: EndPoint, out: EndPoint) => { const diff = out.subtract(ins); const trivial = diff.equal(new Point([0, 0])); const angle = Math.atan2(diff[1], diff[0]); Const distance = math.sqrt (out.distance2(ins)); // Const distance = math.sqrt (out.distance2(ins)); Angle(Angle, ins.angles[0]), Angle(out.angles[1], Angle), Angle(out.angles[1], Angle) subtractAngle(ins.angles[1], angle), subtractAngle(angle, out.angles[0]), subtractAngle(ins.angles[1], ins.angles[0]), subtractAngle(out.angles[1], out.angles[0]), trivial ? 1 : 0, distance / MAX_BRIDGE_DISTANCE, ]; };Copy the code
  • Through model trainingcornersAnd get the corresponding matching score
const input = new convnetjs.Vol(1, 1, 8 /* feature vector dimensions */ );

const net = new convnetjs.Net();

net.fromJSON(NEURAL_NET_TRAINED_FOR_STROKE_EXTRACTION);

const weight = 0.8;



const trainedClassifier = (features: number[]) => {

  input.w = features;

  const softmax = net.forward(input).w;

  return softmax[1] - softmax[0];

};
Copy the code
  • Through the above, a weighted binary graph is finally obtained

  • usingHungarian algorithm, a maximum weight matching graph is obtained. whencornerPoint maximum matching object is not itself, they are joined together to form onebridge(twocornerPoints connected to form a line segment), of course, also take care not to repeat the connectionbridge

Stroke resolution algorithm

Now that we can identify the common boundary of the stroke by generating the bridge, the next step is to use the bridge to split the stroke. [The following is explained through code snippets and corresponding animations.]

. const visited = []; While (true) {/** * add current path segment to result */ result.push(paths[current[0]][current[1]]); */ visited[get2LenArrKey(current)] = true; */ visited[get2LenArrKey(current)] = true; /** go to the start of the next segment path */ current = advance(current); const key = get2LenArrKey(current); / * * determine whether bridge * / if (bridgeAdjacency. HasOwnProperty (key)) {endpoint = endpointMap [key]; /** * If the current point is a common point of multiple Bridges, * will be arranged according to the Angle difference between "the tangent line of the bridge, the slope of the tangent line of the line is equal to its own slope" and "the tangent direction of the current path forward". */ const options = bridgeAdjacency[key]. Sort ((a, b) => Angle (endpoint! .pos, a) - angle(endpoint! .pos, b), ); const next = options[0]; . result.push({ start: current, end: next, control: }) /** * Note a point here: current is added to the path, but it goes directly to the next point without being marked with the visited label. * In order to disassemble the next point, This bridge point is the starting point of the next stroke */ current = next; } const newKey = get2LenArrKey(current); If (comp2LenArr(current, start)) {** let numSegmentsOnPath = 0; /** For (const index in visited) {extractedIndices[index] = true; numSegmentsOnPath += 1; } /** if (numSegmentsOnPath === 1) {return undefined; } return result; } else if (extractedIndices [newKey] | | visited [newKey]) {/ * * visited some skip, */ return undefined; */ return undefined; }}...Copy the code
  • Interpretation animation of the algorithm

The original contour instructions have a default order (strictly ordered, TTF guaranteed), so for non-bridge points, it is easy to know which point is next to the current point

  1. The blue dot represents the point marked “Visited” [When encountering an endpoint of the Bridge for the first time, directly add this point to the path, skip the Visited mark, and then go to the next point]

  2. When there are multiple Bridges at the corner point, the difference between the slope Angle of the selected bridge and the slope Angle of the tangent line in the direction of the current stroke path should be minimum

  3. The red bridge allows the stroke to pass directly through the stroke junction and connects the two points of the bridge with a Line segment

Stroke repair

After separating the strokes by bridge, you can get the following illustration. Behind the seemingly perfect strokes, there are some minor flaws: that is because the places where the bridge is connected are all connected by straight lines, which makes the position of the strokes seem to have been cut by knives

  • I’m going to replace all the lines with cubic Bezier curves
  1. L1And in order toP1Is tangent to the last path segment of the end pointP1point
  2. L2And in order toP2Is the starting point of the next path fragment tangent toP2point
  3. L1withL2toCPpoint
  4. MP1forP1withCPThe midpoint between;MP2forP2withCPThe midpoint between. These two points are going to be the control points for the Bezier curve
  5. Draw three Bezier curves, the black curves in the glyphs

  • The effect after repair

Stroke order animation

After splitting the stroke, it’s time to determine the stroke sequence animation

Gets the stroke center line backbone

  • Increase the sampling points on each stroke (use the quadratic Bezier curve formula to get more key points on the stroke)

export function getPolygonApproximation(

  path: SVGPathType[],

  approximationError = 64,

): PolygonType {

  const result: Point[] = [];

  for (const segment of path) {

    const control = segment.control || segment.start.midpoint(segment.end);

    const distance = Math.sqrt(segment.start.distance2(segment.end));

    const numPoints = Math.floor(distance / approximationError);

    for (let i = 0; i < numPoints; i++) {

      const t = (i + 1) / (numPoints + 1);

      const s = 1 - t;

      result.push(

        new Point([

          s * s * segment.start[0] +

            2 * s * t * control[0] +

            t * t * segment.end[0],

          s * s * segment.start[1] +

            2 * s * t * control[1] +

            t * t * segment.end[1],

        ]),

      );

    }

    result.push(segment.end);

  }

  return result;

}
Copy the code

  • Voronoijs Tyson polygon NPM library (to construct Tyson polygons based on expanded sample points)
  1. Blue dots are the outline points on the stroke outline (also equivalent to sampling points of The Tyson polygon)
  2. The distance between the points in each polygon of Tyson’s polygon and the corresponding sampling point is the shortest. In other words, the tri-junction of multiple polygons is the point with the shortest distance from the sampling points in these polygons (i.e. the black point in the figure).
  3. Leave the black dots in all strokes and connect them to form a median line that controls the direction of the stroke

  1. Control the number of key points in the median line (ensure the longest median line length, but as few points as possible), and finally form the following figure

Stroke sequence animation sort

Once you have the median line of all strokes, you need to determine the order of strokes

  • Structure of Chinese characters: ideographic description characters

  • For a given glyph, look up the Chinese character structure table to obtain the structure split of the glyph and form a structure tree

  • The median line
  1. Add the 'medians' of all substructures to a collection (in the order the structures are disassembled) for easy comparison with the' medians' generated for the current glyphCopy the code
  2. Compare the 'medians' of the child glyph structure and the current glyph structure, and match the corresponding score, and convert into a weighted binary graph matching problemCopy the code
const scoreMedians = (median1: number[][], median2: number[][]) => { assert(median1.length === median2.length); /** let option1 = 0; /* let option1 = 0; let option2 = 0; range(median1.length).forEach((i) => { option1 -= dist2(median1[i], median2[i]); option2 -= dist2(median1[i], median2[median2.length - i - 1]); }); return Math.max(option1, option2); };Copy the code
  1. Hungarian algorithm is used to find out the maximum weight matching relation and get the stroke sequence of the font structure.

Five, the summary

  1. After the above algorithm, brush sequence data can be generated into JSON files and stored in CDN, with an average size of about 4 kB.
  2. In the production process of stroke sequence animation data, more inference and comparison algorithms are used, which can meet many font cases; But the accuracy of the data is still not 100 percent guaranteed (the algorithm is prone to misjudgment when the glyphs are complex). Therefore, in the process of data generation of new fonts, manual intervention is still needed to ensure the accuracy of data.
  3. At present, the brush sequence background also provides a semi-automatic and semi-manual way to produce the brush sequence data under a given font and glyphs. In order to reduce labor cost, error correction algorithm needs to be explored. In this way, when doing batch generation, can be targeted for error location.

Recruiting team

I would like to introduce our team. Our team belongs to bytedance Dailai Intelligence Department. On the one hand, we are engaged in the front-end research and development of Dailai Work Lamp/Dailai Tutoring APP and related education products at home and abroad. In addition, our team has certain practice and precipitation in monorepo, micro front-end, Serverless and other cutting-edge front-end technologies. Common stacks include but are not limited to React, TS, and Nodejs.

Welcome to “Byte front-end ByteFE” resume delivery email “[email protected]”