By Angryli, the client team of Vigorous Intelligence

background

Business needs

Automatic Chinese character tracing is an important functional part of the special program of Energetically tutor-Chinese characters and words. In the early stage of technology research, it is found that the company already has a Native SDK that can meet the requirements, but the following reasons are taken into account:

  1. Word specific technical selection was implemented with Flutter as the main
  2. The need to draw or paint Chinese characters in red appears in multi-function and multi-interface
  3. Extensive use of PlatformView was detrimental to the performance of the Flutter interface. The final decision was to use the Flutter to achieve a set of auto-redding Chinese characters to facilitate integration and subsequent iterations of similar functions.

The implementation of Flutter follows:

  1. Operable data processing
  2. The calibration data
  3. animation
  4. Animation complement frame and other operations, the final realization of the whole Chinese character tracing function of the landing.

Standing on the shoulders of giants

Neither the Native SDK, which has been implemented, nor the landing of Flutter, was an overnight success. Both solutions are based on an open source solution. The more important Chinese character lattice data comes from Github. The input of this part of the resources is completely automated, but its principle has yet to be explored, is a set of Js implementation of open source solutions.

How to express Chinese characters in code

This topic sounds very “code smart,” and code is hard to be smart. Without previous achievements, this question is really puzzling. The above mentioned text resources provide us with a viable solution. Let’s take the “eight” data as an example. The main field involved in this section is strokes:

{ "strokes": [ "M 317 465 Q 318 338 155 190 Q 136 178 79 126 Q 67 110 85 113 Q 110 114 146 137 Q 258 209 325 305 Q 368 363 406 409 Q 419 422 404 441 Q 355 486 329 484 Q 316 483 317 465 Z", "M 446 687 Q 507 636 530 577 Q 608 343 711 190 Q 732 163 846 151 Q 892 147 958 141 Q 983 140 984 146 Q 984 152 963 163 Q  756 269 675 396 Q 621 480 551 644 Q 530 690 483 702 Q 449 709 445 702 Q 438 692 446 687 Z" ], "medians": [[[331, 470], [358, 421], [291, 303], [204, 206], [143, 156], [89, 123]], [[452, 695], [484, 681]. [525, 640], [645, 378], [693, 302], [755, 227], [839, 190], [978, 147]]}Copy the code

The strokes field is an array whose length represents the number of strokes for the current word. The array element is a string of characters, which can be translated into a string of SVG instructions according to a certain format. The coordinate points are connected to form the strokes of Chinese characters, and all the strokes are superplaced to form the outline of the characters.

Translates SVG instructions to readable coordinates

With boilerplate code like this, we usually Google it. We can understand the basic principles very briefly. A string of SVG instructions like this, where M, Q, H, and V are actually a series of instructions, can be compared to the Path method in most UI SDKS.

"M 317 465 Q 318 338 155 190 Q 136 178 79 126 Q 67 110 85 113 Q 110 114 146 137 Q 258 209 325 305 Q 368 363 406 409 Q 419 422 404 441 Q 355 486 329 484 Q 316 483 317 465 Z"
Copy the code

Post a piece of information:

L = lineto(L X,Y) : Horizontal lineto(L X,Y) : Veto (Y1, Y1, Y2, Y2,ENDX,ENDY) : = veto(Y1, Y1, Y2,ENDX,ENDY) : = veto(Y1, Y2, Y2,ENDX,ENDY); S = smooth curveto(S X2,Y2,ENDX,ENDY) Q = Quadratic Belzier Curve (Q X,Y,ENDX,ENDY) : Secondary free-form curves > > T = smooth quadratic Belzier curveto (T ENDX, ENDY) : mapping A = elliptical Arc (A RX and RY, XROTATION, FLAG1, FLAG2, X, Y) : arc

We follow the above instructions, and we end up with a List object (in real code, we end up with a Path object, so it’s easy to understand how to use a coordinate point group instead). These coordinate points are connected to form Path, which is drawn by Canvas, and it is the outline of Chinese characters we want. The outline of the character “eight” mentioned above is drawn as follows:

However, the coordinate values from SVG coordinates do not directly draw the shape shown above.

Convert the original coordinate point to the available coordinate point

In the previous step, we obtained the coordinate points of the text based on the following coordinate system:

  • Tiangzi size: 1024- Y axis up

The field area we need to specify in our business is a random square, and our canvas coordinate system is y-down. We remember that the actual width of the field lattice in the business is width, so how to convert the coordinate point based on 1024 and Y axis up to the canvas with width and Y axis down, we need to go through the following steps:

  1. Scale: Convert a coordinate point of 1024 to a coordinate point based on width
  2. Rotation: Converts points into points facing down on the y axis
  3. Translate: Translate the inverted coordinate points to their “original” positions

Scale down

First, let’s scale down the seating punctuation:

final scale = width / 1024;
final Point point = Point(point.x * scale, point.y * scale); 
Copy the code

Draw the scaled down coordinate points to see the effect:

As you can see, the originally large text outline can be scaled down to display in the specified area, but the direction is not quite right.

Flip Y

In this step, we need to flip the y-coordinate with the X-axis as the center for coordinate points:

final point = Point(point.x, -point.y)
Copy the code

After the rotation of the coordinate axis, the drawing results are as follows:

It can be seen from the comparison of the left and right graphs that the X-axis of the flipped coordinate system is actually the top of the square region, and the coordinate points after rotation appear on the top of the field grid as expected, and the display direction is in line with the expectation.

translation

To solve the problem that the text drawn after flipping exceeds the tian zi lattice, we need to translate the punctuation mark along the Y-axis, and the translation distance is the width of the Tian zi lattice (width and height are equal).

final Point point = Point(point.x, point.y + width);
Copy the code

The result of this step is as follows:

After three steps of narrowing, flipping and translating, the coordinate points obtained are already in an ideal form. If all the Chinese characters are as good as the example, they can be published as online schemes. With the gradual increase of cases, the text that could not be drawn normally was found after three steps of operation:

Obviously, the above “pull up” below the word beyond the tian character. After a long Review of the above three steps, it does not seem to be able to identify the obvious vulnerabilities.

Center of the correction

With the steps above we have the final available coordinate points, but there are still some flaws. As THE number of cases increased, I found that on some words, the final drawn word was beyond the specified range. After log checking in the three steps of operation, it is found that in the process of scale, the values of some coordinate points become negative. This is an obvious problem. Our coordinate points, whether in the y-up or y-down coordinate system, always remain completely in the first quadrant, and the coordinate value becomes negative, which means that the final drawing is bound to exceed the field grid. Because SVG conversion process instruction to the coordinates type boilerplate code, not only for our scenario there is a problem, if you want to dwell on whether shangbu operation sequence and value has a problem, you need to study open source solution, find out its principle to find the right processing sequence, and the fastest solution is for correcting incorrect the coordinates of the point. Considering our writing habits of Chinese characters, most characters are displayed in the center, so we can center the coordinate points after three steps of operation. The steps of center correction are as follows:

  1. Find the four values of (minX, maxX) and (minY, maxY) among all the coordinate points, and calculate the offset values on the x and y axes required to center the text
  2. If the offset value is greater than 0.5, the coordinate values on the corresponding axes are translated. Too small an offset is unnecessary to waste computation.

The final result of our drawing is as follows:

After more case verification, the center corrected scheme can be used for production.

How MiaoHong

Tracing red animation, as shown in the beginning of the GIF, is “smooth write a stroke”, “smooth” is bound to be used in animation. Refer to the iOS implementation:

Flutter does not have the same animation properties as strokeEnd. For red animation, we can think of the most ideal state is:

  1. In a sufficiently small trend zone, a brush of the appropriate width should be used to cross the zone
  2. Calculate the right width: pass through the center point of the stroke in this interval and try to be perpendicular to the edge of the stroke on either side

The “appropriate width” rule above is a difficult task for an average developer to implement in code in a short period of time. Then we have to think about it another way. When I saw that the final drawing was in regular script, I thought of my calligraphy class in my freshman year. We can imagine that when the brush writes a stroke on the rice paper, it conforms to the ideal state described by us, but the calculation process of the appropriate width here becomes the stress degree of the brush. When dissecting the writing process of the brush, his scene looks like this: a circle is constantly moving on a two-dimensional plane, and the trend and size of the circle determine the shape of the strokes. Here are two important points:

  1. movements
  2. The size of the

Stroke direction

The stroke trend is to introduce a second field of font data: Medians. From the above data structure, we can see that the data of the Medians field is a two-dimensional array, and the array of dimension 2 can actually be viewed as a Point structure. The two elements of the array store the x coordinate and the y coordinate, respectively. So the whole Medians is a List, and this coordinate array is the key coordinate points in the stroke trend, which we call bone points. Connecting these bone points gives you the movement of the stroke. The diagram below:

We can already know the trend. The line of bone points is the position of the brush. The remaining size takes us back to the original problem.

The size of the circle

It is still a difficult task to calculate the radius of a circle in a short time. But it was another calligraphy tool – intaglio calligraphy stickers – that gave me inspiration. We don’t need to know how much is the round radius should be, we can give our specify a fixed radius of the circle, in accordance with the above movements of strokes to write a very thick strokes (circle radius is large enough, to ensure the circular area covers the boundary of the stroke, we take a numerical 35 experience, if matts expanded, the geometric expansion radius). We can use the font boundary Path to align the box selection, so that we can achieve neat strokes. Just as Path also provides us with such a method, we used Path.combine to simulate the effect of book Posting. The steps to draw a stroke are as follows:

  1. Draw a circle with radius according to the sequence of bone points, and obtain a Path by taking the union of all the circles. The Path at this point represents a bold stroke
  2. Add a gravure calligraphy post to a stroke: Take the intersection with the boundary Path of the stroke to get Path2. This step can eliminate the part beyond the boundary and only keep the area within the boundary. At this time Path2 after the correction of calligraphy, is already a very neat stroke
  3. Assuming that the number of bone points in the current stroke is N, we animate the drawing process of 0 -> N-1 and automatically trace it in red

Fixed the problem of not enough text frames

Although the red animation appears as scheduled, but such a red animation is not an ideal state, there is a jump frame and stroke fracture.

After visualizing the bone points of the above strokes, we get the following picture:

It can be seen that the overall skeletal points are very sparse, which leads to the above two problems:

  1. In a short time, the distance of the tracing animation is large and uneven
  2. The distance between the third bone point and the fourth bone point in the right stroke is large, and the circle area drawn on these two points cannot reach the intersection of the brush radius, so there is a broken stroke

From the perspective of point square, this is also reasonable, because the trend of the middle stroke basically keeps in the same direction, and the two points before and after are the key points, and the middle area does not need to take the key point. Based on this situation, if we want the red animation to be continuous, we need to record some bones again. Consider that the animation is relatively smooth at 60 frames /s, which means it moves forward at 60 closely spaced bone points per second. So you should follow the same principle when you add points. The supplementary frame steps are as follows:

  1. Calculate the length of all bone points in a stroke
  2. After the previous calculation, given the time to draw a stroke (business custom), the length of the stroke, then calculate the length of the stroke should advance per second
  3. So over here, we know how far we traveled per second, divide by the ideal number of frames, and we get how far we should travel per frame
  4. Between bone points whose length exceeds the advance distance of each frame, the points are added in the unit length of the advance distance of each frame

We end up with bone points like this:

After the completion of the frame, the red animation is relatively smooth. At this point, the whole animation with red strokes can be used for production after more case verification.

Justice overdue

As we said above, we searched for a long time, but did not find the three-step operation vulnerability. It was not until I combed through this document that I reviewed the source code of iOS for evidence and accidentally came across the magic number: 900. So I spent an hour adding this parameter to the three-step operation, and miraculously, all the words were displayed without centering. The steps after adding magic numbers are as follows:

  1. Flip the Y-axis
  2. I’m going to move it along the Y-axis by 900
  3. Geometric narrow

(left is magic 900, right is center corrected)

The result is the same.

This is the SVG code for the word “Po” :

<svg viewBox="0 0 1024 1024" xmlns="http://www.w3.org/2000/svg"> <g transform="scale(1, -1) translate(0, -900)"> <path d="M 519 53 Q 517 149 517 156 L 529 400 L 530 439 Q 531 469 533 518 Q 546 730 559 778 Q 563 793 539 808 Q 508 829 464 837 Q 445 841 433 830 Q 429 825 429 821 Q 428 812 443 790 Q 465 757 466 733 Q 470 664 470 600 L 465 397 Q 461 363 457 296 Q 455 262 443 216 Q 439 171 439 129 Q 437 25 447 -3 Q 462 -58 490 -75 Q 498 -76 502 -71 Q 517 -56 519 53  Z"> </path> <path d="M 529 400 Q 570 394 663 410 Q 784 435 791 441 Q 797 447 798 453 Q 798 470 753 483 Q 725 489 622 457 L 530 439 C 501 433 499 403 529 400 Z" ></path> </g> </svg>Copy the code

You can see that there is a translate(0, -900) sign here, which presumably corresponds to the 900 above, but why this is remains to be seen in the open source library implementation.

legacy

  • Some rare characters are not supported yet
  • You don’t start at the top of the stroke
  • There is extra drawing when drawing vertical hook and horizontal break hook
  • Handwriting verification is not yet supported