IOS implements string animation

preface

Recently I saw a string animation called BadApple on the Internet, which is very interesting. The author is using Python +OpenCv conversion process. OpenCv is an open source project, so why can’t we do this on iOS phones?

The original address

The body of the

First of all, let’s put the effect picture:

This article mentions the relevant content demo address

Video web disk Address:

Link: pan.baidu.com/s/1GjSzUKIn… Password: lojq

If we want to achieve this effect, we can’t just write code, we have to know how it works, and then we can know why we’re doing this. Right?

The principle of

Ps: The principle here is not necessarily so professional, which combined with my understanding, if there is any mistake, please point out

First of all, the essence of video is sound + image. The content related to sound is not our research direction, so it is temporarily omitted.

So if we want to animate a video to a string, the first thing we need to know is, how does an image go to a string

Image to string

Before we can study how to convert, we need to have a general understanding of image storage.

Image storage

An image is made up of many small squares with a clear position and color. The position and color of these squares ultimately determine the appearance of the image. These small squares are also called pixels

Image display

When the image is displayed on the screen, the color of each pixel of the image is rendered at the corresponding position on the screen, and we see the corresponding image.

Computers generally use bitmaps to display images

Bitmaps are typically accessed using two-dimensional arrays that store the color of the pixel.

color

In computers, colors are generally RGB channels (in iOS, I haven’t used any other), which means that a color is created by stacking three base colors (Red, Green, Blue, Blue). All colors can be composed of different intensities of three color channels.

We usually use an image which is usually an 8-bit image, or in this case, three colors with a total of 256 intensity levels from 0 to (2^ 8-1 = 255), and the total number of colors that can be displayed is 256^3.

At this point, the image has changed from a concrete picture to abstract data.

Gray image

Because a color image uses three color channels to describe the color of a pixel, we can also think of a color image as having three dimensions.

However, if we want to display it as a string, we can only display one of the dimensions (leaving aside the fact that strings can display multiple colors when rich text is used).

Therefore, we need to reduce the dimension of the color image, that is, through some algorithm we can use a color channel can describe the characteristics of the image.

Gray image is to meet our needs and appear.

define

In computing, a Gray scale digital image is an image with only one sample color per pixel. Such images are usually shown as shades of gray from darkest to brightest white, although in theory this sample could be different shades of any color, or even different colors at different levels of brightness. Gray image is different from black and white image. In the field of computer image, black and white image has only two colors, and gray image has many levels of color depth between black and white. However, outside of the digital image realm, “black and white image” also means “gray image”, for example, a grayscale photograph is often called “black and white photograph”. In some articles on digital images monochrome images are equivalent to grayscale images, and in others black and white images.

Conversion formula

1. Floating point algorithm: Gray = R * 0.3 + G * 0.59+B * 0.11
2. Integer method: Gray = (R * 30 + G * 59 + B * 11)/100
3. Gray = (R * 76 + G * 151 + B * 28)>>8;
4. Gray = (R + G + B) / 3;
5. Take only green: Gray = G;

Why choose grayscale image?

A string image, meaning that each pixel of the image is represented with a character. A character is a one-dimensional variable (in this case, we ignore the font, color, and size differences of each character). So we need an image that is also one-dimensional — a grayscale image

How to convert grayscale pixels into strings

Here we provide an array of strings representing gray values:

@[@"$", @"@", @"B", @"%", @"8", @"&", @"W", @"M", @"#", @"*", @"o", @"a", @"h", @"k", @"b", @"d", @"p", @"q", @"w", @"m", @"Z", @"0", @"o", @"Q", @"L", @"C", @"J", @"U", @"Y", @"X", @"z", @"c", @"v", @"u", @"n", @"x", @"r", @"j", @"f", @"t", @"/", @"\\", @"|", @"(", @")", @"1", @"{", @"}", @"[", @"]", @"?", @"-", @"_", @"+", @"~", @"<", @">", @"i", @"! ", @"l", @"I", @";", @":", @",", @"\"", @"^", @"“, @”‘, @”.”, @” “] ‘

According to the Gray value of each pixel of the corresponding Gray image, we obtain the percentage of the corresponding Gray value (i.e. Gray * 1.f / 255.0). Then, according to the percentage, the corresponding characters are obtained and spliced into a complete string. This string, this is the string image

The process diagram

Process implementation

Now that we’ve looked at the fundamentals, let’s start writing the application

First up, of course, is the video. If you want to watch the original video directly, you can click here

OpenCV configuration

OpenCV is what?

OpenCV stands for Open Source Computer Vision Library, which is a cross-platform Computer Vision Library. OpenCV is sponsored and co-developed by Intel Corporation. It is distributed under the BSD license and can be used for free in the commercial and research fields. OpenCV can be used to develop real-time image processing, computer vision, and pattern recognition programs.

IOS configuration OpenCV

There are three ways to configure OpenCV

1. Compile directly from source code (but I don’t know much about it, either)
2. Import from CocoaPod (I won’t tell you more about it because it feels like a lot of work to do)
3. Download the packaged iOS Framework

I will only introduce the third type here

After we create the corresponding project, drag the decompressed Framework into the project and add the following dependency libraries:

AVFoundation.framework
AssetsLibrary.framework
CoreMedia.framework
CoreVideo.framework
Copy the code

Here are the dependencies I’ve needed so far to make sure our project compiles error-free

Image processing

First, because OpenCV is c++ code, all classes that we want to interact with OpenCV need to be changed to.mm files so that the compiler recognizes c++ syntax

Next import the header file

#import <opencv2/opencv.hpp>
#import <opencv2/imgproc/types_c.h>
#import <opencv2/imgcodecs/ios.h>
#import <opencv2/videoio/cap_ios.h>
#import <opencv2/core/core.hpp>
#import <opencv2/highgui/highgui.hpp>
Copy the code

These header files need to be imported before we import the other files, meaning they need to be written at the top, otherwise an error will be reported.

Next I write the image-to-string operations as follows:

// Declare a structure to indicate the length and width
typedef struct{
    int width, height;
}SizeT;
Copy the code

1. zooming

(UIImage *)resizeImage:(UIImage *)image withSize:(SizeT)size {CV ::Mat object is the two-dimensional array object of the corresponding image, CV ::Mat cvImage; // convert UIImage object to UIImageToMat(image, cvImage); cv::Mat reSizeImage; CV ::resize(cvImage, reSizeImage, CV ::Size(sie.width, sie.height)); / / / release cvImage. Release (); UIImage *nImage = MatToUIImage(reSizeImage) / / / release reSizeImage. Release ();return nImage;
}
Copy the code

2. Color image is converted to grayscale image

- (UIImage *)grayImage:(UIImage *)image { cv::Mat cvImage; UIImageToMat(image, cvImage); cv::Mat gray; CV ::cvtColor(cvImage, gray, CV_RGB2GRAY); cvImage.release(); UIImage *nImage = MatToUIImage(gray); gray.release();return nImage;
}
Copy the code

3. Converts a grayscale image to a string

- (NSString *)convertImage:(UIImage *)image { cv::Mat gray; UIImageToMat(image, gray); Int row = gray. Rows; Int col = gray. Cols; NSMutableArray <NSString *>* array = [NSMutableArray arrayWithCapacity:row]; NSArray *pixels = @[@"$"The @"@"The @"B"The @"%"The @"8"The @"&"The @"W"The @"M"The @"#"The @"*"The @"o"The @"a"The @"h"The @"k"The @"b"The @"d"The @"p"The @"q"The @"w"The @"m"The @"Z"The @"0"The @"o"The @"Q"The @"L"The @"C"The @"J"The @"U"The @"Y"The @"X"The @"z"The @"c"The @"v"The @"u"The @"n"The @"x"The @"r"The @"j"The @"f"The @"t"The @"/"The @"\ \"The @"|"The @"("The @")"The @"1"The @"{"The @"}"The @"["The @"]"The @"?"The @"-"The @"_"The @"+"The @"~"The @"<"The @">"The @"i"The @"!"The @"l"The @"I"The @";"The @":"The @","The @"\" "The @"^"The @"`"The @"'"The @"."The @""];;

    for (int i = 0 ; i < row; i ++) {
        NSMutableArray <NSString *>*item = [NSMutableArray arrayWithCapacity:col];
        
        for(int j = 0; j < col; Int temp = gray. At <uchar>(I, j); // Calculate the grayscale percentage CGFloat percent = temp / 255.f; Int totalCount = (pixels. Count - 1) * percent; [item addObject: Pixels [totalCount]]; } / / will be converted to a string array [array addObject: [item componentsJoinedByString: @""]]. } gray.release(); // Returns a string of broken linesreturn [array componentsJoinedByString:@"\n"];
}
Copy the code

After these steps, the last step we have left is to display the string on the Label.

Some pits on iOS

1. The string image first needs to be equal width font (because non-equal width font will cause image shift, display confusion), but the system standard font is non-equal width font.

Solution: Use the monospaced Courier font

2. On iOS, UILabel with center setting can cause string rendering to rip

Solution: Use the default Settings (left aligned) for all uilabels. If you need to center the Label, you’d better use AutoLayout to center the Label

To deal with video

Video processing is essentially converting each frame of the video into an image, and then converting the image into a string. Playback is a string that is displayed on the screen at every frame interval of the original video, and then becomes animated.

Specific details are not more verbose, you can see the corresponding demo

At the end

Image processing is actually a very advanced science, so I’m just going to convert the image to a string very simply.

If you have any questions about this, you can also pay attention to my public number and private chat. See will reply and answer