Image rotation design based on FPGA

This project is the work of 2019 National FPGA Competition for College Students. The system mainly realizes video rotation at any Angle. The FPGA chip of Unigroup Co., LTD is used as the development platform. The video image is captured from the camera in real time, rotated by algorithm, and displayed through HDMI interface. The project was finally promoted to the final and won the Unigroup Co-creation Enterprise Special Award.

First, the principle of image rotation

Image rotation refers to the process in which the image rotates at a certain Angle according to a certain position, while the image still retains its original size. After image rotation, the horizontal axis of symmetry, vertical axis of symmetry and the center coordinate origin of the image may change, so it is necessary to transform the coordinates in the image rotation accordingly.

As shown in the figure, the point P0 (Xo, Yo) of the source image moves to P1 (X1, Y1) after the original image is rotated clockwise at an Angle of θ.

After derivation, the above coordinate transformation relation of P0 and P1 can be obtained.

Second, MATLAB simulation

Scheme 1: [Forward presetting] mapping from the original image to the target image

In this scheme, the way to realize the code is forward thinking. The coordinates of pixels in the original picture are rotated and then the amplitude is directly transferred to the output image. This scheme aims to find the algebraic correspondence between input coordinates and output coordinates.

In this method, the original coordinates and the target coordinates are first put into polar coordinates, and through the relations in polar coordinates, the equations satisfying the four parameters X0,Y0,X1 and Y1 are found to solve the corresponding coordinate relations, and on this basis, the matrix operation relations between input and output are obtained as follows:

Matlab code implementation is as follows:

Clear all CLC % im = imread('1.jpg'); figure; imshow(im); The rotation matrix a = 40/180 * PI; R = [cos(a), -sin(a); sin(a), cos(a)]; Ch = channel number h = height w = width sz = size(im); h = sz(1); w = sz(2); ch = sz(3); c = [h; w] / 2; Im2 = Uint8 (Zeros (h, w, 3)); for k = 1:ch for i = 1:h for j = 1:w p = [i; j]; Pp = round(R*(p-c)+c); pp = round(R*(p-c)+c); if (pp(1) >= 1 && pp(1) <= h && pp(2) >= 1 && pp(2) <= w) im2(pp(1), pp(2), k) = im(i, j, k); End end end % figure; imshow(im2);Copy the code

However, in the actual test, it is found that the image rotated by this method has serious distortion, as shown in the figure below:

The original image

The rotated image

It is obvious that after rotation, there are big differences between the two images. Firstly, the original image is trimmed, and secondly, there are more flaws (miscellaneous spots) in the target image. The reason is that the pixel position of the target image obtained from the original image cannot be found in the original image. The other is the problem of edge clipping. Because the display area is constrained in this scheme, some pixels will be clipped due to exceeding the boundary in the process of rotation. In view of the above two problems, the following improvements have been made.

Plan 2: [Reverse preset] Map from target image to original image

Due to the noise points in the previous solution and the problem of image edge cutting, so in this scenario, we adopted reverse thinking, with the coordinates of target image and the coordinates of the original image coordinate matching, if can find a match in the original image of the image, it shows that point after the rotation point coordinates, if can’t find this point in the artwork, do not show this point, In this way, the problem of miscellaneous points is solved.

Where, PP is the corresponding matrix of the coordinate after rotation. The region of the original image is defined in the IF statement, which can be used to exclude coordinate points in the original image. In this way, the problem of miscellaneous points can be solved.

In this scheme, the coordinates correspond as follows:

MATLAB simulation code is as follows:

Clear all CLC % im = imread('1.jpg'); figure; imshow(im); Find the rotation matrix a =20 / 180* pi; R = [cos(a), sin(a); -sin(a), cos(a)]; Ch = channel number h = height w = width sz = size(im); h = sz(1);
w = sz(2);
ch = sz(3);
c = [w;h] /2; Uint8 (zeros(h, w,3));
for k = 1:ch % traverses pixels at all positions of the output imagefor i = 1:h
       for j = 1:w p = [j; i]; % p: the pixel coordinates of the output image % round is rounded pp = round(R*(p-c)+c); %pp: reverse lookup of pixels corresponding to the pixel coordinates % of the input imageif (pp(1) > =1 && pp(1) <= w && pp(2) > =1 && pp(2) <= h)
                im2(i, j, k) = im(pp(2), pp(1), k); End end end % figure; imshow(im2);Copy the code

In this way, the rotated image has a good degree of reduction and meets the requirements of the corresponding topic. The effect of the specific scheme is shown in the figure below:

The original imageCopy the code

The rotated image

As shown in the figure, compared with scheme 1, the image effect is much better, but the image edge still exists the phenomenon of edge cutting.

Solution 3:

Considering that the display area of the rotated image is not divided, such rotation is only the rotation of a single pixel point, and then coordinate points are recombined on the display area of the original image to obtain the displayed image. In the way to solve the problem, the redivision of the target display area is adopted.

The specific idea is to take the length and width of the original image as the benchmark, and then use the coordinate transformation relationship to convert the length and width into the rotated coordinate system to obtain the display area of the target image in the rotated coordinate system. The code is as follows:

Im = imread('1.jpg'); figure; imshow(im); The rotation matrix a = 30/180 * PI; R = [cos(a), -sin(a); sin(a), cos(a)]; R = R'; % Calculate the size of the original image sz = size(im); h = sz(1); w = sz(2); ch = sz(3); c1 = [h; w] / 2; % Calculate the size of the canvas required to display the complete image hh = floor(w*sin(a)+h*cos(a))+1; ww = floor(w*cos(a)+h*sin(a))+1; c2 = [hh; ww] / 2; Im2 = uint8(ones(hh, ww, 3)*128); for k = 1:ch for i = 1:hh for j = 1:ww p = [i; j]; pp = (R*(p-c2)+c1); mn = floor(pp); ab = pp - mn; a = ab(1); b = ab(2); m = mn(1); n = mn(2); If % linear interpolation method (pp (1) > = 2 & pp (1) < = h - 1 & pp (2) > = 2 & pp (2) < = w - 1) im2 (I, j, k) = (1 - a) * (1 - b) * im (m, n, k) + a*(1-b)*im(m+1, n, k)... + (1-a)*b*im(m, n, k) + a*b*im(m, n, k); End end end % figure; imshow(im2);Copy the code

In this way, the problem of image edge clipping is solved, and the whole image can be displayed completely. The actual effect is as follows:

The original image

The rotated image

From the effect of the illustration, we can see that the problem of clipping the edge area has been solved, but the problem is that the area shaded by the image is much larger than the original image.

Based on the above three schemes and the actual requirements, the display range of the whole image is limited due to the fact that our display is on a fixed-size screen, and the adoption of CORDIC algorithm for coordinate transformation will cause too much delay. Finally, considering the balance of processing speed and resource occupation, scheme 2 is selected as the design scheme of image rotation.

Third, rotation coordinate calculation

In this design, the image is required to have any rotation Angle from 0 to 360, and the coordinate transformation requires the sine and cosine values of the Angle.

Use MATLAB to generate cosine and sines table, and expand its 256 times, print to the file. Using the obtained values of the sines and cosines table, write them into verilog code to generate the sines and cosines lookup table. Index its sines and cosines by input Angle values. Matlab generates a list of sines and cosines in the following code;

The sine and cosine are calculated by MATLAB and stored in FPGA’s on-chip storage space in advance. When coordinate transformation is carried out, the sine and cosine values of corresponding angles are read and coordinate transformation is carried out. Because the calculated sines and cosines are floating point numbers, fpGas are good at integer calculations. Therefore, the conversion from floating point to integer is carried out. The specific implementation method is to multiply the calculated floating point sine and cosine values by 256 and then take the whole, and the calculated result is 256 times larger than the original result. In digital circuit, the division operation can be carried out by shift. The result is that a shift of 8 bits to the right is equivalent to dividing by 256.

The core code of coordinate transformation is as follows:

The coordinate transformation calculation module is encapsulated as a sub-module. After the coordinates and rotation angles of the input and output images are input, the coordinates of the pixels corresponding to the input images can be calculated. Then read the pixel value of the coordinate and write it to the corresponding coordinate position of the rotated reconstructed image.

Results show

0. Original picture (normal display)

1. Rotate 22 degrees clockwise

2. Rotate 90 degrees clockwise

3. Rotate 270 degrees clockwise

4, clockwise rotation 341 degrees

Five,

Reply FPGA2019 in the public account “Digital building blocks” dialog box, you can get the project engineering source code, detailed documentation, MATLAB simulation code.