This is the 27th day of my participation in the August More Text Challenge.

1 Camera calibration procedure

Based on 2D target plane calibration method, the calibration process is divided into the following three steps: (1) calculate the mapping matrix between the target plane and the image plane; (2) solve the camera parameter matrix; (3) solve the relative position relationship between the left and right cameras, namely solve the rotation matrix and translation matrix

2. Realization methods of each calibration step

The mapping matrix between the target plane and the image plane is calculated. The mapping matrix is calculated by the least square method according to the data of the coordinate points of the target plane and the corresponding coordinate points of the image, without considering the imaging model of the camera

2.1 Solving the camera parameter matrix

Based on the mapping matrix of the target plane and the image plane, the basic equation related to the internal parameters of the camera was obtained, and the internal parameters of the camera were obtained by solving the equation. Considering the lens distortion model, the above equation was obtained

The obtained internal parameters are used as the initial values, and the nonlinear optimization search is carried out to calculate the exact values of all parameters

2.3 Solve the relative position relationship between the left and right cameras

Let the external parameters of the left and right cameras of binocular vision system be Rl,Tl, Rr, Tr, respectively, that is, Rl,Tl represents the relative position of the left camera and the world coordinate system, Rr, Tr represents the relative position of the right camera and the world coordinate system. Therefore, for any point in space, if the coordinates in the world coordinate system, left camera coordinate system and right camera coordinate system are Xw,, Xl, Xr respectively, then Xl=RlXw+Tl; Xr=RrXw+Tr. Therefore, the relative geometric relationship between two cameras can be expressed by R= rrrL-1; T=Tr- RrRl-1Tl

In the actual calibration process, the two cameras are calibrated simultaneously by the calibration target to obtain the internal and external parameters of the two cameras, so that not only the internal parameters of the camera can be calibrated, but also the structural parameters of the binocular vision system can be calibrated simultaneously. According to the single camera calibration process, Rr,Tr, and Rl, Tl can be obtained every time the calibration target changes a position. Therefore, the formula R= RRRL-1 can be obtained. T= tr-rrrl-1tl, a set of structural parameters R and T can be obtained

CvCalibrateCamera2 function internal process

1. Calculate the internal parameter matrix:

(1) The homography function cvFindHomography was used to calculate the homography matrix of each image; The center pixel coordinates of the image are taken as the initial u0 and v0, and b-vectors containing FX and fy are obtained by using M-TM-1=B.

(2) For each homography matrix H, first normalize, then use and two conditions to list the equations, and finally list 2 N equations (N is the number of images).

(3) Using the principle of least squares, SVD singular value decomposition and pseudo-inverse matrix (cvSVD and cvSVB kSb function) were used to solve the parameters. Fx and. Fy, and finally the camera internal parameter matrix M containing four parameters was obtained.

2. Calculate the external parameter matrix for each image

(1) Firstly, the internal parameter matrix and the known image point coordinates (in pixel coordinate system) are used to calculate the image point coordinates XP and Yp in physical coordinate system with distortion factor taken into account by using equations (2) and (3).

(2) through the relationship, because all known, using the method of homography matrix

The initial values of rotation and translation vectors can be obtained by using the orthogonal property.

(3) Refine the rotation vector and translational vector by gauss Newton iteration method with 20 iterations. CvProjectPoints2 each iteration, the first through the projection function, all of the physical world coordinates, considering the distortion, the internal and external parameters of projection, and image at the same time on rotation vector and translation vector of Jacobi derivative matrix, and then through the SVD decomposition and solution of the pseudo inverse matrix to calculate deviation, Add to the initial value as the initial value of the next iteration.

3. Optimize all parameters globally

Global optimization was carried out for all parameters, and the iteration method was also used to refine and fade all parameters for 30 times). During each iteration, the projection function cvProjectPoints2 was first used to re-project all physical world coordinates, considering distortion, internal parameters and external parameters. At the same time, jacobian derivative matrix of rotation vector, translation vector,, and all distortion coefficients K1, k2, P1,p on the image was obtained, and the deviation value was calculated by SVD decomposition and the solution of pseudo inverse matrix, which was added to the initial value as the initial value of the next iteration.

Copy the code