Halcon Calibration series (4) : a detailed explanation of the nine-point method for hand-eye calibration

The so-called hand-eye system is to let the hand grab something when the glasses see it, and the brain needs to know the coordinate relationship between the glasses and the hand. If you compare the brain to B, the eye to A, and the hand to C, if you know the relationship between A and B, if you know the relationship between B and C, then you know the relationship between C and A, so you know the coordinate relationship between the hand and the eye.



The camera knows pixel coordinates, and the manipulator is a spatial coordinate system, so hand-eye calibration is to get the coordinate transformation relationship between the pixel coordinate system and the space manipulator coordinate system.

In the actual control, after the camera detects the pixel position of the target in the image, the pixel coordinates of the camera are transformed into the spatial coordinate system of the manipulator through the calibrated coordinate transformation matrix, and then the motion of each motor is calculated according to the coordinate system of the manipulator, so as to control the manipulator to reach the specified position. This process involves image calibration, image processing, kinematics forward and inverse solutions, hand-eye calibration and so on.

The commonly used calibration methods are: nine-point calibration

Nine-point calibration:

The nine-point calibration directly establishes the coordinate transformation relationship between the camera and the manipulator.

Let the end of the manipulator go so that 9 points get their coordinates in the robot coordinate system, and at the same time, 9 points are identified with the camera to get their pixel coordinates. So that gives us nine coordinates.

It can be seen from the following formula that at least 3 points are needed to find the calibrated matrix.

(1) Calibration: the operator of 9-point calibration in Halcon

(2) Solution

Explanation of some special cases:

Outside the eye in hand

In some cases we see the camera fixed in one place, and then we take a picture and find the target, and control the manipulator to grab it, and that makes sense. We also call it eye-to-hand

Eye on the hand

Another situation is that the camera is fixed on the manipulator. In this case, the calibration process is actually the same as the calibration method for the separation of the camera and manipulator, because when the camera takes photos, the manipulator will move to the position of the camera calibration, and then the camera takes photos to get the coordinates of the target, and then control the manipulator. Therefore, many hand-eye systems with simple cameras fixed at the end adopt this method, and the calibration process can be treated the same as that of hand-eye separation system. We also call it eye-in-hand

9 o ‘clock calibration in Halcon

Blog.csdn.net/elie_yang/a…

The so-called “calibration” is to obtain a matrix relationship in mathematics. One point in space (x,y,z) can be shifted or rotated to another point in space (x1,y1,z1).

This process of translation and rotation (affine transformation) is recorded in a matrix relation (homogeneous matrix).

For example, the following simple Halcon code (two-dimensional variation)

Hom_mat2d_identity (HomMat2DIdentity) TX :=20 ty:=30 HomMat2DTranslate) * Angle :=rad(90) hom_mat2d_rotate (HomMat2DTranslate, Angle, 0, 0, Affine_trans_point_2d (HomMat2DRotate, 20,30, Qx1, Qy1)Copy the code

Note that the homogeneous matrix is stored as a tuple row by row; The last line is not stored because it is the same for all homogeneous matrices describing affine transformations. Therefore, Hom Mat2DIdentity is stored as a tuple [1,0,0,0,1,0].

The purpose of calibration is to get the HomMat2DRotate in the above code. So if I have this transformation matrix, and I give you another point, I get the same change of target point Qx1,Qy1.

The common calibration is: nine-point calibration and calibration plate calibration.

Calibration at 9 o ‘clock

Principle: To put it simply, if someone tells you that A point (x =10,y=20 pixels) in A coordinate system (such as the camera image coordinate system) looks like (x1=2,y1=4) in A coordinate system (such as the manipulator’s coordinate system), then he asks you what you would look like in B if it were (x ‘=20,y’=30).

There are an infinite number of answers, but given more constraints, such as more points (3 or more) being found in A and B coordinates, the answer is unique if you ask (x’=20,y’=30) what you see in B.

Why more than three points is required is that there are at least three known conditions for solving a ternary equation. 9 – point calibration is to improve accuracy.

The behavior of calibration is to complete the above discovery of more points of correspondence, known some points (x,y) in the COORDINATE system A, to walk or visual recognition in the coordinate system B to obtain (x’,y’).

Then the affine transformation homogeneous matrix is obtained by solving the relation from (x,y) to (x’,y’).

Example:

Ax: * known A coordinate system of the nine points = [- 30,0,30-30,0,30-30,0,30] Ay: = [30,30,30,0,0,0, 30, 30, 30] * to identify the coordinates of point B, Bx:=[] By:=[] for Index := 1 to 9 By 1 dev_display (Image) * Row1, Column1, Row2, Column2) gen_rectangle1 (Rectangle, Row1, Column1, Row2, Column2) reduce_domain (Image, Rectangle, ImageReduced) binary_threshold (ImageReduced, Region, 'max_separability', 'light', UsedThreshold) connection (Region, ConnectedRegions) Select_Shape (ConnectedRegions, SelectedRegions, 'Rountedregions, 'and', 0.7, 1) select_shape (SelectedRegions, SelectedRegion2, 'rb', 'and', 32, 100) fill_up (SelectedRegions, RegionFillUp (Row,Column) area_center (RegionFillUp, Area, Row, Column) Bx:=[Bx,Column] By:=[By,Row] endfor * Get the matrix HomMat2D vector_to_HOM_mat2D (Bx, By, Ax, Ay, Open_file ('my_vector.mat', 'output_binary', FileHandle) fwrite_serialized_item (FileHandle, SerializedItemHandle) close_file (FileHandle) stop () * Open_file ('my_vector.mat', 'input_binary', FileHandle) fread_serialized_item (FileHandle, SerializedItemHandle) deserialize_hom_mat2d (SerializedItemHandle, HomMat2D_9p) close_file (FileHandle) tx:=20 ty:=30 affine_trans_point_2d (HomMat2D_9p, tx, ty, Qx, Qy)Copy the code