This paper will introduce the images required by DJI UAV based on DJI SDK for panoramic photography, excluding the image synthesis Mosaic algorithm in the later stage.

The basic realization process of panorama

Panoramic view refers to a fixed observation point that can provide free browsing in the horizontal direction of 360 degrees in azimuth and 180 degrees in vertical direction. Simplified panoramic view can only provide browsing in the horizontal direction of 360 degrees. Generally, there are two ways to obtain panoramic images: panoramic shooting and panoramic image Mosaic. The former requires special equipment, but it is very convenient and easy to operate. However, this kind of equipment is very expensive and not suitable for popularization. The latter only needs ordinary cameras, but the corresponding image projection and splicing are required. The so-called image projection refers to the process of shooting the real image onto a unified cylindrical or sphere surface, which can eliminate the rotation relationship between images and only retain the translation relationship, suitable for splicing. Image Mosaic refers to the synthesis of an image from two overlapping images from different perspectives. Obviously the latter focus on the image Mosaic and fusion. Knowledge and principles of computer vision techniques in synthetic panoramas

It is obvious that the second scheme mentioned above is adopted to take panoramic pictures with uav rather than using special equipment. The synthetic algorithm processing of the later stage of the panorama picture is consistent. The special equipment just saves the trouble of moving the camera to shoot each Angle, because the fixed camera is placed in each Angle of the special equipment, which is stupid and expensive.

The uav is a natural choice for panoramic photography because of its stable head and convenient camera Angle.

Panorama picture synthesis process

After taking photos of each Angle of the fixed point, the next step is to synthesize the image Mosaic algorithm. Synthetic splicing can be divided into the following processes:

Feature point matching: find the common image parts of the material picture

Image matching: connect matched feature points and estimate geometric transformation between images

Image balancing compensation: globally balance the lighting and tone of all images

If it is cloudy, the light from all angles may be more uniform. If it is sunny, the exposure results from different angles will definitely be different due to the surface light source of the sun. In the future, when the final synthesis looks natural, the global exposure and tone need to be balanced.

For days

At present, conventional unmanned aerial vehicles (UAVs) cannot shoot the sky vertically at 90 degrees. In fact, most panoramas do not require details of the sky, because they are all under the same blue sky. However, it would be very unnatural to have a panorama without the sky, so some sky details are usually added at the end of the composition.

Use drones to take pictures from all angles

The maximum elevation of the head of a conventional DJI drone is 30 degrees, but even the company’s own panoramic camera is only set to 15 degrees. Because when the elevation is 30 degrees above the limit, the top area will have a large area of the blade.

Head +15 degrees 9 images head 0 degrees 9 images head -25 degrees 9 images head -60 degrees 6 images head -90 degrees 1 image

We chose a simpler scheme: 8 photos at 15 degrees, -5 degrees, -35 degrees and -65 degrees, and one photo at -90 degrees vertical to the ground.

Shooting order: horizontal first or vertical first

Once you’ve decided where to shoot, consider the order in which to shoot: horizontal or vertical first.

If the panorama contains an elevation Angle, the vertical movement of the head should be from low to high.

The uav does not have to be horizontal because it needs to adjust its attitude for a certain period of time after each movement. But the PTZ has a stabilization system, so the picture is stable.

Exposure strategy

It is a reasonable choice to use global metering to get as balanced an exposure as possible. If you’re shooting a single photo, that’s fine, if you’re shooting from all angles, and then you’re putting it all together, then global metering isn’t enough. Because the panorama synthesis picture has two characteristics: adjacent pictures need to have obvious feature points; The details of the main body below the horizon are more important than the details of the sky. Using global metering in a scene with a large light ratio, the exposure will not be the same as expected when there is sky in the shot. Because the sky is bright, the exposure will be low, and with the detail in the sky, the subject we want on the ground will be severely underexposed. This is typical when it comes to backlighting:

When the final synthesis can be such a result, the color of the sky is close to white, which is also conducive to the late sky filling process.

Exposure lock

Now we know that the exposure target is accurate exposure of ground objects at horizontal and elevation angles, and the sky can be overexposed. It is natural to think that this is local metering! In THE DJI SDK, the local metering is to divide the picture into 8 * 12 blocks, such as setting the metering area [0, 0], then the upper left corner is used as the metering area. We can set the metering area as [6, 6] when the holder is -5 degrees and 15 degrees, which is the metering area near the bottom of the picture. The idea was beautiful, but when we actually shot in this way, we found that there was no human participation in the shooting process. [6, 6] had a high probability of local deviation, and the result was not very stable. It could be a little bit brighter in this picture, or it could be dark. After abandoning the local metering scheme, we came up with the idea of exposure locking. When the head is at -35 degrees, the picture is the whole ground. After the shooting, the lock exposure is completed. The exposure of -35 degrees is used for -5 degrees and 15 degrees, which not only ensures the accuracy of the ground exposure, but also makes the photo exposure in the vertical direction have better continuity. See this article: How the DJI SDK implements Exposure locking.

Tips

Dji’s product line of uAS ranges from the entry-level Spark at 2,000 yuan to the professional-grade M Series, and the performance of the camera varies widely. In the process of panoramic shooting, the camera needs to take dozens of photos continuously. After every camera movement, it needs to re-focus, measure light and set exposure parameters. Therefore, the performance of cameras must be tested for different models, otherwise the task may be interrupted because the camera busy error is thrown during the shooting process due to the heavy workload of the camera.

The total process