The background,

In how to implement a panorama control in Android (a), introduced some basic situation of the project (there is a demo demo), if the project is helpful to you, I hope the article rewards a thumbup, the project star about.

Project address: github.com/androidZzT/…

In this article, I intend to give a complete overview of the technology stack used in the project, as well as an overview of the process of drawing a panorama onto the Android screen. Without further ado, let’s begin

Two, panorama control to use what technology?

Let’s first say what is the implementation idea of panorama control?

1. What does panorama control do?

Usually, the panorama mode of mobile phone camera can record the scene of 180° before and after. Due to the limitation of lens, the up and down direction may not be recorded, so this kind of photo actually lacks information in a certain direction, and the panorama control is not used to display this kind of “panorama”.

Professional panoramic camera can record the scene of 360° up and down, and can record the THREE-DIMENSIONAL scene into a two-dimensional picture. Therefore, what panorama control needs to do is to restore the two-dimensional image to the three-dimensional scene of the shooting scene.

2. How to recreate a 3D scene on a 2D screen?

Figure 1- Panorama control implementation ideas

As shown in the figure:

  1. Tile each pixel of the panorama onto the surface of a sphere
  2. Imagine our eyes at the center of the ball
  3. The area that we can see on the surface of the inside of the sphere, that’s what the screen is showing
  4. By turning our eyes, we can see the scene from different directions, as if we were standing in the real scene

3. How to draw a ball on the screen?

Figure 2- GENERAL process of 3D rendering

Most of the views used in Android development are 2D. Now we need to draw a 3D View. What techniques should we use?

As shown, from left to right is a general process of rendering the panorama:

  1. The Bitmap is entered into the TextureView in the dotted box (the contents of the dotted box can be replaced by GLSurfaceView, as explained in more detail later).
  2. SurfaceTexture is generated by TextureView
  3. SurfaceTexture and EGL together generate GLContext
  4. GLThread binding GLContext
  5. Call OpenGL API and EGL API through GLThread
  6. Panorama output to screen

Students who are not clear about these concepts in the process may be confused, what is this?? Now, LET me explain one by one

3.1 What is EGL?

EGL is an intermediate interface layer between the OpenGL ES rendering API and the Native Platform Window System

My understanding is that the middle layer of OpenGL dealing with the screen hardware, if you want to draw something to the screen through OpenGL, you have to go through EGL

3.2 What are GLContext and GLThread?

GLContext is the context required to call the OpenGL Api, which consists of SurfaceTexture and some of the contents of EGL. If the OpenGL API is called without an environment, it will fail.

A GLThread is just a normal thread, but with a GLContext attached, it’s called a GLThread

3.3 SurfaceTexture

Captures frames from an image stream as an OpenGL ES texture. Use an image as an OpenGL ES texture

The SurfaceTexture is a bridge between TextureView and EGL, and is necessary to call OpenGL via TextureView.

3.4 Look at the overall process

Once you know the feeling of each link, look at the process and you’ll feel a little clearer.

  1. Prepare a GLContext via TextureView and EGL
  2. Prepare a GLThread and bind the GLContext
  3. Draw a ball by calling OpenGLApi with GLThread
  4. Use GLThread to call OpenGLApi to “paste” the panorama onto the ball

Of course, the technical details are not as easy as the above 4 steps, there are a lot of details, such as how to rotate the camera? What is the process of drawing a sphere with OpenGL? These details will be scattered to a follow-up article, please look forward to.

Interested students can also see their own source first, github.com/androidZzT/… Remember to give a star

3. Complete technology stack

Figure 3- Technology Stack

The figure shows the technologies used in the project. There may be some omissions, which will be improved later.

ZhuanWen statement

If there is a need to reprint the article, please indicate the author and link, thank you for your understanding and support