Graphic rendering pipeline

The three stages of the graphics rendering pipeline:

  1. The Application Stage
  2. The Geometry Stage
  3. The Rasterizer Stage

Application stage

The application stage is realized by software, such as collision detection, input detection, animation and so on. The application outputs the camera position, lighting, and primitives of the model to the geometry stage.

Primitive elements are made up of vertices, which can be a vertex, a line segment, a triangle, or a polygon. Fragment is a unit of pixel size after rasterization. Different from pixel, fragment also holds information such as depth value, texture coordinates and normals. After various tests, the chips compete to become the pixels that are actually displayed on the screen, each containing only RGBA information.

The geometric phase

Geometry phase on polygons and vertices:

  1. View Transform Model & View Transform
  2. Vertex Shading
  3. Projection of the Projection
  4. Cutting Clipping
  5. Screen Mapping Screen Mapping

The geometry stage is very computationally intensive, requiring about 100 floating-point operations per vertex in a single light source.

Model viewpoint transformation

The model needs to transform its coordinates and normals into the world space through model transformation, and all models are located in the unique world space after transformation. In order to facilitate projection and clipping, the camera needs to be transformed to the origin of the world space, and all models in the world space should be transformed with the camera. This transformation is called viewpoint transformation, and the transformed space is called camera space (observation space).

Vertex shader

The calculation in this stage will be applied to vertices (vertex by vertex), and the computed positions, normals and texture colors can be passed to the rasterization stage for triangular interpolation operations.

The purpose of the vertex shading phase is to determine the lighting effect of the material at the vertex on the model.

projection

The projection transform transforms the visual Volume into the Canonical View Volume. There are two main projection methods:

  1. Orthogonal projection
  2. Perspective projection

Orthogonal projection is the combination of translation transformation and scaling transformation. First, the visual body is shifted to the origin of the camera space, and then the cuboid is scaled to the unit cube. The process of perspective projection is to first extrude the visual object into a cuboid, that is, the shape of the visual object of orthogonal projection, and then perform orthogonal projection transformation.

The projection phase projects the model from 3d to 2D

tailoring

After projection transformation, there are three positions of primitives relative to the visual body:

  1. Completely inside
  2. Completely external
  3. Part of it is inside

Direct culling that is completely external, direct next stage that is completely internal, and partial culling that is partially internal.

The purpose of clipping stage is to clipping some primions located inside the visual body.

The screen mapping

In the screen mapping phase, the coordinates are still three-dimensional, but the display state has become two-dimensional after the projection phase. The x and y coordinates of each pixel are transformed into the screen coordinate system, which together with the Z coordinate system is called the window coordinate system.

The main purpose of the screen mapping stage is to map the coordinates obtained in the previous steps to the corresponding screen coordinate system.

Rasterization stage

The rasterization stage can be divided into the following stages:

  1. Triangle Setup stage
  2. Triangle Traversal stage
  3. Pixel Shading stage
  4. Merging stage

Given transformed and projected vertices, colors, and texture coordinates (all from the geometry stage), color each Pixel correctly to draw the entire image correctly. This process is called rasterization or scan conversion.

Triangle setting

The triangle setting stage is mainly used to calculate the difference of the triangle surface and other relevant data of the triangle surface. This data is mainly used for scan conversion and interpolation of various colored data processed by the geometric stage. This process is performed on hardware designed specifically for it.

Triangle traversal

In the triangle traversal phase, each pixel is checked to determine whether the center of the pixel is covered by triangles to generate fragments. The attributes of the fragment are interpolated from the vertices of the triangle.

The process of finding which sampling points or pixels are in a triangle is usually called TriangleTraversal or scan conversion.

Pixel shader

All per-pixel coloring calculations take place in the pixel coloring stage, using interpolated coloring data as input and the output as one or more color information that will be transmitted to the next stage. Texture mapping is done at this stage.

merge

  1. Color buffer: An array the size of pixels that stores the color of each pixel’s pixels
  2. Depth test: there is a depth buffer, which is the same size as the color buffer. When rasterizing, the Z value of the pixel drawn by each pixel is calculated and compared with the original.
  3. Transparency test: Check the alpha value of the fragment before depth test
  4. Template buffer: a buffer that records the location of primiples. this buffer can be used to decide whether to draw, mask, etc
  5. Frame buffer: Usually contains all buffers a system has, but can sometimes be considered a combination of color and Z buffers.
  6. Accumulative buffer: Used for effects such as depth of field, anti-aliasing, soft shadows, and motion blur. Accumulative buffer is supplemented by a set of operators that accumulate images.

thinking

What is the operation of culling?

The following is a quote from Zizhihu user Clawko

  1. Soft Renderer from Scratch (2.5) – Clipping and Culling again – Zhihu (zhihu.com)
  2. A thorough Understanding of homogeneous cutting – Zhihu (zhihu.com)

There are three traditional culls:

  1. Visual cone elimination: the object is wrapped by the surrounding box, and the visual cone to do collision detection, directly put forward completely invisible objects.
  2. Viewport culling: Perform perspective division before clipping space. Objects in camera position will cause perspective division to zero. Discard those outside the viewport directly, and crop those inside the viewport and generate new polygons. This step consumes performance and the GPU is not suitable for work with many judgment conditions. Modern Gpus are typically clipped with a virtual viewport many times larger than the viewport. For suggestions that go beyond a little bit, draw them directly without wasting too much performance.
  3. Backside cull: Runs between vertex shaders and fragment shaders to cull backsides.
  4. Occlusion elimination: mainly used in the scene with many static objects, this paper will not repeat.

How to understand homogeneous coordinates and perspective division?

The following is a quote from Simple book user StanGame

  1. “Perspective Division” – Homogeneous coordinates and Projections – Jianshu.com

The WWW component of homogeneous coordinates is defined as the vertical distance from the viewpoint to a plane. The WWW is the vertical distance between the camera and each plane of the view. If we move the camera forward, i.e. the WWW becomes smaller, the object projected onto the plane by the camera’s view is reduced (the view moves with the camera).

In graphics, only
w = 1 w = 1 Is the“Correct”. In mathematics, there is no such thing as “incorrect” homogeneous coordinates. Let’s make homogeneous coordinates
w w The component remains
1 1 Is just a projection transformation in computer graphics. Because perspective projection transformation will make the homogeneous coordinates of the transformed point
w = z w = z So we have to divide by the coordinates of the transformed point
z z . It also causes the coordinates of the object in the view plane to divide by PI
0 0 Obviously not.

Pback=(x,y,zb,1)P_{back} =(x,y, z_b,1)Pback=(x,y,zb,1), On set again to treat body midpoint Pfront = (x, y, zf, 1) P_ {front} = (x, y, z_f, 1) Pfront = (x, y, zf, 1), clearly the zb > 0, zf < 0 z_b > 0, z_f zb > < 0 0, zf < 0.

For two-point perspective transformation, which point Pback = (x ‘, ‘y, zb’, – zb) P_ {to} = (x, y ‘, z_b ‘, – z_b) Pback = (x ‘, ‘y, zb’, – zb), Some Pfront = (x ‘, ‘y, zf’, – zf) P_ {front} = (x, y ‘, z_f ‘, – z_f) Pfront = (x ‘, ‘y, zf’, – government), and because of the projection transformation to the NDC space will transform and right-handed coordinate system, Zb ‘<0, zf ‘> 0z_B ‘<0,z_f’>0zb ‘<0, zf ‘>0

If you do perspective division with the secondary coordinates without clipping, Pback=(x ‘,y ‘,zb ‘−zb,1)P_{back} = (x, y ‘, \ frac {z_b ‘} {- z_b}, 1) Pback = (x ‘, ‘y, – ZBZB’, 1), Some Pfront = (x ‘, ‘y, zf’ – government, 1) P_ {front} = (x, y ‘, \ frac {z_f ‘} {- z_f}, 1) Pfront = (x ‘, ‘y, – ZFZF’, 1), The zb ‘< 0, – the zb < 0, zf’ > 0, – government > 0 z_b ‘< 0 – z_b < 0, z_f’ > 0, – z_f > 0 zb ‘< 0, – zb < 0, zf’ > 0, – government > 0.

Zb ‘− Zb >0\ Frac {z_B ‘}{-z_b}>0− ZBZB’ >0. But after perspective division, the ZZZ value of PfrontP_{front}Pfront zf ‘− Zf >0\frac{z_f’}{-z_F}>0− ZFZF’ >0, which is also positive, but we expect that the point should be on the − Z-z − Z axis.

Therefore, in practice, we need to first remove the points behind the viewpoint plane, and then perform perspective division. And for the point on the viewpoint plane, since z=0z=0z=0, it will lead to the WWW component of the point is also 000, that is, the error of dividing by zero will occur in perspective division, so it should be removed.