UI is curved in Unity

Under VR, curved UI can enhance the user’s sense of immersion in the scene and get a better visual experience

Scheme selection

1. To make a set of UI based on surface, we basically only use Image and Text. Image is easier to handle, and it is ok to paste Texture directly to the mesh of a surface, but Text is more troublesome. We can’t simply get the Texture of a section of text, we have to cut the Texture of each text from the font file, then stitch it into a Texture, and then paste the final Texture into a surface mesh.

2. Rendering the flat UI to a Texture alone and then attaching it to the surface mesh is not a problem in realizing the surface. However, in order to meet the three-dimensional effect under VR, there are certain differences between what we see with our two eyes. Only through these differences can we have a three-dimensional feeling. Some of the stereoscopic effects on the UI are lost (such as the UI floating forward). Also, using this approach, the UI that our eyes see is very different from the real UI, requiring a redesign of the gaze input.

3. It’s a lot easier to distribute your UI along a surface, and each UI element has a surface effect, but you just have to figure out the right positions and angles to make your UI look like a surface as a whole. At present, there are many VR surface effects are achieved by this simple method, but the surface effect achieved by this scheme is not very good, it is a pseudo-surface effect. As you can see in the picture below, you can see sharp straight lines at the top and bottom edges.

   

   
   So if each of them is distributed along the surface,UI elements have a surface effect“, then the overall surface effect can be seen. As shown in the figure below
   
   

   

Considering the difficulty and effect of implementation, we choose plan 3.

Mathematical principles in surface reduction

Whether you distribute your UI along a surface, or you implement the surface effect of each UI on its own, you’re essentially doing the same math — calculating the coordinates of points on the plane that map to the surface. Since our curved surface is a cylinder, the axis line of the cylinder is parallel to the Y-axis, and the Y-axis coordinates are the same before and after the change. Here is the principle:

Length correspondence before and after transformation
The ratio of arc length to radius is Angle

Surface,

Let’s distribute the UI along the surface

Calculate the curved coordinates for each UI. The calculation method has been given above. Note that after calculating the playing coordinates, you also need to adjust the Angle of the UI so that the front of the UI is the direction from the center of the circle to the transformed coordinates. Such as after the above transformation in point B UItransform. Forward = the transform. The position. The normalized. At this point, the overall UI has the pseudo-surface effect described above.

The surface effect for each UI element

Unity provides BaseMeshEffect to modify the mesh generated by UI elements to achieve some effects. Different Unity versions of this API have some differences. This API uses Unity5.3.4, mainly rewriting the ModifyMesh(VertexHelper VH) method. In ModifyMesh method main content:

public override void ModifyMesh(VertexHelper vh)
{
        base.ModifyMesh(vh);
        if (!this.IsActive() || ! bendEnable)return;

        /* Check whether the vertex coordinates need to be regenerated or modified, if not, use the cached vertex coordinates */

        if (cachedVertices == null || cachedTriangles == null || verticesDirty)
        {
            // To modify vertices, first remove the vertices generated by Unity
            List<UIVertex> originUIVertices = new List<UIVertex>();
            vh.GetUIVertexStream(originUIVertices);
            /* Perform some transformations to vertices, including adding vertices and recalculating vertex coordinates, in different ways for Image and Text */
        }

        // If the material changes, re-color the point
        if (materialDirty)
        {
            UpdateVertiecsColor(cachedVertices);
            materialDirty = false;
        }
        // Clear the vertices generated by Unity and place our recalculated vertices on the mesh
        vh.Clear();
        vh.AddUIVertexStream(cachedVertices, cachedTriangles);
        / / can also use the vh. Depending on the type of generating vertices AddUIVertexTriangleStream (cachedVertices), set the vertex
}Copy the code

1. Check whether vertices need to be updated

Unity calls ModifyMesh(VertexHelper VH) when it sees that the UI needs to be updated. Unity itself triggers UI updates by changing sizes and textures, We can also trigger graphic.setallDirty () graphic.setverticesDirty (). Recalculation of vertices is not necessary in all cases. Recalculation of vertices is rarely required when we calculate the surface of a UI. Recalculation of vertices is triggered only when the UI size changes. Vertex calculation is time-consuming, so it is recommended to calculate the vertex first.

2. Extract vertices

The extracted vertices are a UIVertex list. Normally, every three in the list form a triangle. Changing the position of the elements in the list will cause the UI to display an exception, so VertexHelper outputs vertices in order.

3. Vertex calculation

In this step, there is a big difference between Text and Image. The main reason is that general Text has enough subdivided triangles. You only need to recalculate the coordinates of the vertices to achieve a good surface effect. Four vertices are still a plane after recalculating the four vertices. Therefore, it is necessary to consider adding some vertices to the mesh of Image to make the triangle on Image sufficiently subdivided.

The processing of Text

cachedVertices = new List<UIVertex>();
vh.GetUIVertexStream(cachedVertices);
BendMeshCylinder(cachedVertices);Copy the code

The BendMeshCylinder(cachedVertices) function transforms the vertices passed in to the cylindrical surface. It should be noted that the coordinates in UIVertex are local coordinates relative to UI itself. Process does not change UIVertex list order, processed still keep the triangle of before order, so finally directly using the vh. AddUIVertexTriangleStream (cachedVertices), set the vertex.

Processing of Image

In general, Image only needs 4 vertices to form two triangles. However, there are 6 vertices taken from VertexHelper, and each three forms a triangle. Two of them are reused, as shown in the figure below. 2 and 3 are the same vertex.

List<UIVertex> originUIVertices = new List<UIVertex>();
vh.GetUIVertexStream(originUIVertices);
TrisToQuads(originUIVertices);
for (int i = 0; i < originUIVertices.Count; i += 4)
{
    CreateQuads(originUIVertices, i, cachedVertices, cachedTriangles);
}
BendMeshCylinder(cachedVertices);Copy the code

The originUIVertices of TrisToQuads are the originUIVertices of TrisToQuads. The originUIVertices of TrisToQuads are the originUIVertices of TrisToQuads. And then using CreateQuads(originUIVertices, I, cachedVertices, Triangles), using linear interpolation inside each quadrilateral, The UV of each point inside the quadrilateral can be calculated from the position of the coordinates inside the quadrilateral. As shown in the figure below, the red points are the new vertices. When the quadrilateral is subdivided enough in the X-axis direction, there is no need to add more vertices. Normally, the edge of the picture with Sliced mode is subdivided enough.

vh.AddUIVertexStream(cachedVertices, cachedTriangles)

4. Change the color of vertices

If you change the UI color attribute, can trigger MaterialDirty,. We can through the Graphic RegisterDirtyMaterialCallback to monitor the change, and then in ModifyMesh () changes the color of the vertices. . The other is not recommended to monitor Graphic RegisterDirtyVerticesCallback to determine whether to need to recalculate the vertices, because change the vertex color, this callback will be called.

---

The principle of surface is described above, if you really want to use it, you also need an Editor. Note that it is best not to adjust the COORDINATES and angles of the UI in the curved state, not only difficult to adjust the desired position, but also affect the overall curved effect. In this paper, the cylindrical surface is mainly focused. If it is a sphere, the calculation method of UI coordinates and the refinement method of Image triangle should be modified. If it is a more complex surface, it is not recommended to use this method, because it involves the data calculation of the surface and the effect is difficult to guarantee.