I’m participating in nuggets Creators Camp # 4, click here to learn more and learn together!

In 2018, the film Ready Player One, directed by Steven Spielberg, was released. Oasis, a virtual reality game in the movie, has become a major form of entertainment for human beings in the future with its high sense of immersion and interactivity, which also represents our vision for the development of virtual reality.

Another animated film, Sword of God, was released in 2017. In Sequence War, the story of augmented reality technology and augmented reality games causing waves in the future world is described. People in the movie can directly obtain visual and auditory feedback in the real world by wearing augmented reality glasses, realizing various functions such as communication, shopping, communication, performance and games.

What is the VR/AR/MR concept that has been gaining momentum over the years? Where did they come from, and how far have they come?

I. Virtual Reality

First of all, let’s talk about the most familiar Virtual Reality. Virtual reality, also known as virtual environment and spiritual technology, refers to the use of simulation to produce a three-dimensional virtual world, so that users feel as if they are in the environment, they can observe things in the space immediately and without restrictions.

In fact, at the beginning of the concept of virtual reality is to describe all virtual scenes including theater, but with the rapid development of computer technology, virtual reality technology gradually refers to the construction of virtual environment through computer simulation. Virtual reality has also evolved from computer screens to CAVE projection systems to head-mounted displays (HDM) — virtual reality interactions range from keyboards and mice to specialized controllers to feedback mechanisms.

As immersive interaction continues to improve, virtual reality has begun to be used in various industries. For example, in VR game Half-Life: Alex released last year, players can completely immerse themselves in another virtual world for adventure. Although there is no good solution to the large scene movement in the game, the immersion is undoubtedly a qualitative leap compared to the traditional game mode. In the field of medical education, compared with traditional paper video, VR can also more intuitively complete the display of THREE-DIMENSIONAL human organs and other experimental processes, coupled with interactive THREE-DIMENSIONAL models, can significantly improve the learning effect; VR meetings are more interactive than traditional meetings, which can shorten the distance between teams and improve communication efficiency, providing an efficient way of communication for the popular global virtual teams in recent years. In the field of business shopping, VR is also powerful. For example, the VR display function on the shopping APP and the home simulation in the screen can reflect the situation of the product more truly than the picture, so that users can obtain more comprehensive product information. Finally, the training in our engineering field. Compared with traditional training, VR can enable workers to complete the simulation of dangerous steps in a relatively safe situation, and reduce risks and costs while keeping deep memories.

Second, Augmented Reality

Let’s talk about Augmented Reality, a technology that has made a big push on mobile devices in the past few years. The current widely accepted definition of Augmented Reality was put forward by Ronald Azuma, who believes that Augmented Reality has the following three characteristics: The combination of virtual and reality, real-time interaction, three-dimensional registration.

To put it simply, it is to superimpose computer-generated digital images on real scenes and interact in real time. Augmented reality can provide users with a new user interface and human-computer interaction. Compared with VR technology, AR not only needs virtual information, but also needs to perceive the external environment to achieve accurate registration of virtual information. Technically, it can be divided into three types:

  • Marker-based AR, that is, feature points need to be stored in advance, such as two-dimensional code or map
  • Mark-less AR is a technology that can be registered directly by extracting the characteristics of the surrounding environment without markers
  • LBSbased AR: AR based on location service

For example, pockemanGo, an AR game released in July 2016, uses GPS and mobile phone cameras and gyroscopes to achieve positioning and posture awareness, allowing players to capture virtual spritesin real life situations. At present, the AR tour function of various scenic spots and museum museum combines image recognition technology to identify and position scenic spots according to images, and overlay the information of scenic spots prepared in advance, so that tourists can intuitively feel the historical heritage of scenic spots. Last but not least, augmented reality + operation and maintenance inspection is widely used in the engineering field. By combining Internet of Things, image recognition and SLAM technology, augmented reality technology can realize operation and maintenance path navigation and equipment information management in the operation and maintenance plant. At the same time, it can also be combined with remote auxiliary functions, so that experts can monitor and guide the on-site process.

3. Mixed Reality

The third part is the rekindled concept of Mixed Reality. MR Mixed Reality was first defined in 1994. Paul Milgram and Fumio Kishino proposed Milgram’s reality-virtuality Continuum. They see the real world and the virtual world as two ends of a continuum, and the one in between is called “mixed reality”. Close to the real environment is Augmented Reality, close to the virtual environment is Augmented virtual. So mixed reality initially included augmented reality.

But with the advent of Microsoft HoloLens devices in 2015, the MR concept became independent. It realizes the real-time perception modeling of the external world through four environment awareness cameras, one depth camera and one video camera, and realizes the three-dimensional holographic display effect through head-mounted devices, plus the application of voice recognition and gesture recognition. Compared with the augmented reality devices that still relied on signs or depth sensors to locate, the interactive experience and display effect of HoloLens have been greatly improved. Virtual objects can not only be displayed in reality, but even interact with real scenes. Microsoft calls this technology, which combines real and virtual, physical and digital objects that can coexist and interact in real time, mixed reality. Of course, mixed reality has made achievements in many fields such as mechanical equipment installation and maintenance, engineering design and medical teaching with its excellent visualization effects. At present, there are many manufacturers at home and abroad using Hololens technology solution to study the visualization application in the engineering field.

Although Hololens redefines MR with its effect, with the launch of ARcore by Google and Arkit by Apple, the current augmented reality technology of mobile devices can also realize the perception of reality and further complete the interaction between virtual and reality. Therefore, the dividing line between AR and MR in terms of technology and effect is blurred again.