For more machine learning, ARTIFICIAL intelligence, augmented reality resources and technology dry goods, you can follow the public account: AIRX Community

With the continuous development of technology, AR based indoor navigation applications will see massive growth in different consumer areas in the next few years.

Indoor navigation is very different from outdoor navigation in terms of complexity. For outdoor navigation, millions of people now use this technology because of its low performance requirements; Modern smartphones and even smartwatches come with built-in GPS and maps. The AR indoor navigation technology is quite complex, it contains three modules that must be considered: positioning, mapping and rendering.

Mapping is the only simple module. With maps and coordinates, it’s easy to make a route. This module is easy to upgrade and customize for a given business use case/requirement.

The rendering module manages the design of AR content, and its work depends directly on the accuracy of positioning. We can easily plot routes in 3D, but there are challenges in matching virtual objects to the real world. The quality and accuracy of rendering will depend on THE AR SDK, etc. (e.g. ARKit, ARCore, etc.).

In terms of positioning, things get a little tough. There is no exact way to determine the exact location of the user inside the room, including the exact floor. Determining the right level of accuracy is also a challenge. For example, is ten meters enough? 5 meters? Or one meter?

Available techniques for indoor positioning

GPS for indoor positioning systems: GPS is one of the existing technologies in the field of navigation, however, it does not provide accurate positioning inside buildings. In large and low-rise buildings, such as airports, it is relatively accurate; But it can’t determine more detailed information, such as floor numbers, and the only way to do that is manually, such as using Apple or Google Maps. The point here is that GPS works, but this is not a viable solution.

Visual positioning system (VPS) : Visual positioning system (VPS) has great prospects. Google, for example, uses Street View data to indicate a user’s location in ar-based outdoor navigation, using surrounding buildings as reference points. In addition, ARKit 2 introduces the ARWorldMap class, which can serve a similar purpose.

However, we cannot completely rely on ARWorldMap to determine the exact location of indoor positioning, because:

  • Offices often look the same in different places, that is, rooms and corridors on different floors can be the same.

  • The interior may change at any time.

  • Complex calculations can slow down the solution.

Beacon-based indoor positioning: The most common example is the 2000 battery-powered Bluetooth beacons installed at Gatwick Airport in the UK, which, they claim, provide +/ -3m accuracy. But according to Apple’s documentation, Beacon can only give an approximate distance value, so we can’t rely on signal strength to manually calculate these numbers. When you factor in other factors such as cost ($10-20 per unit), battery replacement (every 1-2 years) and working distance (10-100 meters), it becomes clear that indoor navigation using Beacon only works under certain conditions. But overall, Beacons can still do well because most of the indoor navigation technologies on the market today are beacon-based and use only augmented reality for route visualization.

Visual Markers — Ar based indoor navigation solutions: Visual Markers, also known as AR Markers. Visual Markers are images identified by Apple’s ARKit, Google’s ARCore and other AR SDKS. Visual r tags are used to tell applications where to put AR content. If we place a visual marker somewhere in space, on the floor or on a wall, and store its exact location in the cloud, then, when scanned, we will receive its exact coordinates in the real world.

What is the level of indoor positioning accuracy? Customers often want to know how accurate their applications are. We found that the further away we were from the starting point, the less accurate the results were. To solve this problem, new visual markers had to be placed on average every 50 metres.

How to develop indoor navigation application based on AR

1. Draw maps for AR indoor navigation

Without a map, indoor navigation is impossible. Also, it is not always common to find a map of a particular building with the required picture quality and scalability. That’s why you need custom maps. Our process usually goes like this: create a map using a Cartesian coordinate system, which is used in 3D Graphics and ARKit, and then align the map with geographic coordinates and azimuth using satellite images from Google Maps. Here, not much accuracy is required, so no special equipment is required.

Note: Using bitmap images is not the best choice, as each map is a vector image for high quality scaling and minimal data transfer to ensure high performance. In addition, at the design stage, it is not recommended to use AR ruler, because it has a certain error. This is great for sketching, but absolute accuracy can be ensured by using traditional measuring tools.

Next add metadata for navigation: corridors and rooms to create the graph. Then analyze the surroundings and add visual markers as appropriate, i.e. near entrances, lobbies, elevators, etc.

Care needs to be taken to ensure that the map matches the real world. To do this, physical visual markers must be placed at specific points marked in the plan. The position of the physical markers must match exactly where they are on the digital map in order to correct and adjust their position.

The idea is to place the marker on the plan, transfer it precisely to the real world, scan the marker, and then fine-tune the process by moving and rotating the map until it exactly matches the real world. The final step is to save the updated precise mark location to the cloud.

2. Draw indoor navigation routes

We then use graph theory to construct the route and draw polylines or arrows according to the coordinates (depending on the design requirements). However, using augmented reality to plot routes in the real world presents some challenges. The reality is that AR content is always placed at the top of the camera image, creating the illusion of digital content being placed in the real world. If the entire route was drawn, it would look strange to draw directly through the wall.

Since the goal is to make the route look more natural, the visible part of the route must be obscured in some way. This means that a path must be drawn around the user and fade away along the radius.

The next challenge to solve was how to draw the final target pin of the same size, regardless of its distance, to make it easy to read labels and handle touches. Because of perspective, the farther away objects are from the camera, the smaller they look. Therefore, far needles must be made larger, but they all look the same on the screen.

Here are some visual labeling requirements to consider:

  • The markup must have sufficient visual complexity. The more detail there is on the image, the easier it is for machine vision to recognize it.

  • All tags should have different patterns.

  • Tags must be asymmetric so that the application can clearly identify them from which side.

  • The development suite must know in advance which tags to track.

3. Challenges of indoor positioning based on AR

There are still challenges that need to be addressed before AR-powered indoor positioning systems can gain mainstream acceptance. First, each visual tag or ARReferenceImage has a unique ID that is used to retrieve information from the cloud about the tag’s location in space.

Admittedly, the increasing number of tags degrades performance because all tags have to be checked for matches. If the application is designed for small buildings that require 10 to 20 visual markers, that’s fine. However, if you need to set up navigation in many city supermarket chains, for example, with thousands of visual markers, this can be a challenge.

However, this conundrum can be solved by using GPS, which makes it easier to determine the user’s building. And only local visual markup can be used. In essence, this is how you can add as many maps as you need. There may be millions of visual markers in the cloud, but only a limited set is nearby.

Interestingly, having dynamically updated maps gives your application more flexibility. For example, we created an indoor navigation map for Fruits, a supermarket in the far left corner. If it moves to the right-most corner instead of Greenery, then we have to update its current location in the database and build a new route for it.

4. Wi-fi RTT solves indoor positioning problems

The second challenge is the technical requirement for continuous sessions. To achieve the correct level of accuracy, the user should keep the camera active after scanning the initial marker until the end point is reached. This limitation can be overcome by using new methods of precise positioning such as Wi-Fi RTT (round trip time), an emerging technology that enables mobile devices to be located inside buildings to an accuracy of up to one metre.