instructions

ARKit series of articles directory

New features in ARKit1.5

  • Video resolution on iOS 11.3 increased to 1080p, the rest unchanged;
  • Added vertical plane recognitionARWorldTrackingConfiguration.PlaneDetection;
  • newARPlaneAnchorIn the rough plane estimate, seeARPlaneAnchor;
  • newARSession.setWorldOriginMethod can reset the origin of world coordinates.
  • I came to a point where I thought I had to look for ard-shaped Images
  • ARsessionAfter being interrupted, you can try to resume tracing. As shown in thesessionShouldAttemptRelocalizationProxy method;

One more enhancement:

  • I hit the break point, and then I go back,sessionVIO(Visual Inertia odometer) will also continue to work. The original virtual object placed in the world coordinate system is still visible.

Image recognition

This function is actually very simple to use. When the AR function is started, set the images to be recognized, and then process the identified anchor points in the callback method.

// Load the group of images to be identified
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else {
    fatalError("Missing expected asset catalog resources.")}let configuration = ARWorldTrackingConfiguration()
configuration.detectionImages = referenceImages // Set identifiers
session.run(configuration, options: [.resetTracking, .removeExistingAnchors])

Copy the code
// MARK: - ARSCNViewDelegate (Image detection results)
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    // Determine the type of anchor points identified
    guard let imageAnchor = anchor as? ARImageAnchor else { return }
    let referenceImage = imageAnchor.referenceImage // Get the image corresponding to the anchor point
    updateQueue.async {
        
       // Create a plane to display the detected image (the same size as the image)
        let plane = SCNPlane(width: referenceImage.physicalSize.width,
                             height: referenceImage.physicalSize.height)
        let planeNode = SCNNode(geometry: plane)
        planeNode.opacity = 0.25
        
        /* 'SCNPlane' is vertical in its local coordinate system, but 'ARImageAnchor' assumes that the image is horizontal in its local coordinate system, so it rotates the plane. */
        planeNode.eulerAngles.x = -.pi / 2
        
        /* Image anchors are no longer tracked after initialization, so create an animation to show that the plane appears */
        planeNode.runAction(self.imageHighlightAction)
        
        // Add to the scene
        node.addChildNode(planeNode)
    }
   
}
// Highlight the animation
var imageHighlightAction: SCNAction {
    return .sequence([
        .wait(duration: 0.25),
        .fadeOpacity(to: 0.85, duration: 0.25),
        .fadeOpacity(to: 0.15, duration: 0.25),
        .fadeOpacity(to: 0.85, duration: 0.25),
        .fadeOut(duration: 0.5),
        .removeFromParentNode()
        ])
}
Copy the code

Refer to the referenceImages

Reference images are the images to be identified and need to be placed in Xcode’s material folder asset Catalog:

  1. Open projectasset catalog, click the Add button (+) to add a new AR resource group;
  2. fromFinderDrag an image to a resource group
  3. You need to specify a physical size for each image in the inspector and, optionally, a descriptive name;

Pay attention to image recognition: Images need to be carefully selected, designed, and configured to achieve better reliability and performance:

  • The physical size of the picture should be set as accurately as possible. ARKit needs to use this value to determine the distance between real world images and the cameraARImageAnchorWrong distance to camera;
  • When adding images to Xcode’s material folder, Asset Catalog, you should pay attention to the estimated image quality warnings provided by Xcode. High contrast images work best in image recognition;
  • Use flat images for detection. If the image to be identified is not flat, such as the label on a wine bottle, then ARKit may not identify or give the wrong anchor position;
  • Consider how your images will behave in different lighting. If the image is printed on smooth paper or displayed on a screen, the reflection on these flat surfaces may hinder recognition;

Apply best practices

Use the recognized picture as the AR scene reference frame. Use the identified images to anchor the virtual scene, rather than letting the user choose where to place the scene, or simply placing it roughly in the user’s scene. You can even use multiple recognition images. An app for a retail store, for example, could identify posters pasted on either side of a door (one poster tile on the left and one on the right) to figure out where to exit, creating a virtual character and walking out of the door.

Note: You can use the ARSession setWorldOrigin(relativeTransform:) method to reset the world coordinate system so that you can place all the anchors and related content on the reference point of your choice.

Design the AR process, using the identified images as the starting point for the virtual content. ARKit will no longer track the location and orientation of detected images. If you try to keep the virtual content glued to the recognized image, it may not display correctly. What you should do is start a dynamic scene using the identified image as a frame of reference. For example, your app needs to identify a poster for a science fiction movie, and then several spacecraft will fly out of the poster and fly into the surrounding environment.

Consider when image detection is allowed to trigger (or repeat)AR interactions. For each reference image in the detectionImages array in the Configuration Session Configuration,ARKit adds only one image anchor point. If your AR process adds virtual content to a recognized image, this action will only happen once by default. To experience this process again, call the session’s remove(Anchor) method to remove the corresponding ARImageAnchor. Once the anchor is removed,ARKit will add a new anchor the next time the image is recognized. For example, as in the previous example of the sci-fi movie poster, when the first animation is playing and the spaceship is flying out of the movie poster, you don’t want to have another one. Wait until the animation is finished before removing the anchor points, so that when the user aligns the poster again, the animation can be triggered again.

The creation of ARReferenceImage

In addition to the previously ARReferenceImage. ReferenceImages (inGroupNamed: bundle:) method can be used as a reference for the image from the material pictures, there are two other creation method:

- (instancetype)initWithCGImage:(CGImageRef)image orientation:(CGImagePropertyOrientation)orientation physicalWidth:(CGFloat)physicalWidth;
- (instancetype)initWithPixelBuffer:(CVPixelBufferRef)pixelBuffer orientation:(CGImagePropertyOrientation)orientation physicalWidth:(CGFloat)physicalWidth;
Copy the code

These two methods create reference images from CGImageRef and CVPixelBufferRef, respectively, which is much more flexible.