instructions

ARKit series of articles directory

This is Raywenderlich’s translation of the free chapter on ARKit by Tutorials, chapter 8 of the original book. The book chapter 7 ~ 9 completed a space-time door app. The website address www.raywenderlich.com/195388/buil…

This is chapter 8 of our book ARKit by Tutorials, “Add Objects to your World.” This book shows you how to build five immersive, good-looking AR apps using Apple’s augmented reality framework, ARKit. Let’s go!

In the last chapter of this series, you learned how to build your app with ARKit and explore the water level. In this chapter, you’ll continue to build your app and add 3D virtual content to your camera scene via SceneKit. By the end of this chapter, you will have learned:

  • Handling Session interruptions
  • Place the object at the detected level

Before you start, click On Data Download to download the project data and open the Starter Project in the Starter folder.

start

Now that you can detect and render horizontal planes, you need to reset the state if your session is interrupted. ARSession is interrupted when an app enters the background, or when multiple apps are in the foreground. Once interrupted, the video capture will fail and the ARSession will no longer receive sensor data to track. When the app returns to the foreground, the rendered plane remains on the view. However, if your device has changed position or orientation, then ARSession tracking is no longer valid. At this point you need to restart the session.

ARSCNViewDelegate implements the ARSessionObserver protocol. This protocol contains a number of methods that are called if the ARSession is interrupted or fails.

Open the PortalViewController. Swift, and add the following method to realize to expand the existing classes.

/ / 1
func session(_ session: ARSession, didFailWithError error: Error) {
  / / 2
  guard let label = self.sessionStateLabel else { return }
  showMessage(error.localizedDescription, label: label, seconds: 3)}/ / 3
func sessionWasInterrupted(_ session: ARSession) {
  guard let label = self.sessionStateLabel else { return }
  showMessage("Session interrupted", label: label, seconds: 3)}/ / 4
func sessionInterruptionEnded(_ session: ARSession) {
  / / 5
  guard let label = self.sessionStateLabel else { return }
  showMessage("Session resumed", label: label, seconds: 3)

  / / 6
  DispatchQueue.main.async {
    self.removeAllNodes()
    self.resetLabels()
  }
  / / 7
  runSession()
}
Copy the code

Best code:

  1. Session (_:, didFailWithError:) is called when the session fails. Upon failure, the session suspends and no longer receives data from the sensor.
  2. The showMessage(_:, label:, seconds:) method displays the information on a particular label for a few seconds.
  3. SessionWasInterrupted (_:) is called when the video capture is interrupted, such as after the app is in the background. No new video frames will be updated until the interrupted state ends. Here we show the “Session Interrupted “information on the label for 3 seconds.
  4. The sessionInterruptionEnded(_:) method is called after the session interrupts. The session continues from the state it was in before the interruption. If the device moves, all the anchor points will be offset. This avoids offsets and restarts the session.
  5. Put “Session Resume “on the screen for 3 seconds.
  6. Remove previously rendered objects and reset all labels. We’ll implement this later. Because these methods are going to update the UI, all of them are called in the main thread.
  7. Restart session.runsession () resets the session configuration and restarts tracing with the new configuration.

You will see some compilation errors. These errors can be resolved by implementing the missing approach.

Add some variables below the other variables in the PortalViewController:

var debugPlanes: [SCNNode] = []
Copy the code

You’ll use the debugPlanes array to hold all levels rendered in Debug mode.

Next, add a new method under resetLabels() :

/ / 1
func showMessage(_ message: String, label: UILabel, seconds: Double) {
  label.text = message
  label.alpha = 1

  DispatchQueue.main.asyncAfter(deadline: .now() + seconds) {
    if label.text == message {
      label.text = ""
      label.alpha = 0}}}/ / 2
func removeAllNodes(a) {
  removeDebugPlanes()
}

/ / 3
func removeDebugPlanes(a) {
  for debugPlaneNode in self.debugPlanes {
    debugPlaneNode.removeFromParentNode()
  }

  self.debugPlanes = []
}
Copy the code
  1. You define a helper method to display the message text on a UILabel for a sustained period of time. Once the time has elapsed, the visibility and text of the label are reset.
  2. The removeAllNodes() method removes all SCNNode objects currently added to the scene. For now, all you need to do is remove the rendered horizontal planes.
  3. This method removes all rendered levels from the scene and resets the debugPlanes array.

Now, in renderer(_:, didAdd:, for:),#if DEBUG corresponds to the **#endif** preprocessor:

self.debugPlanes.append(debugPlaneNode)
Copy the code

This adds the level planes added to the scene to the debugPlanes array as well.

Note that in runSession(), the session execution needs to pass in a configuration:

sceneView? .session.run(configuration)Copy the code

Replace the above with:

sceneView? .session.run(configuration, options: [.resetTracking, .removeExistingAnchors])Copy the code

Here, when you run the ARSession associated with sceneView, you pass a Configuration object and an array arsession. RunOptions with two Settings:

  1. ResetTracking: The session does not use the previous configuration of device location and motion tracking.
  2. RemoveExistingAnchor: the session on a configuration object of anchor will be removed.

So let’s run the app and try to detect a horizontal level.

Hit testing

Now you are ready to place the object on the detected horizontal plane. You’ll use ARSCNView’s hit test to detect where the user’s finger touches on the screen correspond to the virtual scene. A 2D point in view coordinates actually corresponds to a line in 3D coordinate space. A hit test is the process of finding the object on the line.

Open the PortalViewController. Swift, add the following variables.

var viewCenter: CGPoint {
  let viewBounds = view.bounds
  return CGPoint(x: viewBounds.width / 2.0, y: viewBounds.height / 2.0)}Copy the code

In the code above, you set the variable viewCenter to be the viewCenter of the PortalViewController.

Now add the following method:

/ / 1
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
  / / 2
  if lethit = sceneView? .hitTest(viewCenter, types: [.existingPlaneUsingExtent]).first {/ / 3sceneView? .session.add(anchor:ARAnchor.init(transform: hit.worldTransform))      
  }
}
Copy the code

Explanation of code:

  1. ARSCNView has touch enabled. When the user clicks on the view,touchesBegan() is called and passed a collection of UITouch objects and a UIEvent that represents the touch event. You override this touch handler to add an ARAnchor to the sceneView.
  2. Call the hitTest(_:, types:) methods on the sceneView object. The hitTest method takes two arguments. It receives a CGPoint of the view coordinate system, which is the center of the screen, and an ARHitTestResult type for searching. Here you use the existingPlaneUsingExtent result type, which searches for the intersection of rays from the viewCenter with the horizontal plane in the scene, and the area of the horizontal plane is finite. HitTest (_:, types:) is an array of results of all hit tests, sorted from near to far. We choose the first plane where the rays intersect. This way, you can get results from hitTest(_:, types:) whenever there is a rendered level in the center of the screen.
  3. Add an ARAnchor to the ARSession where the 3D object will be placed in the future. The ARAnchor object is initialized with a transformation matrix that defines the rotation, translation, and scaling of the anchor in the world coordinate system.

After the anchor is added,ARSCNView receives a callback in the agent method renderer(_:didAdd:for:). From here you will be dealing with the space-time gate rendering.

Add the crosshair

Before you can add the portal to the scene, you need to add one last thing to the view. In one passage, you implement a hit test on the sceneView in the center of the device screen. In this section, you will add a tag to the view in the center of the screen to help the user locate the device.

Open main.storyboard. Go to the Object Library and search for a View Object. Drag and drop a View object to the PortalViewController.

Change the name of the view to Crosshair. Add constraints to ensure that its center aligns with the center of the parent control. Set width and height to 10. In the Size Inspector page, the constraint should look like this:

Attributes inspector
Light Gray Color

Select the assistant editor, you will see PortalViewController. Swift on the right. Hold Ctrl and drag the properties from Crosshair in storyboard to the PortalViewController code over sceneView.

Enter the name crosshair in the IBOutlet and click Connect.

PortalViewController
ARSCNViewDelegate 

/ 1
func renderer(_ renderer: SCNSceneRenderer,
              updateAtTime time: TimeInterval) {
  / / 2
  DispatchQueue.main.async {
    / / 3
    if let _ = self.sceneView? .hitTest(self.viewCenter,
      types: [.existingPlaneUsingExtent]).first {
      self.crosshair.backgroundColor = UIColor.green
    } else { / / 4
      self.crosshair.backgroundColor = UIColor.lightGray
    }
  }
}
Copy the code

Code Meaning:

  1. This method is part of the SCNSceneRendererDelegate protocol, which is implemented by ARSCNViewDelegate. Renderer (_: updateAtTime:) is called exactly on each frame and can be used to perform some of the logic required for each frame.
  2. Run the code to detect whether the center of the screen falls on the detected horizontal plane and update the UI on the main thread.
  3. Here a hit test is performed on the sceneView to make sure that the center of the view does indeed intersect the horizontal plane. If at least one result is detected, the Crosshair background color turns green.
  4. If the hit test does not return any results, the background color of Crosshair is reset to light gray.

Run the app.

Move the device around to detect and render a horizontal plane, as shown in the image on the left below. Now move your device so that the center of the device screen falls into the plane, as shown in the image on the right below. Notice that the color of the center view is now green.

Add a state machine

Now that you’ve built an app that probes the plane and places an ARAnchor, you can start adding space-time gates.

To track the status of your app, add the following variables to the PortalViewController:

var portalNode: SCNNode? = nil
var isPortalPlaced = false
Copy the code

Store a portalNode object of type SCNNode to represent your portal, and use isPortalPlaced to indicate whether the portal has been rendered in the scene.

Add the following methods to the PortalViewController:

func makePortal(a) -> SCNNode {
  / / 1
  let portal = SCNNode(a)/ / 2
  let box = SCNBox(width: 1.0,
                   height: 1.0,
                   length: 1.0,
                   chamferRadius: 0)
  let boxNode = SCNNode(geometry: box)
  / / 3
  portal.addChildNode(boxNode)  
  return portal
}
Copy the code

Here we define the makePortal() method, which configures and renders space-time gates. The following things were done:

  1. Create an SCNNode object that represents the space-time gate.
  2. This step initializes an SCNBox object, which is a cube, and uses the cube as geometry to create an SCNode object.
  3. Add boxNode as a child node to your portal and return the portal node.

Here,makePortal() simply creates a space-time portal node that contains the cube object as a placeholder.

Now, replace renderer(_:, didAdd:, for:) and renderer(_:, didUpdate:, for:) with the following methods:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { DispatchQueue.main.async { // 1 if let planeAnchor = anchor as? ARPlaneAnchor, ! self.isPortalPlaced { #if DEBUG let debugPlaneNode = createPlaneNode( center: planeAnchor.center, extent: planeAnchor.extent) node.addChildNode(debugPlaneNode) self.debugPlanes.append(debugPlaneNode) #endif Self. messageLabel?. Alpha = 1.0 self.messageLabel?. Text = """ Tap on the detected \ horizontal plane to place the portal """ } else if ! self.isPortalPlaced {// 2 // 3 self.portalNode = self.makePortal() if let portal = self.portalNode { // 4 node.addChildNode(portal) self.isPortalPlaced = true // 5 self.removeDebugPlanes() self.sceneView? .debugOptions = [] // 6 DispatchQueue.main.async { self.messageLabel?.text = "" self.messageLabel?.alpha = 0 } } } } } func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) { DispatchQueue.main.async { // 7 if let planeAnchor = anchor as? ARPlaneAnchor, node.childNodes.count > 0, ! self.isPortalPlaced { updatePlaneNode(node.childNodes[0], center: planeAnchor.center, extent: planeAnchor.extent) } } }Copy the code

Code description:

  1. Only if the anchor added to the scene is an ARPlaneAnchor and isPortalPlaced is false do you need to add a horizontal plane to the scene to show the detected plane.
  2. If the anchor added is not an ARPlaneAnchor and the portal node is still not placed, then it must be an anchor added by the user manually clicking on the screen.
  3. Create the portal node by calling makePortal().
  4. Renderer (_:, didAdd:, for:) is called when the SCNNode object, node, is added to the scene. You want to place the portal node at the node position. So you add the portal node to a child node of node and set isPortalPlaced to true to indicate that the portal node has been added.
  5. To clean up the scene, you remove all rendered levels and reset the debugOptions so that there are no more feature points on the screen.
  6. Update the messageLabel on the main thread, reset its text and hide it.
  7. In renderer(_:, didUpdate:, for:), you only update the rendered horizontal plane if the anchor is an ARPlaneAnchor, the node has at least one child, and the portal has not been added.

Finally, replace removeAllNodes() with the following code.

func removeAllNodes(a) {
  / / 1
  removeDebugPlanes()
  / / 2
  self.portalNode? .removeFromParentNode()/ / 3
  self.isPortalPlaced = false
}
Copy the code

This method is used to clean and remove all rendered objects from the scene. Details are as follows:

  1. Remove all rendered horizontal planes.
  2. Removes portalNode from parent node.
  3. Reset the state by changing the isPortalPlaced variable to false.

Run the app; Have the app detect a horizontal level, then tap the screen when the on-screen alignment turns green. You will see a flat, large white cube.

What’s the next step?

It’s fun! Here’s a summary of the chapter:

  • You can detect and handle ARSession interruptions when the app goes into the background.
  • You understand how the hit test works in the ARSCNView and the level detected.
  • You might use the results of the hit test to place ARAnchors and SCNNode objects. In the next chapter, the final part of the series, you’ll put everything together, add walls and ceilings, and add a little light to the scene!

If you enjoyed this tutorial series, buy the full version of this book,ARKit by Tutorials, Available on our Online Store.

This chapter materials download