Due to work needs, recently began to learn the Unity platform AR development, found a lot of learning materials and watched a lot of videos, and finally follow the Zhihu great God cattle star tutorial for learning. Unity from The beginning to the grave — bookmark this one
It mentions the introduction to Catlike Coding, which is my main learning content. It is very friendly for beginners and explains in detail, from easy to difficult. From Unity interface layout familiar, common operations, to C# grammar basic explanation, framework principles are mentioned.
Strong Unity
After completing the introduction, I was struck by the power of Unity and its ecology:
Convenient graphical interface Settings: a few drags and clicks of buttons complete the basic Settings.
Powerful preview debugging functions: real-time preview effects in multiple Windows, Camera, X/Y/Z and other angles to view the scene, there are console and performance parameters interface.
Integration with the VS editor: Despite the power of the VS Universe first editor, Unity automatically updates previews after saving code, and VS breakpoint debugging compatibility makes the two feel like one.
Full built-in and 3rd party components: Comes with common components, common 2D/3D objects, common material types/lighting types/shaders, particle effects, tail effects, etc. When that’s not enough, you can easily import third-party components for even more powerful effects.
Rich built-in object types and functions: common geometry functions, vector/matrix/quaternion conversion and operations.
In addition to Unity’s strength in 3D, it’s not letting up on THE AR side either. ARFoundation makes it easier to develop on Android /iOS platforms simultaneously, and integrates with the original ecosystem to make it easy to deliver powerful visuals. And ARFoundation itself is also rapidly updated, constantly follow up the new features of ARKit and ARCore, there is basically no new AR features lag.
Does Apple still have a chance?
After feeling the power of Unity, I can not help but think: Unity is so powerful, then AR development is not directly use Unity good? Better visuals, cross-platform, who wants to learn ARKit and ARCore? What’s good about Apple’s native ARKit? Apple vigorously promoted AR will be willing to do for others to marry clothes?
Let’s start with an overview of apple’s strengths:
• Master the underlying hardware and graphics technology: The Gpus for a-series chips are now designed by Apple, the Gpus for Macs are currently co-designed by AMD, Metal technology is now supported across all platforms (yes, Unity is now based on Metal);
Master AR hardware: mainly refers to camera and LiDAR, U1 chip, iBeacon/WIFI/ gyroscope and other indoor positioning and LAN communication related hardware. Apple doesn’t build the hardware itself, but it’s deeply customized or designed, with key metrics and experience in debugging.
Master AR tracking and 3D reconstruction algorithms: ARKit and RealityKit are based on these algorithms, which is one of the most core AR technologies of Apple. In Unity, ARFoundation is a repackage of Apple’s ARKit, replacing its own rendering technology.
It has its own software ecosystem and user base: Mainly based on the App Store and iOS platform, it has developed its own user base and accumulated a large number of well-known software and developers.
Master your own language and compiler: From exclusive OC to open source Swift, the Xcode compiler has been polished over the years and has been criticized for its stability, but it is good for its comprehensive support. And now Apple is working on Reality Composer, which is Unity.
Development framework linkage: Apple AR is in development, it can be easily integrated with its own framework, such as map framework MapKit integration, and formed this year’s Location Anchor, LAN communication framework MultipeerConnectivity integration, can support multiplayer games and so on. Cross-platform, as Unity calls it, is either handled separately, with iOS doing one thing and Android doing another, or it doesn’t provide any encapsulation and developers write native code and mix it up.
Then there’s apple’s downside:
The rendering technology is behind schedule: ARKit is developed based on SceneKit, and the rendering work is all done by SceneKit. SceneKit, while easy for iOS developers to get started with, has few built-in features, little external third-party support, and few tutorials compared to Unity. It is no wonder that Apple will abandon SceneKit and start all over again in rendering, creating a new RealityKit (in fact, it has not completely abandoned SceneKit, but no longer directly inherits SceneKit, which is a closed and improved version). But RealityKit is still not mature and is still far behind Unity.
Lack of 3D graphics development interface: Xcode’s XIB and StoryBoard in 2D is not good, and the Scene Editor in 3D is not good either, so Reality Composer is developed. Unfortunately, it is not mature now.
Goofy, but not open source: Many of Apple’s frameworks are highly encapsulated and almost goofy, which has the benefit of attracting a large number of developers and better focusing on the product itself. The disadvantage is that when developing large complex programs, the native framework support is not enough, need to expand their own, but at the same time the programmer level is not enough and difficult to manage. Unity is partially open source, Unreal is fully open source, and many of their third party frameworks are open source, making it easier to customize complex features.
To put it bluntly, Apple is strong in every way, except for the upper echelon of 3D graphics development tools and development ecology. Looking back at Unity, it also has a number of drawbacks:
Licensing issues: Non-professional individuals and small businesses can use it for free, but when the size or company turnover reaches a certain level, they need to buy it. In addition, better professional third parties also need to charge. The free version also has an open screen Unity AD display.
Package size: Whether it’s purely Unity-developed apps, or packaging Unity code as a Framework and putting it into a native iOS project, the size of the app package can be significantly increased, with any package around 100 MB. If an app like wechat needs an AR function and uses native ARKit to develop, it will only increase the volume by 10M, but with Unity, it may increase 100M or even 150M, which will make users seriously dissatisfied.
Poor performance: First, slow startup. In the case of my iPhone 8 Plus, the Unity project starts at least 2 seconds slower than a normal iOS app, and more than 5 to 8 seconds slower. If it’s a big game, it’s acceptable, but a regular app is so slow that it’s totally unacceptable when used frequently. There is also a large memory footprint, which is not obvious in the process of use, but in the background mechanism of the iOS platform, very uncomfortable, because the memory footprint is more likely to be killed by the system, coupled with slow startup, very painful to use. In addition, Unity will still consume a lot of system resources at low load, resulting in heat, battery life and so on.
Debugging difficulty: For common iOS projects, you can use Xcode built-in tools for debugging, breakpoint preview SCNNode, SceneView preview in bread graph, GPU Debug tool for debugging, etc., which is very convenient. While Unity itself has many debugging tools, on iOS, AR projects can only be tested on real machines, since the camera is on the phone, and Unity’s own tools are not available. At this time, the use of Xcode tools will be limited, only Unity project package as a Framework embedded in Xcode project, can use the GPU Debug debugging, and see the Unity rendering pipeline and related materials, and its own code structure is completely different.
For normal iOS/MacOS projects, click the Camera button to enter the GPU Frame Debugger
Pure Unity project without this camera button. As a Framework mix
Maybe those big companies like Tencent have enough talent, ability and experience to solve these problems, but as an ordinary developer, faced with Unity’s shortcomings, there is really not much to do. After all, Unity is not a fully mobile development platform project, and it is impossible to keep its advantages and optimize its performance to be as good as a native project.
AR development direction of the guidance: Google is also very strong, but a few steps early became a pioneer martyr, in THE AR aspect of Apple or play a leading role, Unity can only follow Apple/Google kept packaging;
Integration with other Apple frameworks: for example, this year’s geolocation anchor is the integration with our own MapKit, will it be integrated with Bluetooth iBeacon indoor location in the future? With a device with the U1 chip? And their own App Clips code with NFC fusion? In this case, cross-platform complexity increases and the significance diminishes.
Optimization of mobile platform performance: Apple hardware and software integrated, the main energy is also on the mobile platform, performance optimization is bound to be much better;
Lowering the threshold of entry for developers: The premise of the popularization of any technology is the lowering of the threshold of use, AR is the same, Apple brought silly operation, in order to attract a large number of developers, and then the formation of ecology;
So I think in terms of mobile AR development, native development is still going to be the focus, and Unity, like it is now, is going to be focused on games.
What should developers do?
So what do we do as an AR developer, or as an iOS developer who wants to move in the AR direction? Be optimistic and defend ARKit, or actively embrace Unity great Unity, which is the future direction?
My personal opinion is that iOS developers should start with ARKit, learn the ARKit API and vector matrix basics, learn about materials, rendering, lighting, etc.
Now you’re faced with a dilemma: you’ve learned the basic API, and then you want to learn more about Shader/Metal, only to find that there’s too little data and it’s too difficult. Is stuck here, become a skilled API porter? And then at 35, facing obsoletion. switching to delivery?
I think, next should switch, try to learn Unity development: a lot of information, the entry is relatively simple, visual development, timely feedback high sense of achievement; At the same time Unity development process of mathematics and geometry requirements to be slightly higher, can help us better grasp the relevant algorithm; Using Shader in Unity is extremely simple, and there are plenty of materials to help you get a better understanding of rendering.
So I think: Whether Unity can integrate AR development or not, every ARKit developer should learn Unity!
