Augmented Reality made a huge buzz when Pokemon Go initially launched. Using simple 3D objects in AR View, Pokemon Go introduced millions of people to a simple AR experience. Apple recently announced ARKit at the latest WWDC. It didn’t really bring anything new from the existing Augmented Reality frameworks and SDKs. But ARKit is quite robust, free for developers and can run great on millions of iPhones. It has already gotten really popular with developers as we can see with examples on Made with ARKit. All these new ARKit experiences can actually make AR accessible to the consumer.
1 Visual Inertial Odometry: Used for tracking the real world data using a combination of Apple’s CoreMotion and camera data. It helps apps sense how the phone is moving in the room without using any calibration. This was decently implemented in the Wikitude SDK that was available a few years back.
2 Scene Understanding: Using this, apps can sense horizontal and vertical planes in the real-world. For example, an app can estimate where the table surface or wall is.
3 Lighting estimation: The camera is used to understand the lighting around the room so virtual objects placed in the room or outdoors can adjust their lighting & shadows giving it a better real world placement.
4 Rendering: Rendering 3D objects in ARView needs intense graphical processing. Apple had to limit ARKit to devices with an A9 or A10 chip for this. ARKit works great with rendering platforms like Metal, SceneKit and even Unity.
Although most of this tech has been in 3rd party SDKs for a long time, Apple has done a great job in it’s implementation. I saw some really robust demos with Google’s Tango platform at this year’s Google I/O. The only problem with Tango is that it sold on less than 100,000 Lenovo phones. Though it has a great 3d mapping & sensing functionality. When iOS 11 is releases in September, it will be in over 100 million iPhones by the end of the year.
Issues with AR
We have seen some interesting use cases for AR coming up in games, navigation, data visualisation and retail. Though their are some fundamental issues that it will face with smartphones. Battery issues- with intense processing power needed to detect and render a 3D scene & also the camera sensor. This causes major battery drain and the phone heating up. Also, all use cases won’t work well with low-lighting. Industry analysts like Robert Scoble predict the iPhone 8 will have a bigger battery and a better GPU to handle AR in a better way.
Will consumers pay?
While working on an ARKit side project, one of the biggest questions in my mind is that, will consumers actually pay for a cool experience rather than a big utility? The @madewitharkit Twitter handle has shared some really awesome demos showcasing the power and stability of the ARKit platform. With some big ARKit apps, major design & hardware upgrades, Apple should be able to push the new iPhones at a huge scale. Even though the final AR platform will probably be wearable, mixed-reality glasses, the iPhone will be the first major scale adoption of AR.
— Made With ARKit (@madewithARKit) July 12, 2017