Shashwat Pradhan Shashwat Pradhan

Privacy & Security with Augmented Reality

Apple’s ARKit and Google’s ARCore is helping developers bring Augmented Reality experience to millions of smartphone users. AR software has really grown in user adoption and also is now being used in new verticals. Along with AR & mixed reality headsets, we are also seeing new smartphone software experiences coming up like AR medical applications and enterprise software. With Augmented Reality apps & hardware having complete access to our visual and audio data in our surroundings, security & privacy can be a crucial aspect to think about.

Privacy Risks

Computer vision technologies have really matured, we’ve seen cryptocurrency wallets getting hacked through blurred QR codes of wallet keys. People and even facial expressions can be identified through Computer Vision technologies. On the other side, audio conversations can be monitored for private and confidential information like passwords. I definitely wouldn’t feel safe wearing an AR Headset like the Google Glass and entering my internet banking password as it streams my video feed to a cloud server. The camera permission granted to any app is enough to record people’s lives without them even knowing.

The latest Google Glass being used by some of the biggest enterprises for hands free applications

Apart from all these privacy risks, security should be an important part of AR/MR platforms. With critical medical or driving applications of AR glasses, malware can cause harm towards human life. There need to be security protocols built in these devices so that people can rely on them.

Possible solutions

Tech leaders like Mark Zuckerberg keep their webcam covered with tape on their laptops for privacy. Similarly, a private mode, where wearables don’t record data or upload it to the server can be a consideration while designing a good AR experience. Companies are designing AR headsets for users to wear throughout the day, selected filtering of private content being uploaded to the cloud would be a good feature to have. On the other hand, new tech in mobile phones like the AI processors can really help solve this problem. Rather than sending all video & audio data to the cloud, AR headsets could do a lot of this audio/video processing locally on the Neural Processing Unit. For example, applications like image recognition or language translation can be done offline on the AR headset directly. It would be faster, private & secure since no data would need to leave the device. Apple is also using a Neural Engine in their latest A11 chip which is in the iPhone X. It powers the facial recognition feature which allows the iPhone X to detect and authenticate the user’s face in a matter of milliseconds, also maintaining the user’s privacy.

Apart from the big social challenges that AR glasses bring to the table, it is important that we start weighing in the privacy risks. With the new iteration of the Google Glass being used at healthcare, medical and other enterprises that give it access to confidential data, it essential we have a proper security & privacy framework built-in.