Google AI lets phone owners know about shoulder surfers

Phones. Pffft! They’re so passive. They just sit there, warming your sweaty palm in a crowded subway or elevator as mouth breathers shoulder-surf your PINs or swipe patterns – or your texts, or what should be your private, confidential web browsing – out from under your nose.

But not for much longer, if two Google researchers have any say about it! They’re working on a machine-learning, facial recognition system that will pick up on when some nosey snoop – one who’s definitely not you, given that it’s a facial recognition system and knows what its master or mistress looks like – is staring at your screen.

Researchers Hee Jung Ryu and Florian Schroff show in a demo use case video that when the system detects that somebody’s gaze is pointed at a phone, a Hangouts messaging conversation will come to a screeching halt.

As a front-facing camera detects strangers looking at the screen, it will interrupt the conversation, show both the phone owner’s and the shoulder surfer’s faces, put a red bounding box around the stranger’s face, and then flash a warning in full-cap red letters:

A STRANGER is LOOKING ALERT!!!

On top of that, it will show the stranger vomiting a rainbow. With sparkles. Alarming, but pretty!

Besides rainbow effluent, the system’s got a mouthful of a name: according to the demo description page, it’s called the Electronic Screen Protector with Efficient and Robust Mobile Vision.

Technical details are scarce. It looks to be an academic project at this stage, rather than an upcoming feature. The Electronic Screen Protector is due to be presented at the Neural Information Processing Systems (NIPS) conference next week in California.

The researchers do note that using facial recognition – which has typically been used for authentication – to protect privacy is a logical extension.

Face recognition alone is not enough when you want to have private online conversations or watch a confidential video in a crowded space where there are many other people present. Each of them may or may not be looking at your private content displayed on your device, e.g. a smart phone.

They said that the Electronic Screen Protector is fast, robust, and accurate at detecting gaze and face identity simultaneously, in real time.

Hence, the application, an electronic screen protector, can enable its enrolled users to continue reading private and confidential contents on [their] mobile device, while protecting their privacy from onlookers in a crowded space such as the subway or an elevator.

The researchers also say that the system is robust under varying lighting conditions and head poses.

Well! It’s high time that somebody thought to teach a phone self-defense, wouldn’t you say? Lord knows we humans manage to screw it up.

We’re terrible at creating and remembering secure passwords and PINs, for one thing.

We’re also bad at choosing and answering password recovery questions.

Most of us can’t even cook up an unlock pattern for our Androids that’s not crazy easy to predict, be it by shoulder-surfing or the tell-tale streaks we leave with our greasy fingers.

Bring on the face-detecting, gaze-tracking, alarm-sounding phones that can protect themselves – sparkly rainbows and all!