Really, Facebook, I simply can’t make it any easier for you to suss out my emotional state. Have I not already mentioned, for example, my heartstrings, in a comment on BenBen, the Saddest Cat on the internet?
Sappy sadsack cat comments aside, Facebook has filed a flurry of patents that seek to figure out our emotions.
One of the patents, titled Augmenting Text Messages with Emotion Information, concerns fancying up text messages to fit our moods, which would be interpreted via how we peck away at our phones.
For example, what form of text input is being used? What font are we using? Where, exactly, are you? Sunny beach, or moldy dungeon? Could affect your mood. What’s the typing speed? Machine-gun rat-a-tat-tat or laid-back Swyping? And how much abuse is the phone taking? Vicious jabs or loving caresses? The phone knows: just ask it for touch pressure and/or what’s going on with the accelerometer.
Match up one or more of those characteristics of keyboard input with one or more words in the text message, and presto! If Facebook’s plans pan out, it will be able to determine whether we’re laughing, crying, terrified (as in, lunging for sticky notes to cover our gadgets’ cameras, for which it also has emotion-gleaning plans) or filled with rage (as in, making our smartphones’ accelerometers quiver as we shake at the notion of Facebook invading our headspace).
In short, Facebook plans to add features to texts to reflect the feelings of the sender. That could mean adapting the visual format of the text message in real time based on a user’s predicted emotion. You can see where this might come in handy: as the patent notes and many of us have experienced, despite the universe of emoticons out there, it’s still tough to convey everything you want to say in a brief text message. 😻 😳 😸
Here’s an image of how it plans to augment text messaging with the emotions-mixed-with-text-content technology:
All the better to look you over, dearie
And then there’s Facebook’s other proposed emotion-reading patent, titled Techniques for Emotion Detection and Content Delivery. In this one, Facebook plots out how it’s going to have its way with our cameras, be they in laptops, mobile phones or tablets, be they activated or even just passively sitting there, waiting for somebody like Facebook to come along and watch us with what it calls “passive imaging.”
Most devices nowadays include at least one camera, some of them forward-facing. Why not use them to observe us as we peer at our screens, reacting as we do to streams of content by outputting our own streams of emotions or invective?
All the better to target content at us, Facebook proposes in the patent:
A need exists for a content delivery solution that takes advantage of available passive imaging data to provide content to a user with improved relevancy.
Faceook’s proposing a system that would capture a user’s image, identify the user’s emotion and store the data. Then, Facebook could determine what emotions a given piece of content elicits and use the data to target users.
Facebook described yet another emotion-gleaning technology last month: one that would generate emojis based on the facial expression of a user.
You can see below an example of how that might be used: in the drawing Facebook included with the patent, a user will take a selfie, and the system would produce an emoji expressing the same emotion and gesture.
So much Facebook attention to so much touchy-feely. Of course, when it comes to recognizing emotions, you’re talking marketing gold.
Back in December 2015, when Microsoft was training facial recognition apps to recognize emotions, Ryan Galgon, a senior program manager within the company’s Technology and Research group, suggested that developers might want to use these type of tools to create systems that marketers could use to “gauge people’s reaction to a store display, movie or food.”
He also suggested exactly what Facebook’s patents are now describing: consumer tools that recognize people’s emotions and cater to them, ka-ching!
Do we trust Facebook to use our devices’ cameras to watch us? As I recall, that’s what creeps do: talk women into “fixing” their gadgets by taking them in the shower, then triggering video or stills capture.
I’m sure Facebook isn’t up to something quite that lascivious. But I can’t help but think about that stunt it pulled in 2012 with its “emotional contagion” research.
As you may recall, for one week, Facebook researchers intentionally modulated the news feeds of Facebook users.
Not “passively monitored”, mind you; rather, actively manipulated.
Some saw a dash more positive items in their feeds; some received a more grim daily dose, as the researchers snipped out happy tidings, all of which led to the conclusion that yes, emotional states are contagious, and no, seeing friends post happy news does not necessarily make people want to jump off ledges.
Emotionally manipulating uninformed Facebook users – or, as the New York Times referred to the 700,000 subjects at the time, the “lab rats” – stirred up an ethical, legal, and philosophical hornet’s nest, as fury erupted from politicians, lawyers, and internet activists.
Of course, this is all just patent talk. A Facebook spokesperson mailed a statement to that effect to Mashable, saying that the company…
…often seek patents for technology we never implement, and patents should not be taken as an indication of future plans.
Let’s hope that if these patents do lead to the technologies described, Facebook will put a hell of a lot more thought into how it will play off our emotions than it did in the 2012 experiment. In the meantime, here’s how to secure all those cameras that plenty of hackers already know how to use for “passive imaging.”