Skip to content
Naked Security Naked Security

Sleep trackers: more of a data nightmare than a data dream?

Do you drift off each night with an app? Take what it says with a pinch of salt, says the researcher who tried out 10 of them

I like my Android phone just fine… but I don’t want to marry it and have its babies.

Nor do I want to sleep with it perched on my mattress. Many people do, though: Sleep as Android, the most downloaded sleep tracking app in the Google Play store, had been downloaded nearly 250,000 times as of May 16.

The app functions the same way as many other popular sleep trackers, be they smartphone apps or wearable personal devices (there are more than 500 options from more than 15 brands now available). The apps track a sleeper’s movements with an accelerometer, listen in on whether they’re snoring, and chirps at them to make them roll over if they are. It also tracks how much sleepers toss and turn, and how much time they’re lightly or deeply sleeping. Unlike other sleep trackers, this one promises to track even REM, which is the phase that’s the most difficult to detect.

But here’s what a researcher at Brown University wanted to know: is the data being collected by these devices and apps worth anything?

To find out, Jina Yoon, an undergraduate researcher, teased out data from 10 popular sleep tracking devices, then spent nine days conducting a minute-by-minute comparison.

As Yoon tells it, she wanted to do the comparative study as a followup to one done three years ago by sleep specialist and physician Dr Christopher Winter, Personal Sleep Monitors: Do They Work?.

As both Yoon and Winter have noted, only true sleep studies can determine the quality of sleep. Trying to do so with accelerometers is like trying to gauge how well a car drives by looking at its odometer reading, as Winter says. The gold standard is a study that combines brain wave tracings, muscle-tone evaluation and eye-movement analysis, along with a live audio/video of sleep that allows researchers to see exactly what happens during slumber.

That’s what Winter himself underwent. Yoon did not; she notes that it’s too expensive and inconvenient to do a polysomnography to detect REM sleep with a worn device or app. A polysomnography records brain waves, blood oxygen levels, heart rate, breathing, and eye and leg movements. Such a test requires the placement of electrodes on the head and is usually performed in a clinic for one night.

Instead, Yoon strapped on 10 of some of the most popular sleep trackers (including one research-standard actigraph as a control). She wore them for 10 nights and then graphed the data against each other.

Those were some miserable nights:

I wore all five wristbands on the same arm (Fitbit Alta, Jawbone UP, Microsoft Band, Microsoft Band 2, AMI MotionLogger), three phone apps placed side-by-side on my bed (Sleep as Android, SleepCoacher iOS, SleepCoacher Android), one phone app placed on my nightstand (Sleep Cycle iOS), one device clipped to my pillow (Hello Sense Sleep Pill) and one product that required two sensors clipped onto my pillow and placed on my nightstand (Hello Sense). It was, as expected, not very comfortable – which might be why my sleep patterns seem quite restless over the 10 nights!

How do you even pull data out of those trackers? Yoon published a data extraction guide for all the devices here. One of them, Hello Sense, had to be hacked a bit. With the help of her advisor, Yoon says, she and her helpers were the first to ever unlock the thing.

As you can see in Winter’s results, back in 2014, the data from various devices appears to be all over the map. But Winter didn’t trash any of the tested devices; they all contributed something to sleep and fitness activity tracking, he said, and “anything that makes you think twice about staying up too late is a good thing,” he concluded.

Fast-forward to 2017 and Yoon’s study: how are the trackers doing in terms of accuracy?

Pretty much the same. Yoon notes that the data was “better than random,” with most devices recording similar findings if they’re based on the same type of sensor (accelerometer vs noise) and are the same device type (phone vs wristband vs other).

Ultimately, sleep is best tracked through polysomnography, Yoon says, but these trackers are good for those who can’t afford or don’t care to spend time in a clinic, strapped with electrodes.

But here’s the data privacy question: given that these things are not exactly the peak of scientific accuracy, then it’s reasonable to think about whether the trade-off – interesting but not particularly reliable stuff about your sleeping patterns vs. where the hell does all that data go – is worth it?

Yoon herself says that reports coming from consumer-level trackers should be “taken with a grain of salt”.

These devices offer some powerful data tracking tools, but it’s best to let the experts analyze your sleep if you have any chronic conditions.

Both Yoon and Winter before her still came to a similar conclusion: the researchers’ thinking goes that sleep trackers are great, given that anything that helps you to get a good night’s rest or reminds you to avoid staying up late is a positive thing.

But remember: personal data, once collected, is out there for good or bad. We’ve seen Fitbit data compromised, for example.

Is data compromise worth the risk, if it means your data gets pulled into, and may be exposed via, the gloriously security-addled Internet of Things (IoT)?

My pillow will remain bereft of gadgetry, though the lullabies of whalesong and the morning alarm of birds chirping offered by that Android tracker do appeal.

Do you use a sleep tracker? Please do let us know in the comments section below how you like it and what you think of being tracked while you’re sawing wood.


So is “better than random” really the only result she got? How did they fare against the control? Any useful information?


I’ve tried a number, and found a few of the accelerometer + sound ones to be somewhat reasonably accurate as far as tracking when you’re awake and when you’re asleep, and their “best guess” as to when you’re in deep sleep is so-so. REM sleep, they’re all over the map.

Basically, they’re worth it if they help you see your wake/sleep patterns and when you get your best sleep; this could be done almost as well with a pad of paper, noting when you go to sleep and when you wake up, but the apps include an alarm to wake you up at a reasonable point in your sleep cycle (if you’re moving around, the alarm goes off earlier) and a pretty good indicator of when you fell asleep.

When I was looking, I checked a few to see which ones were attempting to connect out through my firewall or needed “cloud” access to operate. Those ones got binned immediately, leaving me with a few apps that just communicate data back to the iOS health app and warn me if I’ve been displaying any unhealthy sleep patterns.


I use one. I subscribe to the “anything that encourages you to try to sleep is a good thing” logic and while I already take the light/deep sleep determinations with a pinch of salt, the tracker I use has a heart rate sensor to aid in that determination and a battery life of over a month. The data is of interest to spot trends and helps me recognise the times when I am generally overdoing it at work.

I don’t really want that data all over the Internet, but as I enter rubbish into the app anyway, it is as good as anonymous anyway.


The sick irony that seems to go unmentioned here is that the better way to get quality restful sleep is to sleep NOWHERE near your cell phone (or any other electronic device, especially the ones that constantly bombard you with Electromagnetic Radiation, which has been demonstrated as far back as the 1970s to cause DNA damage, Oxidative Stress, and damage to the testis .
You NEED to put that thing on airplane mode if you have it anywhere near you while you sleep. Otherwise you are literally sleeping next to a high tech computer which is constantly transmitting and receiving radio frequencies that are known human carcinogens.
Turn off your wifi at night, when you go to sleep. Like the wifi router… unplug it. You have no need to be radiated while you sleep, so just unplug it. I dare you. See how much better your sleep is now? Yup, it’s called an invisible assault that is happening to us right now, a huge experiment. Stay away from cell phones, wifi routers, and EMF devices of all sorts as much as possible. And please, if you are pregnant, please look in to this, and please try not to subject your fetus to wifi radiation in utero – it’s not good. It’s really, really, really bad.

Thanks for reading this far; I hope someone might take action from this message.


I tried some mobile apps, and you can get an ok baseline. I’m using an Oura ring now. It’s a little bulky, and some will not like this. They do send a ring sizing kit with plastic ring replicas to get a feel for the size.
The product is for tracking sleep. The pairing app offers more extensive and useful data points than I’ve seen in the few other products I tried. Oura touts that their sensors are better than other products, but who knows.
It’s still early on, but the product is not unpromising. Be warned though. It isn’t cheap, but it doesn’t feel cheap either. Just to note, I see a sleep specialist, and he did recommend this product over everything else, but he could get by with FitBit data. I didn’t go with FitBit because, at least at the time, their sales were declining, and I didn’t know if they would last as the wearables market didn’t take off as hoped.


Hey Lisa, with Jawbone being bankrupt it seems millions of people’s sensitive sleep data, will be sold to data brokers, that will further resell personal data to life insurers, etc… According to their privacy terms “We may share your personal information for the purposes of a business deal (or negotiation of a business deal) involving sale or transfer of all or a part of our business or assets. These deals can include any merger, financing, acquisition, or bankruptcy transaction or proceeding” this will happen. Doing a google search on privacy, personal data and Jawbone, I noticed that nobody seems to care. Any opinion on this?


Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to get the latest updates in your inbox.
Which categories are you interested in?
You’re now subscribed!