Sleep is essential, without sufficient sleep – and more specifically, sufficient deep sleep – you might get into real troubles. So it seems a godsend that activity trackers and smartwatches give you a detailed analysis of your sleep. But one important question arises: how accurate is this kind of tool? Since I have a bunch of them, I did the test. And as the reference, I used a device equipped with EEG-sensors, which uses brainwaves as input for sleep analysis, compared to movement and hearth rate. And that seemed to be quite interesting…
Analyzing sleep isn’t easy: give the same readings to two different sleep experts and you might get (slightly) different results. There is a lot of interpretation involved. In a sleep lab, the amount of information is enormous: the amount of wires attached to your body (head, legs, fingers) makes you look like spaghetti. Can one fit all of this in a tiny device? A device that might even cost you less than 100 euro/dollar?
While sleep labs have a very diverse amount of input, including the reading of brain waves – the most important input in this case – activity trackers and smartwatches only use a small amount of data. Some are solely based on movement, most also include heart rate and heart rate variability.
Having had a severe fatigue issue, which was partially linked to a lack of deep sleep, I’ve invested in different tools over the years. Five of them offer sleep tracking, showing the different sleep phases. Two are general activity trackers (an older Garmin Vivosmart 3 and the more recent Honor Band 5). SleepOn is a tracker for oxygen saturation and apnea detection. DreamOn is a device that uses pulses for ‘brain entrainment’, to get your brain to exhibit ‘Theta’ brainwaves, which help you relax and sleep. And then there is the Dreem headband. This is a very different kind of tool: for starters, it is a headband. And it uses EEG-sensors to measure brainwaves for a more accurate sleep analysis. Next to that, it also stimulates deep sleep through ‘pink noise’. Dreem is an interesting company, that wants to revolutionize the way we look at sleep. And in that endeavor, it has done extensive testing, including comparisons with sleep labs. It is achieving 95% comparable results, which is probably quite close to the differences between different sleep experts.
The test
For the test, I put on all devices and not only for one or a few nights, no I did this for 18 nights in a row, to get enough data to have a clear comparison.
(in case you want to see a picture of me wearing all the devices, sorry, there are none)
In the background, the gray, you see the amount of deep sleep according to the Dreem, which is my golden standard in this test. As you can see, it fluctuates a little bit, which is of course normal. If you look at the other devices, they don’t seem to agree with Dreem…
Before jumping into conclusions, I should add something: absolute agreement (i.e., exact the same number of minutes in deep sleep) isn’t necessary. What is required for people who want to track their sleep, is repeatability: the same ‘input’ should always deliver the same numbers. If that is e.g., 15% or 15 minutes off, that doesn’t matter that much. As long as you can see how your sleep evolves, it’s ok. That might enable you to find patterns.
With that in mind, I also created graphs showing the deviation compared to the Dreem readings, both in percentage and in minutes.
And as you can see: no one device has similar results to the EEG-based Dreem headband.
In case of SleepOn, I should add that it seems to work properly for detecting oxygen saturation and sleep apnea, but not the different sleep phases.
A few years ago, before I had the Dreem headband, I made a comparison between the Microsoft Band 2 and the Fitbit Charge 2. At one time, I had the opportunity to compare the Band 2 with a medical-grade device, for three days, the doctor concluded that the results were comparable. And the amount of deep sleep the Band 2 showed, was in agreement with my feelings, my energy levels. Since the Band 2 had hardware issues (in this case the band itself showing cracks, other devices also had battery problems), I had to replace it. But I had the change to use both of them together for a few days. And this shows the disagreement between both devices…
Microsoft – unfortunately – decided to get out of the smartwatch market. There was a prototype for a Band 3, but it never made it to the market.
Yes, but…
I hear you: I only tested low end and niche devices. But while I was working on this article, I got a present at work: the Samsung Active 2 smartwatch, a high-end device. After only three nights, I can assure you that it’s not accurate. E.g., in two from the three nights, it had missed the first 1,5 hours of sleep (which is btw the part where you are in a deep sleep). The other night, it recorded two sessions. I will continue to compare the Active 2 with the Dreem, as soon as I have sufficient results, I will update this story.
UPDATE 28/12/2019: below are the figures from 11 consecutive nights with both Dreem and Samsung Active 2. Next to the graph with the amount of deep sleep in hours, I’ve again added the deviation compared to the EEG-readings, both as a percentage and in minutes. The conclusion? Again it shows that wrist based sleep trackers aren’t reliable.
(for the record: I’m very happy with the Samsung, but I don’t use it for sleep tracking)
Conclusion
I would have loved to give you a very different outcome, one in which I could recommend a cheap but reliable device. But I can’t. None of the tested devices, not even the high end Samsung Active 2 (added 28/12/2019), is in agreement with EEG-based readings. And please note this was a long-term test, to eliminate ‘lucky shots’.
This has a serious consequence: if you need accurate sleep tracking, whatever the reason might be, you will need a device that’s based on EEG-sensors. There are a few on the market, Dreem is the one I’m using.
When you look at their website, you will notice it’s much more expensive than an activity tracker, even more expensive than many smartwatches. But let’s make a small calculation: in Europe you get a 2-year warranty, which means that you can at least use it for over 700 nights. Which makes the cost much less than 1 euro/dollar per night… (that’s how I convinced myself ? and I have never regretted making that investment) If you’re serious about tracking your sleep, you need to get this kind of device.
PS: being a Dreem-user, I can offer you a discount code: DTUOXLXWD
(for the record, this will also give me a small discount on my next purchase, but even without that incentive I would strongly recommend Dreem; I published my first, quite positive evaluation long before they started with this referral and discount program)