Navigation
  • Home

Sleep research is always terrible

Reporters don’t seem to understand that sleep research is even worse than nutrition research. When the Atlantic published a new article claiming that 9-5 work was bad for creatives, I knew the research “backing this up” would turn out to be terribly performed:

Study 1 (n=20) 1 day of observations

Study 2 broken link

Study 3 (n=21) 6 weeks of self-reporting

Study 4 (n=15) 3 days of testing

Study 1 – Really? 1 day? That’s your study design when you’re trying to figure out how much lack of sleep leads to fatigue? This is like having a drug study with no washout period.

Study 2 – If I were a sleep researcher, I’d probably want to break my web server too so no one could critique my terrible work.

Study 3 – Diaries and self-reporting. That will be a cheap study to run! Oh look: 47% of the variation in students’ grades were determined by how much they slept two weeks before exams. Wow — what a profound sounding result! But the correlation of “grades” to sleep only applied to one class. The researchers actually collected all the students’ grades and when you look at all classes, the correlation evaporates. I guess peer review can’t block things like this since your peers do things like publish 1 day sleep studies on chronic fatigue.

Study 4 – Let me get this straight… you measured 11 days of data but published your results based on only the first 3? Go home sleep researchers, you are drunk!

See on www.theatlantic.com

Author Description

Louie Helm is a Machine Learning Engineer

4 Responses to “Sleep research is always terrible”

  1. December 16

    Will Sawin

    Maybe the researchers need to get more sleep.

  2. December 16

    Rethink Evrything

    Haha, I’m so glad you posted this 🙂

  3. December 17

    Amit Amin

    Beautiful.

  4. December 18

    Alexander Macrae

    You can hit an almost awake zone where a Gestalt form of thought takes over. About 3 am in my experience.