Karpathy's Sleep-Tracker Experiment: Two Months, Four Devices, One Winner?

It started with a video watched two months earlier: Bryan Johnson explaining how he fixed his sleep, and Karpathy resolving that day to improve his own. But first it had to be measured, because you can only manage what you measure. At the time he only had an Apple Watch paired with AutoSleep; an Oura he once liked sat aside, and a new 8Sleep Pod 4 Ultra was on order. The question was plain: which of the four candidates is actually trustworthy? He launched a two-month comparison project and published the results on 24 March 2025.

The four candidates could hardly be more different. Whoop is a screen-free wristband, the Apple Watch was tested through a third-party app (AutoSleep), Oura is a ring worn on the finger, and 8Sleep is the bed itself. The last one promises tracking plus temperature control but cannot measure the pulse directly, since it sits under the body rather than on it. So the race had three wearables and one non-wearable.

The method was simple and stubborn: every morning for sixty days, he logged the numbers right after waking. Light, REM and deep sleep plus time spent awake; resting heart rate and heart-rate variability; each app's sleep score. Then he set those numbers next to how he felt that morning. The raw spreadsheet is public, so anyone can redo the math.

After two months the verdict was intuitively clear: Oura and Whoop are tier one, their scores mostly in line with how mornings felt. 8Sleep landed in the middle as merely fine. The Apple Watch plus AutoSleep pairing disappointed; its scores swung almost randomly and barely tracked morning feel.

The score histograms sharpen the picture. AutoSleep and 8Sleep give generously, hugging the 100 ceiling; Whoop hands out 100s regularly too. Oura is stingy: its peak over two months was 92, the distribution looks bell-shaped and the range is wide. Karpathy counts that as a virtue, since a score glued to the ceiling leaves no room to improve.

He also measured how much the devices agree with each other. Whoop and Oura topped the list at 0.65. Oura with 8Sleep came in at 0.59, Oura with AutoSleep at 0.47, 8Sleep with AutoSleep at 0.42, Whoop with 8Sleep at 0.38. At the bottom sit Whoop and AutoSleep: 0.14, essentially no relationship. If Whoop is taken as good, AutoSleep looks like a noise generator.

On heart rate the picture flips. Oura, Whoop and AutoSleep agree closely on nightly pulse, with correlations between 0.88 and 0.95. 8Sleep, being a mattress, is naturally the weakest link yet still surprisingly close. The link between resting pulse and sleep score is thin: a faint 0.13 tilt on the Whoop side, nothing on the Oura side. A lower pulse does not reliably buy a higher score.

Training was logged too: about half an hour of easy cardio daily, plus intense four-minute intervals twice a week. Over sixty days resting pulse improved by a bit less than 3 bpm. That is the charming side of the experiment: the numbers reflect not only the devices but the body's answer to training.

The video's author, Aladdin Persson, pushes back, and from an informed place: he has built an app for the same need himself. Correlating with morning feel is no scientific yardstick, he argues; the right question is the objective counterpart of each sleep stage. Worse, the Apple Watch raced with a mediocre third-party app rather than its own software. Burying the watch because native tracking ships no score is unfair, since deriving a score from the raw graph is easy work; what deserves attention is the trend and consistency of the raw numbers.

Enter Rob ter Horst, known as the Quantified Scientist. A PhD in biological data analysis, he tests wearables against lab-grade references (nightly head EEG, ZMax, Dreem 2, polysomnography). His finding runs against Karpathy's: Whoop is not that good, the Apple Watch is much better. Apple's sensors rank among the best on the market in his view; the fault lies with the app interpreting the watch, not the watch. His summary: Oura leads in sleep, Whoop in sport, the Apple Watch is solid at both.

The closing makes the everyday case for measuring: the author traced worsening variability and resting pulse to pollen allergies and acted on it, and reads training returns off the numbers. He finds Bryan Johnson's Don't Die movement a little extreme but interesting. He himself wore Whoop and Apple Watch together for five months, then gifted the Whoop away and stayed with the Apple Watch. The line I keep: measuring works, provided you know what you are measuring with what.