Held-Out Set

Every week an AI lab publishes a number. This channel goes and checks it.

I read the actual benchmark documentation, the actual leaderboard, and the actual announcement — then explain in plain English what was really measured, and whether it means what the headline says it means.

No hype, no press-release voice, no reaction takes. Sources appear on screen as they're used, so you can check me in real time.

A held-out set is the slice of data you keep back to find out whether a model learned something or just memorized the answers. Same idea here, applied to the claims.

New video every week. If you work in ML or research — or you just want the version that slows down and reads the footnotes — this is for you.

Corrections welcome. I pin them.