We Built 252 Ways to Predict the Playoffs. The Best One Lost to the Simplest One.

Our winner picked 59.2% of postseason series correctly. Shown a season it had never seen, it fell to 48.9% — worse than a coin. A plain run differential, with nothing tuned at all, beat it.

The Sports Page · The Professor · how a good number goes bad

The question: can you build a number that tells you who wins in October?

We tried 252 of them. The answer is no.

Here is the surprising part. The failure was not that our metrics were bad. The best one looked excellent. It only fell apart when we showed it a season it had not already studied.

What you get from it: the single question that separates a prediction from a story.

What we were trying to fix

A reader raised a fair objection to measuring hot teams by run differential. A club that outscores people by 42 runs might have earned it, or it might have played the Rockies six times.

There is a standard fix. Add the average quality of everyone a club faced to its own run differential, then solve until all the ratings agree. Then blend that with how the club played over its last N games, and you have a measure of current form that does not reward beating up the helpless.

Two dials: how many recent games to count, and how much weight to give them against the full season. We tried every setting. Windows from 5 games to 60, weights from none to all. That is 252 different metrics, each one tested against every postseason series from 1998 through 2025 — 223 series in all.

First, the part that worked

Before the tuning, one plain result is worth your time. We asked how often each simple measure picks the winner of a playoff series.

Won-lost record: 50.9%. The standings, the number every broadcast opens with, are a coin flip in October.

Run differential does better at 57.8%, and the schedule-adjusted last twenty games manages 57.4%. Real, but modest.

Then we optimised, and it felt wonderful

Out of the 252, the best came in at 59.2%: sixty-five percent weight on the season rating, thirty-five on the last twenty games.

That is better than anything simple. Written up carelessly it becomes a finding, with a name and a chart and a claim that we had beaten the standard measures.

Two honest tests. It failed both.

A metric picked for fitting the past has to be tried on something it has not seen. We did that twice, in two different ways.

Hide one season, tune on the other twenty-six, predict the hidden one, repeat: 48.9%. Worse than guessing.

Or be kinder and more realistic. Tune only on seasons that had already happened, then bet the next October, the way anyone actually would: 56.4%.

Plain run differential over those same seasons: 58.3%. Under neither test did our clever number beat the boring one.

The tell we should have caught first

Look again at the crowd of dots. Seventeen of the 252 sit within a single point of the winner. The champion was not a champion; it was the tallest blade of grass.

Worse, the settings never held still. Tuned year by year, the best window jumped between 10 games, 20 games and 30 games, and the weight wandered from 0.35 to 0.90. If the ideal window is ten games in one season and thirty in the next, there is no ideal window. There is only noise, and we kept renaming it.

Why it had to happen

With 223 series, the uncertainty on any one of these rates is about six points either way. We were choosing among 252 numbers that all sat inside each other’s error bars, which is another way of saying we never had enough games to tell them aparti.

Take the maximum of 252 noisy numbers and you have mostly measured the noisei. Our winner beat the middle of the pack by four points; the error on any single one of them is larger than that.

And the ceiling was never high. Set the best measure in baseball, 57.8%, against the 75% we use as the point where a difference becomes noticeable, and it is under a third of the way from guessing to that line. To even prove 57.8% is not 50% would take about 332 series. Baseball produces eleven a year.

What to take home: ask what it had not seen

When somebody shows you a number, ask one question. Was it tested on data it had never seen?

If it was chosen for fitting the past, it is a description of the past. That can be worth having; it is simply not a prediction, and the two get sold at the same price.

You will meet this everywhere. A trading strategy that would have returned 40%. A diet study with the outcome picked after the data came in. A hiring model that is 94% accurate — on whom? Every one of them can be checked with the same question, and most of them have never been asked it.

We wanted the metric to work. That is exactly why we had to try to break it, and it is the part of this we would most like you to steal.