Is the Full Model Worth It?
A coin flip already gets a prediction most of the way there. So why does this site bother running a full goal-scoring model for whatever's left over?
The problem
Predicting a hockey game means putting a probability on three outcomes — home win, draw, away win — before it's played. Whether that probability is any good isn't obvious just by looking at it: a model can sound confident and still be wrong constantly, or hedge its bets and still be the sharpest one on the ice. Before trusting a model with real complexity behind it, it's worth asking plainly: does that complexity actually predict better than something simpler, or is it just more machinery?
How we grade a model
Every candidate below is fit on the same historical games, then scored the same way: how close its predicted probabilities landed to what actually happened, using the Ranked Probability Score (RPS), a standard tool for 3-way sports forecasts. RPS treats home/draw/away as ordered — a draw is a near-miss on either side, not an unrelated third bucket — and it punishes overconfidence: a model that says 95% and loses pays a much bigger penalty than one that hedged at 60%. Lower RPS is better, and RPS = 0 would mean predicting every outcome with total certainty and always being right — impossible here, since hockey games have real, irreducible randomness. Even a model that knew every team's true ability exactly would still lose sometimes, and would still score above zero. We turn each model's RPS into a Skill Score: 0% means no better than blind guessing; the chart below climbs toward a 100% summit that no model can actually reach.
The climb toward perfect prediction
Four genuinely different models — not stages of one pipeline — each fit independently on the same games. They differ in how much information they use and how they're fit; only the tallest bar, Full Dixon-Coles, runs on this site today. None of them, including that one, can ever reach the summit.
Ignores the teams entirely — every game gets the league's overall home/draw/away rates from the training data, regardless of who's playing. Uses no team information at all. This is the no-skill floor everything else is measured against: a Skill Score of 0%, by definition.
Gives each team a single strength number, fit by maximum likelihood from wins, draws, and losses alone — it never sees the score, just who won. Uses match results only, no goals. Already closes most of the climbable distance to the summit.
Nudges each team's rating up or down after every game — bigger wins earn bigger nudges — the same way chess ratings work, updated game-by-game rather than refit from scratch. Uses results plus how big each margin was. A small further step up the hill.
Fits every team's attack and defense strength jointly from the full season's goal counts, producing a complete probability for every possible final score — not just win/draw/loss. This is the model that actually predicts games on this site — the highest anyone's gotten so far, still well short of a summit nothing can reach.
So is that last 2.1% worth building a full goal-scoring model for? Most of the time, honestly, no — if all you want is a coin flip that knows the teams' names, win-loss records plus goal margins already get you almost all the way there. But this site's harder jobs are the ones where that last 2.1% is the whole ballgame: seeding a pool-play bracket, separating 3rd place from 4th in a tight standings race, ranking Parity Tournament teams that all look interchangeable on paper. Those aren't blowout problems — they're close-teams problems, and close teams are exactly where the extra complexity earns its keep.