Win Probability for NFL Games: How Accurate Is It?

If you have been watching the NFL, you have probably heard some reference to win probability (or WP). It most likely came up when something really crazy happened, like when the Jets won the opener on an improbable penalty, or at the end of the Ravens-Vikings game when the teams combined for five touchdowns in the last 2:05. Maybe it was in reference to the Dallas Cowboys, who seem to make the improbable happen all the time.

How accurate is win probability? To test, I looked at a specific point in time for every game in the NFL this year, and logged who was favored to win  and by what win probability according to pro football reference. The Pro Football Reference win probability uses current margin, remaining time, and expected points on the possession at the point in game being measured, and also uses the point spread. Thus, a heavy favorite is seen as more likely to make a comeback, or win a close game, than a heavy underdog would be.

The point in time I used was the start of the 4th quarter for each game. There were three games that were at 50%. We also had the Packers-Vikings tie. Here is a summary of every other game, grouped by how heavy of a favorite the team seen as most likely to win was.

There are several things to take from this. Almost exactly half of the games were seen as one team having a 91% or greater chance of winning by the start of the final quarter. Most NFL games feel closer than that, right?

Let’s give some examples. On Thanksgiving, Dallas was tied with Oakland at the start of the fourth quarter. However, they had the ball and a 1st and 10 inside the Oakland red zone, and were also at home as heavy favorites. The WP estimate was 87% from Dallas in a game that was tied. Washington was also tied with San Diego, but inside the 10 at the start of the 4th. They were not favored by nearly as much, and were given a 78% chance of winning.

Less than 10% of all games were seen as even being close enough to call them 60%/40% or closer at the start of the fourth. That might seem like some very aggressive forecasting to take “strong positions” on the likely winner.

So far in 2013, the win probability estimates (at the start of the final quarter) have been almost exactly on target for games where the leader was viewed as having between a 51% to 90% chance of winning. These teams were collectively expected to win 78.8 games, and they won 78.

At the extreme levels, there have been more comebacks than win probability would predict. Of the 111 games where one team was given less than a 10% chance of winning, 9 have come back to win, when only 2 such comebacks would have been expected. The most improbable was Seattle coming back to beat Houston thanks to this Matt Schaub pick six while playing with the lead. Seattle was down by 14 points, and had the ball at their own 1 yard line, when the last quarter began, and were given a 1.5% chance of winning because of the combination of large deficit and bad field position.

A pick six when leading that late is a rarity. Of course, that win estimate was based on the teams being close (Seattle was only a 2 point favorite at the time), though that perception doesn’t match the rest of the season, where Houston is in line for the first overall pick.

The second least likely win was the Bengals vs. the Packers. Green Bay had a nine point lead and had the ball on Cincinnati’s side of the field when the quarter began, and Cincinnati was given only a 2.6% chance of winning at that point. An interception a couple of plays later began the swing. Of course, this one took another bizarre play involving a double fumble when the Packers went for it on 4th and 1 with the lead, to ultimately swing the outcome.

So overall, pretty good, but more extreme outcomes. Is it bad luck, some bizarre games, or is the system slightly overconfident at the extremes?

I don’t know, but my guess is the win probability model does better than the average fan, or even the average coach, at properly assessing the likelihood of winning (and for coaches, thus determining strategic decisions like punting vs. going for it or kicking).