A Look At Players Where ESPN's QBR Differs From Passer Rating
By Jason Lisk
ESPN is trying to selling advanced analysis to a broad audience, and the broader audience doesn’t necessarily want to see all the details, just whether the results make sense. As I said yesterday, it has a lot of the principles of things like DVOA or Advanced NFL Stats EPA and WPA. I looked at the QBR for the 33 quarterbacks who threw at least 200 passes in 2010 and appear on the QBR list, and also looked where each ranked in the official NFL passer rating formula, adjusted net yards per attempt (which includes sacks, but not completion percentage separately from YPA), and the three advanced measures–Football Outsiders’ DVOA, and Advanced NFL Stats’ EPA/play and WPA/game.
Here are the correlation coefficients between this new QBR and these five measures, using the ordinal rank of quarterbacks in each category–remember, anything over +0.7 is pretty directly correlated fairly strongly. As we will see, the measures are pretty highly correlated, which makes sense. Tom Brady was really good by any measure, and we didn’t need an ESPN special to tell us that Derek Anderson, Brett Favre, and Jimmy Clausen were brutal in 2010.
QBR and Football Outsider’s DVOA: +0.89
QBR and Advanced NFL Stats’ EPA (expected points added): +0.87
QBR and Adjusted Net Yards Per Attempt: +0.83
QBR and Passer Rating: +0.83
QBR and Advanced NFL Stats’ WPA (win probability added): +0.76
While WPA may be the lowest of that group standing alone, it’s pretty clear that adding it into the mix explains some of the rankings changes for QBR. WPA, or Win Probability Added, is the more volatile stat compared to EPA, because it depends on high leverage situations. A quarterback makes a big touchdown pass late to win it or throws a late interception in a close game, the WPA will reflect that swing in game winning probability for his team. David Garrard’s hail mary touchdown, for example, shot his WPA up (where he ranks 8th) compared to his EPA (19th).
It also makes sense that EPA and DVOA have the highest correlations, since they are also based on play by play data. If I take the average ranking in EPA, DVOA, and WPA, and compare it to QBR, the correlation coefficient rises to +0.92.
But ESPN is going up against the established Passer Rating, so let’s compare the players with the largest difference in where Passer Rating ranked them in 2010, compared to QBR.
Eli Manning (7th in QBR, 18th in Passer Rating). This is an interesting one, because QBR is basically on an island with this one. Passer Rating is always going to underrated passers like Eli, who throw downfield (thus harming comp%) while also not taking sacks. The thing is, the other rankings all had Eli closer to average as well in 2010, as he threw a high percentage of interceptions. My guess as to why QBR differs: (1) sacks, and we know that he and Peyton rarely take long sacks, something the video may have credited Eli for, (2) many of the interceptions may have been faulted to the receivers more heavily, as there were many that hit his guys’ hands, and (3) air yards, we know they are measuring this, and he would get credit if he is completing deep throws with less YAC.
Mark Sanchez (18th in QBR, 28th in Passer Rating). QBR says he was near average, while Passer Rating says he was bottom quartile. Sanchez doesn’t complete a high percentage, and is decent at avoiding sacks, so the passer rating will tend to under value him. Meanwhile, he was 10th in WPA because of several overtime victories and late game heroics, and my guess is that “clutch” bonus akin to adding in WPA boosted his rating to average.
Matt Ryan (3rd in QBR, 12th in Passer Rating). This was the one that generated the most buzz. Keep in mind that Ryan was 2nd in WPA in 2010, so his best play tended to occur at crucial times in close games in the regular season. His yards per attempt was also lower than his play by play efficiency, as he had a tendency to pick up just enough yards to keep drives going at a good rate. Combine these two together, and Ryan jumps in 2010. His EPA was 10th, and I think that further reflects where people put him (and since EPA is more stable, probably better reflects what he will be in 2011).
Peyton Manning (2nd in QBR, 11th in Passer Rating). Peyton’s going to benefit from any system that accurately measures how infrequently he takes sacks and keeps plays alive. He had a heavy interception penalty in passer rating in 2010, but QBR may forgive those if he is efficient at never taking sacks. Peyton’s DVOA (5th), EPA (7th), and WPA (7th) are all between the two measures, so it also must be something about the game charting that is attributing more success to Manning, like air yards or time elapsed on sacks taken.
Colt McCoy (20th in QBR, 29th in Passer Rating). This is an interesting one, because adding in sacks don’t help McCoy’s case. The guess is his low TD% rate unnaturally hurt in passer rating, but his efficiency was pretty good elsewhere.
Jay Cutler (25th in QBR, 17th in Passer Rating). This is the first on this list where QBR thinks a QB played worse than passer rating. It’s all about sacks and fumbles here. QBR’s definitely right on this one, as failing to account for sacks taken is the big problem with the current passer rating formula. Not all those sacks were on the line either, and the guess is many of them were attributed to Cutler as well, because he didn’t throw to his hot reads quickly enough.
Alex Smith (30th in QBR, 22nd in Passer Rating). QBR is harsh on Smith, placing him as the worst QB besides the three that were consensus bottom 3 in every system (Anderson, Favre, and Clausen). Smith is capable of making plays but is not consistent, so his play by play numbers look worse than his yardage averages. Here, QBR is simply in line with DVOA (28th)and EPA (29th), and compared to those measures, Passer Rating is overvaluing Smith when it thinks he is near average.
[photo via Getty]