NFL Draft Study On Character Issues Does Not Add Up

"“So if you’re on the fence about a player and worried about his criminal record,” said Stephen Wu, an economics professor at Hamilton College in Clinton, N.Y., “the data says take a chance.”"

Not so fast. I know the words “regression analysis” make journalists’ eyes gloss over, but I went and found a draft of this study. I’ll go through it in a moment. Before we get to the numbers, though, take a look at this paragraph and consider whether the best teams are the ones employing this smart strategy.

"Of the teams mentioned above, Arizona used 27 percent of its picks during the five-year span on players in the last three groups. Cincinnati, which has become a sort of “Boys Town East” for troubled free agents as well as draftees, was second at 25 percent, with San Francisco and Chicago tied for third at 20 percent. At the other end of the spectrum, Seattle drafted no players with character issues, followed by Atlanta (2 percent), Baltimore (3) and Green Bay (6)."

The teams taking advantage of this supposed market inefficiency won 48.9% of their games from 2005-2011, and made the playoffs a combined nine times. The prudish teams who were most likely to avoid pre-existing character issues won 56.3% of games and made the playoffs a total of sixteen times.

There may be other factors–team chemistry, the same behavior that drives drafting based on character is correlated with other front office behaviors–but a prime issue is the use of games started. Games Started is like RBI’s in baseball: it reflects both ability and opportunity. An average talent would start more for Cincinnati or San Francisco over the last seven years than Baltimore or Green Bay.

Turning to the actual study set up, my first issue is the use of draft position as a linear measure. Draft value is not linear. The difference between pick 1 and pick 31 is greater than the difference between pick 201 and 231. If I were running a regression on the draft, I would select some value measure that reflects this. If the character issue guys were at the top of the draft or the very bottom matters as to how strongly these findings are, and I don’t see any info on where the average character issue guy was drafted.

So, basically, this study ran two separate regressions. The first tried looked at Draft Position, using character variables as well as a player/college characteristic variable (all-american status, games started, newspaper mentions, team wins). The second looked at Games Started in the NFL as the dependent variable.

The results were that players with character concerns fell 11 spots in the draft. When those character issues were sub-divided, the arrests/not charged were not “drafted significantly earlier or later than those with a clean history.” (Though they were actually drafted 14.6 slots earlier with a small sample size, more on that in a minute). The arrested/charged players were selected 16 spots lower, and the players who had team suspensions were drafted 26 slots lower.

Then, the second regression showed that arrested/not charged players started 2.1 more games per year than expected, the arrested/charged started about the same as expected (0.3), and the suspended players started 1.6 fewer games per year.

"Nonetheless, for teams that are sitting on the fence about a particular player on draft day, the results of this paper suggest that they steer away from players with a history of trouble with coaches and teammates, but they be willing to take a chance on those that may have had problems with the law."

So, here’s my problem with those interpretations/conclusions based on the results of the two separate regressions. The results generally show that higher drafted players start more games and lower drafted players start fewer. “Not significantly different than” is not the same as zero, even though it is often treated as such. Here, we are talking about approximately 26 total players who were arrested but not charged, and they did actually get drafted higher than their peers with similar college/team traits. If we assume there is no difference in draft position, then yeah, the separate regression result that says they started 2.1 more games sounds great. But what if we phrased it as follows:

• The approximately 26 players who were arrested but not charged were drafted 15 slots higher and averaged 2 more starts per year than others with similar college/team traits;
• The approximately 77 players who were arrested and charged were drafted 16 slots lower and averaged the same number of starts per year compared to others with similar college/team traits;
• The approximately 51 players who were suspended for other team violations were drafted 26 slots lower and averaged 2 fewer starts per year than others with similar college/team traits.

In 2011, for example, among the draft classes included in this study, 1st rounders averaged 9.8 games started while 2nd rounders averaged 7.3 games started. The difference between 6th and 7th rounders was 2.3 and 1.3 games started, much smaller. Thus, those players suspended for team violations are both drafted lower and produce lower. Okay, that doesn’t mean teams should avoid them. Looks to me like teams are collectively already properly discounting them and getting similar production to others similarly drafted.

If I were trying to study the interplay between arrests/suspensions, draft position, and performance in the NFL (and had to use games started as my flawed variable), I would have run one regression with games started as the dependent variable, and used draft value and character variables. What we have instead is two separate regressions, where the factors are running in similar directions. For all I know, this just proved that draft position matters.

If anything, the strongest conclusion I could draw from looking at the two regressions in unison is the one that wasn’t drawn. Those players arrested and charged are the best value in terms of games started. They were drafted 16 slots lower, but provided similar production.

[photo via US Presswire]