NCAA Hosting Analytics Meeting to Look at Potential Changes to Tournament Selection Process
By Jason Lisk
The NCAA will host a meeting in Indianapolis next Friday between several people who work in basketball analytics, to discuss tweaks to how it selects teams, the NCAA announced:
"The NABC, representing the coaches, has sent a message that it would like to look into the use of more advanced metrics in the selection and seeding process. An even more powerful microscope to go with the time-honored RPI. The NCAA listened and agreed. A group of coaches and committee members is now at work, and this get-together is for everyone to hear the possibilities for the future — from those who know."
Jeff Sagarin, Ken Pomeroy, Ben Alamar of ESPN’s BPI, and Kevin Pauga, assistant AD at Michigan State who developed the KPI, are identified as among those who will be attending.
I’ve been to the NCAA for the mock selection exercise, do projections for how I think the committee will select and seed the teams, and have had plenty of criticisms about the method.
Things like the Pomeroy rankings are available to committee members in the past, but I can tell you being in that room and trying to get through the discussion of 68 teams, they are pushed down in importance. Everything becomes record versus the RPI top 25, top 50, and against the field. There are nice cool color-coded columns on each team page brought up in the room on the big screens, showing wins and losses against other good teams (as defined by the RPI). Things like location, timing are also diminished because you have to look into the details. More details require an extra click.
So I’ll be curious as to how this is addressed. I’ll buy it’s meaningful reform if they actually change the underlying language of how teams are evaluated and ranked.
Heading into that meeting, here are my thoughts on issues that should be addressed:
The RPI Rankings Heavily Penalize a Couple of Bad Opponents who you Beat Badly.
I talked about this with Kansas and Indiana’s schedules after the Hoosiers beat the Jayhawks early this year, and how Indiana would have no chance of finishing ahead of Kansas. I shouldn’t know the likely results before games are played. Astute schedule makers should not be able to game the system before a ball bounces. Any tweaks should counteract that.
RPI would say that Indiana has played an easy non-conference schedule, because they played a bunch of bad teams they blew out. They also played neutral games against Kansas, Louisville, and Butler, a home game against North Carolina, and a road game at Fort Wayne. How many true bubble teams would go 2-3 against that combination? Remember, Fort Wayne is 14-4 with losses at Notre Dame and Arkansas, and it was on the road against a potential tourney team. Losing at Fort Wayne is like losing at Georgia Tech, in terms of quality, and we’ve already seen North Carolina and Clemson do that.
A non-conference schedule where even good teams would have 3-4 losses is not an easy non-conference schedule.
The Committee Overvalues “Big Wins” in Seeding and Selecting Teams
I talked about this pre-tourney last year with seeding of Pac-12 and Big 12 teams. Among teams that are ranked similarly even within the RPI, there is a bias in favor of those with big wins. But it is not applied even on a ratio basis (i.e., they got 10 opportunities for top wins, and got two of them, while that other team only got one, on the road), but in raw totals.
I’m not sure, other than our analytics types showing this bias, how it is removed from selection, because it’s not actually part of the rankings. Virtually all of the major seeding errors, where many of us knew a team was a underseeded, were because the team was seeded down for not having as many “big wins”.
Recent examples include Wichita State and Gonzaga as 11-seeds last year (no opportunities in conference, losses early), Wichita State in 2015 as a 7-seed, Michigan State as a 7-seed in 2015 (0-5 against Kansas, Duke, Notre Dame and Wisconsin entering tourney), Kentucky as an 8-seed in 2014 (1-6 against Florida, Michigan State, Baylor, North Carolina, and Louisville; 23-4 against everyone else).
There Doesn’t Seem to Be Any Consistency in Putting Road Games in Context
Road games are tough to win. I already referenced the Indiana game at Fort Wayne. This will likely get tossed around as a bad loss. Good tournament teams probably lose that type of game close to half the time, if we compare it to similar conference opponents. The difference is that there are way fewer examples of teams playing on the road at higher caliber mid-majors like Fort Wayne.
But even among conference games–especially with the large mega conferences and imbalanced schedules–a road game at an average team may be tougher than a home game against a borderline NCAA tourney team. Yet the latter would get called a top 50 win and added to the ledger when the committee does its quick evaluations. The former would be largely forgotten.