Note: I’m rushing this off a bit and not being as thorough as I should be, so if I miss a detail or you have a question, let me know and I’ll post an update.
The NCAA announced some changes to NET today. Here’s what to know:
The Changes Themselves Aren’t Major
It’s unclear how much has changed, but the change does not appear significant. The NCAA “removed” winning percentage, scoring margin (capped at ten points, I believe), and something they called “adjusted winning percentage.” What remains, though, is adjusted net efficiency—which is a nuanced and effective way to evaluate scoring margin—and something the NCAA calls Team Value Index—which is possibly a nuanced and effective way to evaluate a team’s record while adjusting for strength of schedule (I say “possibly” because unlike adjusted net efficiency, we don’t know with as much certainty how the NCAA’s constructed this formula). In other words, the now-removed variables are still being considered, just not as explicitly.
My personal guess here (and I could go on about why this is my guess, but won’t for the sake of brevity) is that the NCAA had some very small coefficients attached to these now-removed variables that weren’t making a big difference. In examining it, they recognized these were useless, so they removed the variables as explicit inputs. Just an educated guess, but it aligns with what they’re saying.
The New Formula Might Just Be KenPom Crossed with SOR
If you’re in the weeds on this stuff, you’re familiar with KenPom and SOR. If you aren’t, a quick primer:
KenPom, Ken Pomeroy’s rating system, is adjusted net efficiency (margin of victory/defeat adjusted for strength of opponent and location) with a few added features (margin of victory matters less and less as a blowout gets bigger, and recent results are given a slightly heavier weight). KenPom is the most predictive mainstream rating system, which means it’s the best at predicting future results, which means it’s the best at saying how good a team really is.
SOR is Strength of Record. To be completely honest, I don’t know where exactly it originated, though I should. ESPN makes heavy use of it, featuring it alongside its BPI (Basketball Power Index, or something like that—it tries to do the same thing as KenPom, but my impression is it doesn’t do it quite as well). SOR measures how strong a team’s record is in light of their schedule by calculating the likelihood that a team of a specific caliber (Top 25, NCAA Tournament, etc.) would achieve the record that team has achieved against that specific schedule. It doesn’t consider margin of victory/defeat.
Put more simply: KenPom measures how good a team is. SOR measures how well they’ve played (but only using wins and losses).
It’s been clear since we started getting NET rankings two falls ago that the adjusted net efficiency the NCAA’s using is not dissimilar from KenPom. We don’t know if the “Team Value Index” really is just another name for SOR, and even if we did, we wouldn’t know how exactly this specific SOR works, because my impression is that SOR requires ratings of teams’ quality to work with as a starting point (my further impression is that ESPN uses BPI for this starting point, but I can’t say that with certainty).
NET Is Still Not All-Important
NET is still only one of six rating systems listed on the Team Sheets with which the selection committee is supplied. It is the one the NCAA uses to group results into the four quadrants on the Team Sheets, meaning that a team’s opponents’ NET rankings do, at the margins between quadrants, matter somewhat, but even that’s a limited impact. There’s little evidence to support NET being any more important than SOR or KPI (Kevin Pauga’s rating system, which bills itself as a way to quantify a team’s résumé, and displayed a particularly strong correlation to 2018’s NCAA Tournament seedings), and a not meaningless portion of its importance comes not from a team’s own NET ranking, but from those of its opponents, which can change the number of Quadrant III losses a team has (or Quadrant I wins, or any of the other quadrant wins/losses). In other words, the NET receives an unwarranted amount of the attention paid to the selection process.
How You Should Feel About NET Depends on What You Think the Committee Should Value
With that said, it’s fair to care about NET, because while it isn’t the most important thing, it isn’t unimportant, and its construction is something of a proxy war between a “KenPom is the best way to seed teams” way of thinking and an “SOR is the best way to seed teams” way of thinking.
There are generally two reasonable schools of thought (to go with plenty of unreasonable ones) about how the selection committee should choose and seed teams in college basketball (and college football, for that matter). One is that the committee should place the best teams in the bracket and give them the best seeds. The other is that the committee should place the teams with the best résumés in the bracket and give them the best seeds. There’s also a minor division within this second school of thought as to whether or not game control (or margin of victory/defeat) should matter in evaluating a team’s résumé (since teams that are thoroughly winning really have performed better than teams that are scraping by).
If you subscribe to the first school of thought, you’ll like how essential adjusted net efficiency is to NET, and you’ll likely agree with the NCAA’s explicit goal of making NET predictive. If you subscribe to the second, and you don’t think game control should matter, you probably don’t like how heavily adjusted net efficiency is utilized. If you’re somewhere in between, you’ll probably conclude that NET seems reasonable, but you might wish they’d just spell out the formula explicitly and publicly.
The Strength of Schedule Metrics Are Getting an Update, and That’s Important
Possibly the most important thing in the whole announcement is that SOS and Non-Conference SOS are getting what sounds like it might be a major makeover. That makeover is that rather than using what seemed, by my impression, to be a formula based on opponents’ winning percentage and opponents’ opponents’ winning percentage (three quarters of RPI, if you remember the RPI formula), the SOS numbers listed on Team Sheets will use an SOR method. This is objectively a more reasonable way to measure SOS, provided the initial ratings input is solid, and while SOS and Non-Conference SOS aren’t the backbone of how the committee seeds teams, they do in certain occasions play a significant role (see: North Carolina State, 2019). On the surface, the revamp seems like good news, and given the pivot from RPI to NET, it also seems overdue. The new way should, if it operates as described, more accurately and precisely measure how difficult a team’s schedule really is, which is preferable.
