There are a few points in the year in which our Bracketology model hits a wrinkle. It’s not a bad thing—it’s by design, though to be fair, we’re working on ironing out the wrinkle in future iterations. But it happens, and it sometimes augments or dampens the results from a given few days, shaking up the picture more than our model expected.
All of this has to do with the rating systems the NCAA puts on its committee’s team sheet. Half of these systems—KenPom, BPI, and Sagarin—are purely predictive. They’re easy. We just plug them in as they are every time we run the model. Two of the other three—SOR and KPI—are purely reflective. NET is a mix of predictive and reflective. The ones with some reflectiveness are more difficult to incorporate accurately.
From our explanation of how the model works:
Early in the season, NET and KPI aren’t available, and when they first become available, they and BPI SOR aren’t wholly indicative of their eventual end. There’s a lot of noise early in the season, so instead of drilling down on the current NET/KPI/SOR rankings and believing them to be too indicative of their final state, we use an elo-like system for each. For NET, the system is based off of KenPom, given the similarities between the two. For SOR, the system is based mostly off of BPI, just as BPI SOR is naturally, but also incorporates a bit of KenPom and Sagarin. For KPI, the system is based off an average between KenPom, Sagarin, and BPI.
…this is something that will be changed around the end of January, when KPI and SOR and NET have had a chance to find their level. We’ll still be using an elo-based system to simulate where the rankings will go in conjunction with specific simulated results, but at that point we’ll have enough data in those systems to make them their own bases. NET’s elo will be based off of NET. SOR’s will be based off of a BPI/SOR combo. KPI’s will be based off of KPI.
January’s over. We’re making the transition. And while we have yet to flip NET and SOR to their eventual formulas, we flipped the KPI formula last night to be entirely based off of KPI (we also adjusted SOR to slightly incorporate KPI).
We eventually want to reverse engineer these systems. That’s how we’ll iron out the wrinkle. But that’s not the easiest thing in the world to do, and it’s going to take a lot of time when we do it (likely this summer). Instead, we use these elo-like proxies. They aren’t perfect. We know they aren’t perfect. But they’re our best attempt right now.
KPI, the one that got flipped over last night, is the toughest for us to create a proxy for, because it isn’t inherently tied to another system the way NET’s parallel to KenPom and SOR’s built off of BPI. So when we flip its proxy formula, it’s a bigger wrinkle than will be the case with the other two.
For most teams, nothing happens. The proxy formula was doing fine. For a few, though, there’s a jolt. It only happens once—it’s not like these teams will keep getting yo-yoed by KPI—but it shakes up the picture, so for those of you who check our bracketology regularly, we wanted to explain what happened.
Now, the movement in today’s updated bracketology:
Moving Up: Virginia Tech, Rutgers, St. Bonaventure
St. Bonaventure is a KPI bump team. The Bonnies are 9-1 with a worst loss at Rhode Island, which really impresses KPI. Predictive systems aren’t as high on them (and neither, looking ahead to future updates, is SOR or NET, so don’t expect similar jumps from those). Our model expects their KPI ranking to drop as they play (and lose) more games, but the median simulation now has them rather safely clear of the bubble, even with no wins yet over projected NCAA Tournament teams.
For Rutgers, KPI played a role, but this was more a results-based jump. Winning on the road is hard, and winning at Northwestern helped the Scarlet Knights a good deal, even if Northwestern isn’t great. There was one more model wrinkle that affected Rutgers a little: Our model gets updated BPI, SOR, NET, KPI, or Sagarin on a one-day lag, because we have to run the model overnight to get an adequate number of simulations and those updates come in the morning. To account for this, we use our normal proxies to adjust NET, KPI, and SOR, but we use KenPom’s % change in rating to adjust BPI and Sagarin, which sometimes slightly misestimates how those systems will react to an individual result. Rutgers’s Thursday night blowout of Michigan State was slightly underestimated by the KenPom-based BPI/Sagarin proxy system. So, Rutgers’s rise is partially their Sunday result, partially our better understanding of systems’ reactions to their Thursday result, and partially the KPI wrinkle.
Virginia Tech is purely results. The Hokies beat Virginia by 14 points. That’s a great win.
Moving Down: Colorado, Virginia, North Carolina, UConn
Colorado lost at home to Utah and promptly dropped four seed lines, down to the 9-line. Virginia, as mentioned, lost badly to Virginia Tech. Neither UNC nor UConn played, but KPI is lower on them than our model expected, pulling them both back towards the bubble.
Moving In: Penn State, Georgia State (auto-bid), Grand Canyon (auto-bid), Prairie View A&M (auto-bid), Norfolk State (auto-bid)
Penn State’s lost seven times. They’ve only won six times. Their worst loss is to Seton Hall at home. Seton Hall is the team they replace this morning in the projected field. Half their wins have come against projected tournament teams, with the latest Saturday’s victory over Wisconsin in State College.
Penn State’s got a lot of work to do. Their median simulation has them finishing right around .500, which would make them a historic tournament team in terms of win percentage. It’s possible this would be cause for the committee to hold them out, and it’s a concern of ours with regard to the model, which doesn’t directly say, “If a team is ____ games above/below .500, _____ happens.” But in a season with a shortened nonconference schedule, one wouldn’t be surprised to see a team with a historically poor overall win-loss record make the field. It would fire people up, sure. But it would be the sensible thing to do. We’ll see.
In auto-bid land, Georgia State climbs back in on the back of Texas State getting swept by Louisiana-Lafayette. The Sun Belt is tight. Grand Canyon swept New Mexico State, who had two lengthy Covid-related breaks, and is now the team to beat in the WAC. In the SWAC, Prairie View A&M won handily while Texas Southern struggled with a worse opponent, flipping the former narrowly ahead of the latter. And in the MEAC, Norfolk State got back on its feet after a rough weekend last weekend, pushing back ahead of an idle-in-conference N.C. Central.
Moving Out: Seton Hall, New Mexico State (auto-bid), Texas State (auto-bid), Texas Southern (auto-bid), North Carolina Central (auto-bid)
Seton Hall, after spending some time projecting clear of the bubble, had slipped back towards the NIT’s grasp in recent weeks. This morning, they’re projected out of the field, a casualty of their 71st-place ranking in KPI, which is lower than our model would’ve guessed.
***
To make a few things clear as we wrap up:
- This bracketology is still our best projection of the eventual field. Don’t expect any of this stuff to regress just because it was a jolt.
- The previous bracketology was also our best projection of the eventual field. At the beginning of the season, a proxy’s our best bet for estimating KPI (and SOR and NET). At the end of the season, we don’t need a proxy. At some point, we have to flip from one to the other. This year, that’s happening right now. It’s a flawed system, and we’re working to improve it, but it’s the best we’ve got right now.
- Not too many teams were gravely affected, which is a good sign for our initial proxy. But some were, and we’ve addressed the most significant ones here.
The next update should come Wednesday morning, as we move to thrice-weekly bracketology for the near future. We plan on flipping the NET and SOR proxies by the Saturday after next, so within the next two weeks, these specific jolts will be over (and we’ll be into scheduling jolts as conferences finalize their tournament plans and makeup games or lack thereof).
