The first CFP rankings came out last night, and after a season we’ve spent defending the committee…they did a couple odd things. Let’s talk through them.
First, here’s how much each team’s ranking differed from our model’s expectation. The number, FPA, designates the minimum percentage-point margin by which our model missed, where 100.0% is the “ranking score” of the top-ranked team, in our model, and 0.0% would be the ranking score of the last team in the FBS. FPA stands for Forgiveness/Punishment Adjustment, signifying how certain teams’ losses are forgiven and other teams receive punishment.
Second, here’s how much each relevant team’s playoff probability changed in our model’s eyes because of the rankings:
| Team | Before Rankings | After Rankings | Magnitude Change | Percent Change |
| Penn State | 60% | 83% | 24% | 39% |
| Tennessee | 65% | 82% | 17% | 27% |
| Boise State | 85% | 90% | 5% | 6% |
| Clemson | 15% | 19% | 4% | 27% |
| LSU | 8% | 12% | 4% | 51% |
| Notre Dame | 86% | 89% | 3% | 3% |
| Georgia | 94% | 96% | 2% | 2% |
| Texas | 85% | 88% | 2% | 3% |
| Colorado | 9% | 10% | 2% | 18% |
| Iowa State | 20% | 22% | 1% | 7% |
| Oregon | 99% | 100% | 1% | 1% |
| Kansas State | 16% | 16% | 0% | 1% |
| Ohio State | 99% | 99% | 0% | 0% |
| Alabama | 64% | 63% | 0% | -1% |
| Miami (FL) | 96% | 95% | -1% | -1% |
| Army | 9% | 7% | -1% | -14% |
| Tulane | 13% | 11% | -1% | -10% |
| BYU | 74% | 67% | -7% | -10% |
| SMU | 66% | 52% | -14% | -21% |
| Mississippi | 37% | 23% | -14% | -38% |
| Indiana | 75% | 52% | -23% | -31% |
Takeaways:
Indiana Is Half a Mid-Major
The number’s actually closer to 75%, not 50%, but the point is this: The discount the committee gave Indiana’s performance to date is comparable to what we’d expect if Indiana was doing what it’s doing in the AAC, Sun Belt, or Mountain West. This is how far short the committee sold Indiana. Sometimes, FPA is justified—the committee is catching something our model is missing. In this case, it’s reflecting an odd choice by the committee. If Indiana finishes 11–1, that’s currently only barely likelier than 50% to give them a playoff berth. That is unfair.
The Committee Loves Penn State
The committee effectively treated Penn State’s Ohio State loss as something that didn’t happen. Maybe Ohio State is so good, to the committee, that any loss to them is entirely excused. Maybe the committee respects the rest of Penn State’s résumé more than our model expected. More likely, this is a little bit of a helmet ranking. Penn State was undefeated and then lost to Ohio State, and they’ll be treated differently than what will happen if Indiana is undefeated and then loses to Ohio State.
I’ll pause for a second here to say that the committee’s decisions are at this point entirely reversible. I will also say, though, that FPA tends to stick around. That’s why it’s so useful to our model. The committee’s rankings especially tend to be “sticky” when a team doesn’t lose. Penn State’s probably going to win out. The committee probably isn’t going to reevaluate Penn State’s performance. Penn State’s probably going to get a home game. This, despite Penn State’s most impressive win being worse than Cincinnati’s. This isn’t great.
Boise State’s Got a Bandwagon
We have more on this in today’s bracketology, but the committee gave Boise State a lot of love, so much that we now have it slightly likelier than not that Boise State gets a playoff bye. This is bad news for the Big 12 champion, but to be fair to Big 12 teams, the conference’s average FPA was positive.
Was the ACC Disrespected?
We asked before the rankings whether the committee would treat the landscape as one with a Power Four or one with a Power Two. They didn’t exactly do either. Big 12 teams were treated very similarly to their SEC and Big Ten counterparts in terms of how much their accomplishments meant. ACC teams were treated more negatively, most notably SMU, who dropped alongside Indiana to a coin flip’s shot at a playoff berth.
It’s possible some of our conference-level FPA is reflective of a small sample. For example: We’re about to talk about South Carolina’s large negative FPA. Take that out, and the SEC is suddenly more respected by the committee than both the Big Ten and Big 12, at least in the eyes of FPA. Similarly, Colorado got a good bit of positive FPA, which is a little suspect when BYU had negative FPA. The ACC piece, though, is probably real. After years of giving the ACC more love than our model expected, the committee is probably turning, unimpressed by Miami beating Cal and SMU beating Pitt. That might be fair. The Indiana and Penn State things are unfair, but the Boise State and ACC things might be fair.
South Carolina Hasn’t Won Enough Games
If I had to guess, the reason Missouri is ranked and South Carolina is not is that Missouri is 6–2 and South Carolina is 5–3. It didn’t matter to the committee that Missouri’s best win isn’t as impressive as South Carolina’s third-best. It didn’t matter that South Carolina’s most excusable loss was about as excusable as they come (a two-point defeat in Tuscaloosa). This is silly, and it’s likely going to lead to more South Carolina disrespect going forward. But Missouri’s an underdog each of the next two weeks, and if South Carolina can win at Vanderbilt on Saturday as a narrow favorite, I’d expect we at least see the Gamecocks crack the top 25. South Carolina didn’t have much of a playoff shot anyway. This is disappointing but mostly meaningless. Even to South Carolina.
Tennessee Caught a Break. Mississippi Did Not.
Getting back to FPA’s definition, Tennessee’s loss to Arkansas was largely forgiven. Mississippi’s loss to Kentucky was not. In both cases, this is compared to historic norms in terms of how the committee treats losses. Normally, we wouldn’t expect Tennessee to get so much love. Normally, we wouldn’t expect Mississippi to get so much hate.
Still, this is subjective. Objectively, Penn State is ranked too high and Indiana is ranked too low. Mississippi and Tennessee? Subjective. I’d characterize this more as a miss by our model than a miss by the committee. Our model should have expected this, or at least something like it.
What did our model do wrong? First, it overestimated how Mississippi and Tennessee would show up through the eye test. Our Adjusted Point Differential variable occasionally overvalues high-octane offenses and undervalues defensive strength. Second, our model doesn’t account for the hoops the committee will jump through when it can to avoid ranking a team above a team who beat it. In other words: We need to figure out how to tell our model about head-to-head results. It’s tough, because sometimes they’re the final verdict on a team in the committee’s eyes and other times they don’t matter at all. But we need to figure that out.
Overall, Tennessee should feel better about its playoff probability and Mississippi should feel worse, but again, these two were largely our model missing the mark.
LSU’s in the Picture
The only other thing to note here is that LSU is a more realistic playoff team than Army. Some of this is LSU’s positive FPA. Some is Boise State’s eating up some scenarios where Army makes the field. But in terms of playoff probabilities, that was a leapfrogging which occurred.
