Believe It or Not, the College Football Playoff Rankings Are Inconsistent

Well, we have rankings now. They told us a lot. And one of the biggest things they told us is how little we still know.

Our college football model—which remains unpublished this year thanks to the uncertainty we’re about to talk through—has a variable in its ranking formula called FPA. If you read our college football stuff last year, you might remember it. FPA stands for “Forgiveness/Punishment Adjustment,” and we use it every week to calibrate our model to the committee’s rankings. In its essence, FPA is a metric for all the things the committee sees that the numbers don’t—some of which could be injuries or other circumstance, others of which are likely narratives, opinions, and other things that exist only in the mind.

When we back-test our model against last year’s season, the totality of our final FPA adjustments is just 40 points, on a scale in which 0 is the worst FBS team’s valuation and 100 is that of the top-ranked team. Last night’s rankings yielded a total FPA adjustment of 140 on that scale. Three and a half times more than the entirety of last season. Which means: These rankings were surprising compared to historic precedent. Very surprising.

Now, that 140 number might shrink as the season goes on and both our model and the committee stop having to try to rank teams who’ve played three games (and played them well, in the case of Western Michigan and Buffalo, both of whom our model expected the committee to like—which was our first sign we were going to have a busy FPA week) against those who’ve played nine (we’ll get to BYU). But regardless, at this point, it’s a reminder of how uncertain this all is, and how hard it is to predict the actions of an already subjective committee when they’ve been tasked with comparing apples to oranges.

Now, what we did learn, from 1-25:

1. Alabama
2. Notre Dame

No surprises here for our model. Undefeated teams with quality wins and strong margin of victory numbers. Even Alabama’s third-best win (in our model’s eyes—second-best, in that of the narrative), that second-half runaway from Georgia, grades out as the 32nd-best in the country by our model’s grading, which accounts for location and margin. They are the clear number one, and there’s no reason to expect that to change without them losing.

3. Clemson

Again, no surprise, but perhaps a relief for Tigers fans, and a bit of an indictment against a publicly available model that you see a lot of and should not trust (if you’ve been giving percent likelihoods of playoff berths before last night, they’ve been wrong, and even after last night you better have a sophisticated way of accounting for the uncertainty if you want to use them for anything more than entertainment).

4. Ohio State

One more non-surprise, but it’s worth remembering that Ohio State only has one quality win at this point, and it was this weekend’s controlled-but-not-impressive victory over Indiana. They have room to rise and aren’t far behind Clemson, but the remaining schedule, aside from the presumptive Big Ten Championship against Northwestern, doesn’t offer much in the way of opportunity.

5. Texas A&M
6. Florida
7. Cincinnati

Now we start getting interesting. And consequential. Texas A&M ahead of Cincinnati.

This was one of the biggest questions answering last night, and while on the one hand, it’s positive for the Bearcats to be historically ranked for a Group of Five team, it’s not exactly hopeful to sit behind the Aggies.

But.

We don’t really know how close these teams are, and it’s fair to assume the answer is close. Texas A&M has the best win of the three, but it’s a narrow lead when one accounts for margin and location. Cincinnati hasn’t lost, but it’s hard for any loss to Alabama to be disqualifying, even when it comes by 28 points (that loss remains one of the best losses in the country). APD (Adjusted Point Differential, our effective proxy for the eye test and legitimate advanced metrics) has Florida and A&M both ahead of Cincinnati, but again—it’s close. Overall, our model expected Florida to be ranked the highest of the three, but it thought they’d all be within three points of each other on that 100-point scale we referenced earlier, with Cincinnati in the middle.

Cincinnati, surprisingly, has two opportunities left. Tulsa wound up with one of the smallest FPA adjustments in the country and a top 25 spot, and Cincy gets to play them on the road. They then have an AAC championship that should be a rematch of the Tulsa game. After all’s said and done, if Cincinnati wins both of those Tulsa won’t be in the top 25, but compared to A&M and Florida, it’s not a bad slate. A&M’s best chance remaining is a visit to Auburn which 1) they’re not guaranteed to win and 2) will come on the heels of Auburn likely falling out of the top 25 following destruction by Alabama on Saturday. Florida’s best chance is, of course, Alabama in the presumptive SEC Championship, but if they win that, we’re having an entirely different conversation, and if they lose that, it would have to be the closest of close losses to help them much.

Circling back to Notre Dame for a second: Cincinnati trailing the SEC pair is good news for the Irish, implying that if they lose the Clemson rematch, they shouldn’t get passed by the Bearcats, which was a marginal but real worry.

Finally, before we move on, remember that public perception shapes the committee. One could argue it was designed with just that intent: To reflect the “common sense” consensus. The Tulsa games might not matter as much for Cincinnati as actual résumé-changers as they do as excuses for the committee, if under pressure, to vault the Bearcats above the SEC also-rans.

8. Northwestern

Need evidence that perception shapes the committee? The Fighting Rece Davises held a fireball of an offense to seven points, have yet to lose, and came in ranked behind a Texas A&M team whose second-best accomplishment is keeping the Alabama deficit under 30. The “eye test” is often an offensive test, of course, but this is still a thing to notice.

Now, if Northwestern wins out through the regular season and plays Ohio State to a very close loss, they could certainly jump the three ahead of them and make the field in the Alabama/Notre Dame/Ohio State undefeated scenario. But this is still a bit of a thinking emoji.

9. Georgia

Georgia was not the cause of the biggest FPA adjustment this week—that would be North Carolina, followed by Auburn, followed by Texas. But man. They have done nothing aside from play a couple good first halves.

10. Miami
11. Oklahoma
12. Indiana
13. Iowa State
14. BYU

Yes, this is surprising. No, it doesn’t really matter that BYU has played an easy schedule. They’ve obliterated everyone they’ve played, and schedules of the others in this block aren’t particularly grueling. It makes one wonder whether the Washington scheduling leak—if you missed it, Washington said they offered to play BYU this weekend when the Apple Cup got canceled, then told the media that BYU had said no, but then BYU said they just wanted to wait and get confirmation that the Pac-12 wouldn’t swoop in and make Washington play a Pac-12 game this weekend, which the Pac-12 ultimately did, proving BYU’s alleged point—swayed the committee a bit.

BYU needs to find someone good to play them. It’s not entirely unlikely they won’t. But it might take a Power Five team agreeing to it, or Cincinnati agreeing to it, and it’s hard to see why any of them would, with nothing in it for them. BYU might need to buy a game from a team that’s used to buying games themselves.

For what it’s worth, the model only thought BYU would be ranked 10th. For what it’s worth, a rising Boise State tide could lift BYU’s ship. For what it’s worth, that might not matter much.

15. Oregon
16. Wisconsin

Now we wait to see how high Oregon can rise. But again, compare Oregon to Ohio State and tell us this is remotely objective. No, Oregon isn’t on Ohio State’s plane, but the wins line up comparably when you look hard, implying they should at least be closer than this.

17. Texas

As with Georgia…what the hell?

18. USC

USC, like Oregon, isn’t on Ohio State’s plane, but their résumé isn’t this much worse.

19. North Carolina

As with Georgia and Texas, etc. etc.

20. Coastal Carolina
21. Marshall

Again, kind of weird, but not at all surprising. The thing about the Group of Five rankings this year is that the win quality is kind of there…

We do have a Group of Five variable that, in accord with history, pushes them all down a bit. But it doesn’t push them down this much. This year is new.

22. Auburn

What, and I cannot emphasize this enough, the hell.

23. Oklahoma State

Sensible.

24. Iowa

Fair.

25. Tulsa

Reasonable.

***

Overall, stinks to be BYU right now. They have legitimate complaints. Ditto Oregon, though they have a bit left to gain. In total, it probably doesn’t matter. It’s probably a Notre Dame/Clemson/Texas A&M/Florida/Cincinnati/Northwestern race for two spots, and in the most likely scenario, Florida and Northwestern will be eliminated rather naturally.

We’ll see.

The Barking Crow's resident numbers man. Was asked to do NIT Bracketology in 2018 and never looked back. Fields inquiries on Twitter: @joestunardi.
Posts created 3304

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts

Begin typing your search term above and press enter to search. Press ESC to cancel.