How This Year’s College Football Playoff Rankings Compared to the Four-Team Era

It is with great shame that I come to you, twelve days after the CFP committee’s final rankings of the year, and I tell you what our model thought about them. For those who’ve been wondering:

As a refresher: FPA stands for “Forgiveness/Punishment Adjustment,” and it’s basically a measurement of how far the committee is straying from precedent. It accumulates over the course of the season based on where the committee ranks every team every week. It bends after each new set of rankings, accounting for 1) what the past indicates the committee would think of each team, 2) what the committee has shown it thinks of each team in previous weeks’ rankings, and 3) what the committee is showing with these specific rankings. BYU has a large negative FPA, signifying that it’s unusual that the committee ranked them so low. Syracuse has a large positive FPA, signifying that it’s unusual that the committee ranked them so high. Syracuse was forgiven. BYU was punished. For what? It’s our job this offseason to figure out if there was something there. If it was, we’ll need to build it into our model. If it wasn’t, we’ll need to adjust our uncertainty variables so our model knows to expect that certain level of uncertainty.

A few trends…

Conference Championships Were a Free Play

The biggest difference between the four-team and twelve-team era so far is that in the first year of the twelve-team era, teams in conference championships had nearly nothing to lose. As we theorized ahead of the selection show, the committee was willing to move teams based on conference championship losses, but it was unwilling to move teams far. I’m not sure how the committee felt about this in hindsight. I’m curious if anyone on the committee felt boxed in by previous public statements or a fear of backlash. Personally, I’m in favor of reworking the rankings every week from top to bottom based on everybody’s new résumé, which changes even when teams don’t play. But I understand that a lot of college football fans like the horse race, where teams’ previous ranking becomes gospel and teams only move based on what they did in a given week. It’s a foolish way to do things—it makes the order in which games are played really, really matter—but some people do like it, and thanks to the AP Poll, foolishness is the industry standard.

The Committee Liked Penn State

Going off of that: Penn State’s ridiculously large FPA actually mostly originated before the conference championships were played. Most of that was already there. The committee majorly messes up a few things a year. Most years, there’s a big enough result that they’re able to correct it, or they can just pump the brakes and let precedent catch up to where they have a team. (That’s what happened with Boise State, who had a huge FPA in early November and a reasonable one when the bracket was determined.) With Penn State, the problem for the committee was that when this chance arose—a chance to correct a previous mistake—they seemed to feel beholden to the idea of not punishing a team who played in a conference championship. Again: Penn State’s final ranking was foolish. But Penn State played Oregon close enough that precedent suggested Penn State shouldn’t fall very far. The issue was how high they were ranked to begin with.

Teams With Gripes: Indiana, Miami

It’s probably true that had Penn State achieved the exact same scores as Indiana against Indiana’s schedule, Penn State would have been ranked in the top five. Indiana was likely punished because the committee didn’t believe they were good before the season began.

In a parallel development, Miami was punished for the timing of their loss. A close loss on the road to a decent team shouldn’t have moved the needle that much. A close loss on the road to a ranked team definitely shouldn’t have moved the needle like that. But it did, and there’s an element of fairness because Miami had its opportunity and didn’t take care of business, but there’s also an unfairness to it because Miami could have lost to Syracuse in September and had a playoff shot.

The Committee Might Overthink Things

Looking ahead to next year, something to watch for from the committee:

Do they get too strategic for their own good?

The committee always responds to public pressure, but they especially seemed to respond this year, panicking over the possibility of a conference championship loss being treated the same as any other loss. If they’re happy with how it turned out, they might feel encouraged to engage in similar behavior in the future. Maybe they’ll think they can game some system, holding off on ranking ACC and Big 12 teams highly to avoid feeling pigeonholed to put a team like this year’s SMU into the playoff. Maybe they’ll hear the public’s discomfort with the ease of the 5-seed’s path and intentionally shuffle the deck, making the 5-seed not the real 5-seed. There are a number of ways to do these rankings. Generally, the committee does a pretty good job. But after this year, I’m concerned that they’re going to try to do too much, doing things in one week’s rankings based on expectations of future weeks’ results, trying to set themselves up to avoid backlash. Given the customary poor understanding of probability in the sporting world, that could really mess things up.

In the meantime? When we build next year’s model, we’ll probably split the difference between conference championships counting and not counting as football games.