Where the First College Football Rankings Surprised (Oregon), Where They Didn’t (Cincinnati)

It’s helpful, now and then, to be reminded of the College Football Playoff committee’s instructions. The committee was and is instructed to choose the four “best” teams. Not the four most deserving. Not the four with the most plausible claims to being the one best team. The four “best” teams. There are advantages and disadvantages to each approach, but the College Football Playoff’s creators (BCS Properties, LLC; the management committee of which consists of the ten FBS conference commissioners plus Notre Dame’s athletic director) went with “best.”

Now, we must also acknowledge that the committee doesn’t actually choose the “best” teams. If they did, their job would be much simpler: Call up the oddsmakers in Vegas and ask which four teams are best. Instead, they choose a blend of “best” and “most deserving,” taking a subjective process and making it even more subjective. In the end, they often go with a rather common-sense approach, but to do this, they’ve relied a lot on luck. In 2014, they were able to split hairs between five one-loss conference champions by excluding the two whose conference had “co-champions.” In 2016, Penn State’s loss to Pitt was bad enough to raise few protests when the Nittany Lions were excluded and Ohio State, who lost to them head-to-head and finished behind them in the Big Ten, was included. In 2017, UCF was rated poorly enough by oddsmakers that the committee could point to the “best” instruction with some justification, especially after Alabama, the only questionable inclusion, won it all. There have been protests, a few somewhat justified (2017 UCF’s was the most legitimate—that team had scheduled two Power Five opponents, one rather good at the time of the scheduling, and had a comparable best win to that of the Tide), but by and large, the committee’s chosen acceptable teams, which is what we’re really after.

Finally, I’d like to point out that most arguments people employ when disputing rankings are rather dumb. People wanted Notre Dame out of the playoff last year for getting blown out by Clemson, but wanted Texas A&M in, when A&M had…gotten blown out by Alabama. People currently want Oregon ahead of Ohio State by virtue of the head-to-head win, but they also want Oregon to face due punishment for losing to Stanford, implying they want Oregon evaluated based on overall performance and Ohio State capped by one game. Pointing at one piece of data as an end-all-be-all trump card is a fun form of argument for people who enjoy feeling outraged or victimized (or second-hand victimized), but it’s not a reasonable one. Similarly, solely comparing teams head-to-head in the rankings ultimately always leads to circular rankings. It’s a fool’s errand to try to tell people to construct a comprehensive complaint with the rankings rather than to simply shout about head-to-head results, but I am a fool.

Now…the rankings.

We have a College Football Playoff model. One of the things it does is project where it expects each team to be ranked by the committee, based on the committee’s past rankings. You can read more about how it does that here, but the short version is that it evaluates teams the way our data suggests the committee values them, which is a combination of each team’s wins, losses, and win percentage; each team’s scoring margin; each team’s three best wins (based on margin/location/opponent); each team’s three worst losses (based on margin/location/opponent); whether each team is a Power Five team or not (this might sound like committee jackassery, but something like it is necessary in our model to counteract scoring margin advantages for Group of Five powers); whether a team has won a Power Five conference championship (this might actually be committee jackassery, but we don’t have a ton of data on it); and FPA, or Forgiveness/Punishment Adjustment, which mostly reflects where the committee has already ranked a team within a season. It’s imperfect, but so is the committee, and historically, the model has nailed the eventual College Football Playoff selections. 28 for 28 pre-FPA (and therefore also 28 for 28 post-FPA). We can’t guarantee perfection, but we can guarantee that this is the best playoff estimator you will find in the public sphere, and that we won’t do something like a certain college football media outlet did last year and completely whiff on our projection for, to stick with the same example, the committee’s ranking of USC. You can trust systems saying how good or bad teams are, but don’t trust systems saying where the committee will rank teams or what teams’ chances are of making the playoff. They are not as good as ours, empirically speaking, and if anyone knows where their historic projections are, we’d love to demonstrate that.

Something to emphasize regarding the model: It’s built off of precedent. We aren’t purporting to know what the committee should do. We’re purporting to know what the committee will do, and we’re using past committee decisions to do that. So when we say, “Oregon is getting way too much credit,” we’re saying that they’re getting way too much credit relative to committee precedent.

Now. How our model views last night’s rankings (if viewing on a phone, works best flipped sideways):

The “Expected Ranking” column here is where our model expected the committee to rank teams. The “FPA” column is the Forgiveness/Punishment Adjustment, and its units are the same as that of the Estimated Ranking Score, where the top-ranked team is estimated at 100.0 and UConn, our current projected lowest-ranked team in the FBS, is at 0.0. Excluding instances involving the Kelly Bryant Rule™ (if a rankings-relevant team loses in a game in which its first-string quarterback is hurt, they get half-credit for the loss), FPA sums to zero, and since we’re not aware of any noteworthy Kelly Bryant Rule™ examples at the moment outside of Penn State’s loss to Iowa, FPA in our top 28 (our top 25 combined with the committee’s top 25) sums to zero, as Penn State’s 31^st in our model even with the Kelly Bryant Rule™ applied.

Now. Notes:

Lucky Ducks (and Inland Californians)

Oregon’s loss to Stanford, at the moment, drops their estimated score by 7.6 points in our model, with 1.2 of those coming from the fact it’s a loss in the first place. We don’t know if it’s that loss that’s getting discounted or the Oregon win that’s getting boosted or the Fresno State win that’s getting boosted (Fresno State’s also getting a lot of love, with their loss at Hawaii evidently outweighed by…something), but Oregon is ranked way higher than they should be, based on precedent. Obscenely strong ranking for Oregon.

My guess as to what happened here is that the committee looked at Oregon, said they needed to be ranked ahead of Ohio State (to give them credit for scheduling a game in Columbus, as well as winning it), and then slotted that in as a guarantee. Then, they said, “Well, Fresno State played Oregon tough and beat San Diego State,” and ranked the Bulldogs.

If you’re pissed about this…I don’t know that you need to be. Oregon’s playoff path hasn’t really changed. They need to beat Utah, likely twice, and avoid another pitfall. They might be underdogs against Utah on a neutral field right now. The committee can and does change its mind, and if Ohio State wallops Michigan and Michigan State, which they’re kind of expected to do, the committee might call that enough to push the Buckeyes past the Ducks, eliminating Oregon’s evident floor. Similarly, if Ohio State falls again, Oregon’s evident floor will drop.

In other words, there are a lot of ways for this to work itself out, chief among them Oregon losing. And on that note, don’t worry too much about Mississippi State’s love at the moment. It’s silly, yes, but we often see such things deep in the pack, where the committee doesn’t stress the details as much.

Cincinnati’s Where We Thought They’d Be

This was the biggest thing to watch, and the fact we’re addressing it second further highlights how egregious the Oregon FPA is.

The Bearcats actually have one of the largest positive FPA’s out there, reflecting the fact that, like it or not, their second-best win isn’t very good. Their best win is great—second-best of all wins won this year by anybody, by our measure. But they haven’t done much else.

Thankfully for Cincinnati, they have a game remaining with SMU, and probably a game remaining with Houston, and though neither of those teams is ranked, we estimate them to be 29^th and 28^th, respectively (again, giving the committee the assumption that they’re as close to precedent as possible). Those are meaty potential wins. Akin to playing Clemson, or Penn State, or Purdue. Not outrageously good wins, but meaty potential wins.

More than anything, this is a reminder of how lucky Cincinnati still needs to get. The Bearcats need their good win to stay good, which relies on Notre Dame staying at one loss (and, very indirectly for Cincinnati, Wisconsin taking care of business). They need another win or two to become good: Their best bets, again, are SMU and Houston, and they should seek to win those games by three digits each. They likely need Oregon to, as is expected, stumble. They likely need Wake Forest to, as is expected, stumble. They could use another loss by Ohio State or Alabama, and not just to Georgia in the case of the Tide. They could use Oklahoma to stumble, probably twice. Basically, the SEC Champion is almost 100% getting in, and the other three spots are most likely going to Alabama and/or other Power Five champions. Something needs to go wrong for three of those five placeholders. It’s expected to go wrong for two of the five. In addition to taking care of their own responsibilities, Cincinnati needs one more stroke of luck, and probability suggests they’ll get it. The odds of Ohio State or Michigan State winning out, Oklahoma finishing with one loss or zero losses, and Alabama finishing the regular season 11-1 are below fifty percent. I don’t know that that’d be enough. But it might be.

Alabama Is Lucky Texas A&M Is Good

Texas A&M was reeling going into that Alabama game. They’d struggled against Colorado. They’d lost to Arkansas and Mississippi State. They were a mess.

These days, they’re not a mess. They’re 6-2. They blew out Mizzou and South Carolina their last two times out there. Neither of those is that impressive of a win, but the Aggies have pulled it together, and that’s reflected both in where Alabama is ranked and in where we expected Alabama to be ranked, each of which are in the top three.

If you’re wondering, yes, our model would expect Alabama to still be ranked highly if they lost to Georgia in the SEC Championship. If you’re also wondering, yes, that makes us highly nervous about our model’s perfect streak, since the human factor (not a part of our model) says we aren’t sure the committee would let itself put a two-loss team in ahead of an undefeated team or a one-loss Power Five champion.

UTSA Has Legitimate Complaints

One thing about FPA is that it’s close to as small as it can possibly be to align our rankings with those of the committee. Basically, we assume the committee is as close to precedent as possible. With that established…

The committee does not think highly of Conference USA. Maybe this is fair. Maybe our blanket approach to Power Five/Group of Five status is misguided, and Conference USA is getting a bigger discount (in the bad way) than the AAC is. This would be reasonable. But right now, the Group of Five discount is only 2.4 points. UTSA has an FPA four times that, and they already got the Group of Five discount to begin with.

Is this impactful? Not for the playoff. But for the New Year’s Six bowls…potentially. Again, though, it can change.

Wake Forest Has a Complaint

One of the biggest recipients of negative FPA is the Deacons, and while we’ve been pointing out a lot that they’re going to be underdogs often from here on out, it’s still a break from precedent by the committee. That said, the committee gave the three measured ACC teams an average of -1.4 FPA overall, matching that which the Deacs received, so maybe it’s an ACC discount, which would be pretty reasonable, since the ACC stinks this year (If you’re about to ask this, yes, we’re going to try to spend some time this offseason looking at whether the Go5/P5 discount should be conference-based rather than binary).

Again, this doesn’t matter. If Wake Forest wins out, they’re in. If they only lose one more game and they still win the ACC, they’re probably in. The problem for Wake Forest is not the committee’s impression of them. It’s whether or not they win the games ahead of them.

BYU Is a Power Five School

This is fun. If you eliminate BYU’s Group of Five status from last year (BYU also got a big negative FPA last year), their FPA is zero. So…we did that. We didn’t know how they’d be viewed, but it turns out the committee seems to be viewing them identically to Notre Dame in terms of Group of Five/Power Five status. We think that’s reasonable.

Some Editorializing

Look. The committee’s going to make mistakes. It’s going to be imperfect. It’s going to change its mind. It’s going to try to end up doing the reasonable thing, and the fact of the matter is, Cincinnati’s ranked in a fairly reasonable spot for a team with one great win and a whole lot of nothing. UTSA got ignored, but Conference USA is, in a literal-but-not-physical sense, falling apart. Of the four big deviations from precedent, two are meaningless, one is only meaningful for UTSA at the moment, and one is likely to sort itself out, because Oregon isn’t actually very good. Also, if the biggest problems are teams being encouraged to get themselves out of Conference USA and teams being encouraged to schedule cross-country road games against national championship contenders…I have a hard time complaining about that. Do I want Cincinnati to get in? Yes. In a subjective way, because I think that’d be fun, and in a practical way, because I want to know what it actually takes for a Group of Five to make the field, partially so I can build that into this model.