Welcome to our daily college football newsletter. If you like what you read and want it delivered to your email inbox every morning, subscribe to our Substack.
**
I’m not going to say how, for the most part, I think the College Football Playoff committee does a good job ranking teams. I may harbor such feelings privately, but I must never say so, aside from all the other times I’ve said so and will say so in the future. Anything resembling a compliment towards the College Football Playoff committee is blasphemy in the eyes of the blogosphere tribunals, and I don’t have the time to battle excommunication today.
Instead, I’ll offer this: The process by which the College Football Playoff committee constructs its top 25 is better than the process used by the Associated Press.
The AP Top 25 was a good idea at its onset and it serves a good purpose. The purpose it serves is to give us those little numbers next to teams’ names, a highly efficient primer for casual fans pondering the stakes of the games in front of them. The context in which the AP poll was a good idea was that of an internet-less world with minimal college football broadcasts on television. Regional journalists carried regional expertise back then, and they were paying more attention to the national picture than just about anybody.
Today, the situation is different. Consider the average AP voter’s Saturday: Given most cover one specific team, their attention must be focused entirely on that team and that team’s opponent for at least one of the four primary broadcast windows. If they’re attending the game in person, there are travel considerations. Even for home games, the necessity to show up early often wipes out a second window, and the dash to submit their postgame recap obstructs a third. At a minimum, the average AP voter spends half their college football Saturday employing tunnel vision. This, in an age when it’s possible for even casual fans to follow every consequential college football game at once.
This isn’t an endorsement of AP voters’ eye tests. We’re not saying AP voters should be watching every snap and styling themselves experts in the distinctions between the skill sets of various ACC tight ends. We’re simply pointing out that AP voters miss a lot of games. Sometimes, a lot of them miss the same game. We don’t care that they didn’t watch every game closely. We care that sometimes, a substantial portion of AP voters are hardly aware a game happened at all.
I’m sure this reads as a criticism of the voters, and to some extent, it is. The poll has some consequence, and voters who don’t take it seriously are doing a disservice to entire universities full of people. AP voters who don’t take the time to figure out what happened while they were locked in covering UNC vs. JMU are engaging in some degree of laziness. It’s hard, I know, there are sometimes 45 consequential games in a Saturday, but there are AP voters out there who are not doing an acceptable job.
Still, the bigger problem is the process. Not only does it rely on journalists half-aware of the national picture, but there’s no clear definition of what the poll is polling. Is it the best teams? Is it the teams with the best résumés? Is it the teams who’ve lost least recently? Is it the teams who are fun stories? Is it the teams who “deserve” it? The only answer the AP poll gives to these questions is that it’s the “top 25.” That means nothing.
In contrast, the CFP committee approaches the task with seriousness. There are certainly mistakes, and there’s certainly bias at play (some of it subliminal, as we’ll get to in a second). The process is imperfect, in ways both necessary and not. But there’s a process, and there’s a mandate (rank the best teams), and there’s a focus which is lacking in the AP poll.
Thankfully, the latter portion of the season is governed by the CFP rankings, and the latter portion of the season is what ultimately counts. When possible, the college football season converges upon sanity as its weeks go on. Unfortunately, though, there are times when the early-season AP poll does matter. We’ve talked about it plenty in previous posts, but one more time, humor me and consider Washington State:
Washington State is 4–0. They’ve beaten Washington, San Jose State, Texas Tech, and Portland State. The best of those four—Washington—they beat at a neutral site inside the Huskies’ home city. The other three games were home affairs played in Pullman. The San Jose State and Washington wins were both close and dramatic. But San Jose State is expected to contend in the Mountain West, per our best estimates, ranked fourth in the conference by each of Movelor, FPI, and SP+. Washington might be one of the 25 best teams in the country, with an average ranking of 23rd across those three systems, each of which evaluates the quality of a team’s play (as opposed to evaluating the quality of a team’s wins and losses, where Washington is still more lacking).
This is not a résumé that makes anybody swoon. The Cougars didn’t go to Ann Arbor and throttle the defending champions, like Texas did, or knock off Notre Dame at Notre Dame Stadium, like Northern Illinois. But compared to Illinois’s résumé, which consists of wins over Nebraska, Kansas, Central Michigan, and Eastern Illinois—like WSU, two of these wins were close—the Cougs’ stacks up pretty well. ESPN’s FPI-based Strength of Record metric grades Washington State’s résumé as the seventh-best in the country coming out of last weekend. Illinois’s is also good. ESPN grades it as a little better than Washington State’s. But only a little bit better. Washington State’s is seventh. Illinois’s is sixth.
The AP poll ranks Illinois 19th. It does not rank Washington State.
Is Illinois a better team than Washington State? We’re not sure. Movelor narrowly says yes. FPI and SP+ narrowly say no. It’s very close. The point is: Washington State and Illinois are comparable teams with comparable résumés. The difference is that Illinois’s marquee win—a victory over a Nebraska team who hasn’t made a bowl in eight years—was played on national television on a Friday night. Washington State’s? The Apple Cup happened on Peacock in the middle of a busy Saturday, while AP voters were busy doing their primary jobs.
If the AP poll is currently ranking the best 25 teams, then sure, Washington State should be unranked. But under that logic, Illinois should be unranked, and NIU should have been unranked last week, and Boise State (average Movelor/SP+/FPI ranking: 36th) should be unranked as well. The truth is that sometimes, being good matters, and sometimes, being accomplished matters, which means that really, nothing matters. There is no rubric. There is no criteria. A number of AP voters likely couldn’t describe their own rationales. They certainly couldn’t do it coherently. Incoherence comes out because incoherence goes in.
For Illinois, the ranking doesn’t matter that much. It’s nice—Illinois can sell more tickets and sell more jerseys, and even if they come up short against a tough upcoming stretch of schedule, their fans can remember a recent time their team had a little number next to its name—but their Week 5 ranking is not going to help or hurt their playoff chances. Playing in the Big Ten, Illinois will be thoroughly vetted through its own play. The CFP committee will be highly aware of every performance on Illinois’s schedule, and it will be familiar enough with every opponent Illinois plays to render a just verdict.
Washington State does not enjoy this luxury.
Thanks partly to economic and cultural forces and partly to mistakes made by Washington State’s administration, the Cougars were relegated this season to purgatorial status, somewhere between the Power Four and the Group of Five. They can’t win a conference championship, something which for five of nine champs will come with automatic playoff admission. They play mostly a Mountain West schedule, limiting their exposure to the committee.
Washington State is a playoff contender. In our most recent simulations, they finish the regular season undefeated 13.5% of the time, and their schedule—though weaker than that of Power Four teams—is strong enough that undefeated would for most teams in most seasons result in a top-ten ranking, going by the committee’s established precedent. But because their schedule offers few opportunities to prove themselves, and because they can’t win a conference championship, Washington State will be the most subjectively evaluated team in the country. They must be. It is their lot.
This, again, would be fine. The committee’s process is robust enough for Washington State to get a fair shake, provided there aren’t external factors working against them. The reason some degree of subjectivity is necessary in the committee’s process is that exceptions exist. You can build the perfect ranking formula—and believe me, we’ve spent days and days working at this task—only to see it surprised by an exception. Formulas malfunction when they were built on a sample not representative of the reality they proceed to encounter.
The problem, though, is that AP poll issue. Washington State is unranked. Committee members don’t know to pay attention to them yet. Committee members’ subliminal impressions are anchored on what early-season AP polling tells them. Committee m members, like casual fans, see the little number next to a team’s name and, whether they want to or not, they take note.
This is anecdotal, but if you don’t believe me, go back and look at USC’s final scores, AP ranking, and CFP ranking throughout the 2022 season. The team had a strong résumé, for a while and to a point, but it never matched the traditional résumés of playoff teams. The wins weren’t good enough, going by opponent or going by game control. USC entered that year’s UCLA game, on November 19th, 1–1 against opponents in the top 25 that week, their only win a narrow survival in Corvallis over a team the committee ranked 23rd on November 15th. USC, in those same rankings, saw itself ranked 7th. Why? Because they opened the season ranked 14th in the AP poll, and while their performance never reached top-ten status, perceptions were anchored high enough that the committee just kept moving them up as other teams lost.
In a fair world, one where Washington State is given treatment in between Power Four and Group of Five status (going by what’s customary from the CFP committee), Washington State would have a playoff chance even with an 11–1 finish. That’s what our model, with a strong track record of predicting CFP rankings, indicates. In this same hypothetical fair and just universe, Washington State would be in at 12–0. In the real universe? Even 12–0 might not be enough. That’s the damage perception can do.
Thankfully for WSU, their opponent this weekend is inexplicably ranked. Boise State’s a good and fun team, but they’re 43rd in SOR and they aren’t in the top 25 of any of FPI, SP+, and Movelor. AP voters are trying not to screw the little guy, and they’re trying to make it look like they know what they’re talking about, but there is no logic by which Boise State belongs in any top 25 right now. Washington State, meanwhile, is the first unranked team, because at least some AP voters are good and decent people. If Washington State wins on the blue turf, Washington State will finally get that little number next to their name. It won’t be as good of a number as it would be for anyone else with their résumé, but it’ll be a start. Thankfully for college football’s integrity, WSU is an underdog this weekend who will probably lose, and while they’ll be favored in possibly every other remaining game, they’ll probably lose one of those too. Washington State will probably not go 11–1, let alone 12–0. All of this will probably not end up mattering.
Still, there’s a chance it does matter, and even if it doesn’t, an AP poll governed by ignorance and sloth is a bad institution to keep around. We need to get rid of it. It’s time. How do we do that? By creating a better alternative, promoting it, and eventually—in a few years—getting networks to use our little numbers instead of the AP’s.
**
Enjoying what you’re reading? Subscribe to our Substack to receive it every day in your email inbox.
**
We’re calling this approach the BETTER Top 25. That’s it. There’s not actually an acronym. (Because Evidently This Thing Is NEcessaRy was the best we could do on short notice.) We understand the value of a top 25, and we want it to be better than the AP’s approach. Here’s how we think it should work:
Once the CFP committee releases its own rankings, the BETTER Top 25 should go away. There’s no need for a second top 25 during CFP ranking season. The CFP rankings are the rankings that matter, and that is good.
The ranking should, in the preseason, revolve around how good every team is expected to be. That’s the point of a preseason top 25, and that again is good. (This is the one thing—and I mean the absolutely only thing—that the AP poll does well.)
Résumé should matter as early as it can. That’s something fans want. At this point, it’s still too early in the season to go fully off of résumé (we just checked what our CFP rankings predictor would say at this point in the season, and it has Missouri 5th but Ohio State 15th), so September will necessarily involve some blending of how good teams are and how accomplished they are.
As far as résumé is concerned, the rubric should mirror the CFP committee’s implied criteria as closely as possible. The ideal state here is one where this rankings rank what the committee would rank were the committee releasing rankings earlier in the year.
Here’s how we’re going to aim ourselves towards that objective. Consider this the first attempt.
For preseason (this would come next year), we’ll use an average of Movelor, SP+, and FPI. SP+ and FPI are available quickly and publicly, which makes them useful. All three of these systems have an established track record of fairly accurately predicting how good teams are. They tend to average error margins within a point per game of those produced by betting markets.
As the weeks progress towards the point where every Power Four team has played a conference game (this is when the CFP rankings predictor can start working respectably well), we’ll gradually shift the proportion between the predictive, “how good is this team” ranking composite (Movelor/SP+/FPI) to the reflective, “where would the committee rank this team” ranking. In practice:
- Preseason: 100% how good teams are (predictive rating)
- After Week 1: 83% predictive rating, 17% CFP committee proxy
- After Week 2: 67% predictive rating, 33% CFP committee proxy
- After Week 3: 50% predictive rating, 50% CFP committee proxy
- After Week 4: 33% predictive rating, 67% CFP committee proxy
- After Week 5: 17% predictive rating, 83% CFP committee proxy
- After Week 6: 100% where the committee would rank the team
Right now, then, we’re using Movelor, SP+, and FPI as one third of the BETTER Top 25, combined, while our CFP ranking predictor comprises the other two thirds. Here’s what we get:
1. Alabama
2. Texas
3. Tennessee
4. Miami (FL)
5. Mississippi
6. Georgia
7. Missouri
8. Ohio State
9. Utah
10. Penn State
11. Notre Dame
12. Louisville
13. Oregon
14. UCF
15. Indiana
16. BYU
17. Michigan
18. Illinois
19. Oklahoma
20. Washington State
21. Iowa State
22. Oklahoma State
23. Arkansas
24. Pitt
25. Texas A&M
This is far from perfect, especially towards the top. Alabama shouldn’t be ranked ahead of Texas, based on how good they are and what they’ve shown. Missouri shouldn’t be in the top ten, by that same logic. Notre Dame doesn’t deserve such a good number, having lost at home to a team who’s now 0–1 in the MAC. You could argue for ranking LSU, USC, or Clemson, as the AP Poll has done. But this gets Boise State out of there until they earn it. It removes Kansas State, who just got punked in Provo. It gives credit to teams who’ve accomplished something so far, even if the accomplishments didn’t bring a lot of attention their way. Most importantly, it comes from a coherent and consistent line of thinking. It means something. It isn’t just an exhausted beat writer scribbling down a top 25 while they drink their morning coffee after a long day.
This isn’t perfect. It’s BETTER.
We do want to fine-tune this before we start promoting it at all, maybe through fiddling with the weighting between predictive and reflective rankings, maybe through working in a Strength of Record component. This is an offseason project. We want it polished before we really start our AP war. For now, though, this is a decent start. This is where a reasonable poll would rank Washington State right now. This is where the AP voters would probably have the Cougars, if all of them knew what the Cougars have been up to. We may revisit it next week.
**
Thanks for being here. If you like what you just read and want it delivered to your email inbox every morning, subscribe to our Substack.
