Joe’s Notes: A Bad Week for Probability and Sports

The NFL Draft began last night, with ESPN showcasing an…interesting…set of something they called probabilities.

At a glance, the second tweet holds up. It has its issues, but it mostly holds up. It’s probably wrong, because we now know the model is a farce, but it’s not obviously dumb taken alone. The first tweet is the problem. If your model says something has worse than a 1-in-1,000 chance of happening and it happens, there is almost certainly a major flaw within that model. That’s ok—all models have problems, and building one for something as complex as the NFL Draft is an admirable aspiration—but the response at that point shouldn’t be to continue to refer to your model as the standard of truth. 0.049% (or whatever the number was) wasn’t the true probability of Will Levis dropping to the second round. 0.049% (or whatever the number was) was the probability ESPN’s model put on Will Levis dropping to the second round. The model was imperfect, and again, that’s ok, but the fact ESPN kept referencing it and keeps referencing it now tells you a lot of what you need to know about ESPN’s probabilities in every realm—the Allstate Playoff Predictor, college basketball bracketology percentages, even strong models like NFL FPI. What does it tell you? Well.

ESPN doesn’t care about its analytical integrity.

ESPN uses probabilities for content and content alone.

This isn’t to say that those working in ESPN Analytics aren’t serious about their work. My impression is that they’re serious, smart people who work hard. My impression is that they’re in a difficult position in which pieces of the organization which outrank them are asking for probabilities they cannot accurately give and then broadcasting those probabilities no matter how accurate they are. If analytics were the point, and if analytical integrity were embraced, ESPN Analytics would answer for its foibles. We’d get a mea culpa and a what–went–wrong about the Levis probabilities. We’d have gotten the same back in the 2020 college football season when, while others (us) shelved their College Football Playoff models in the face of too-difficult Covid scheduling uncertainty, every ESPN broadcast every weekend kept trotting out obviously wrong probabilities about USC’s playoff chance.

Two problems arise from this approach.

The first is that it’s hard to know which ESPN probabilities are worth looking at and which are, more or less, clickbait.

The second is that this sets back the credibility of the sports probability industry. If the Worldwide Leader’s probabilities are so clearly and badly wrong, fans are less likely to trust blogs like The Barking Crow. ESPN is ruining this for the rest of us, accustoming fans to blow off probabilities when it’s only ESPN’s which should be blown off.

In related news, FiveThirtyEight is going away. Pieces of it will almost certainly return in some altered form, but layoffs there this week gutted the organization, Nate Silver has said he won’t remain with ABC when his contract soon expires, and the FiveThirtyEight name is owned by Disney and won’t join Silver wherever he pops up. This isn’t the worst thing in and of itself. Much like forests need fires, I’d argue every organization needs restructuring now and then if it’s going to grow better, though it’s obviously terribly hard when people are put out of work. Silver will return in some place in some capacity, and one would imagine that FiveThirtyEight’s most useful models—the political ones, the NBA one, the soccer ones—will be out there in some form, especially with Silver having consistently made their methodology so public.

No, the destruction of FiveThirtyEight isn’t the worst thing. The destruction of FiveThirtyEight is, though, a canary in the coal mine. When FiveThirtyEight was initially acquired by Disney and assigned to ESPN, the obvious thought was that probability models like Silver’s would be a staple of sports media. The obvious thought was that people wanted serious probabilities. Now, at least on the ABC and ESPN front, it’s become clear that they don’t. Between ESPN’s willingness to insult its viewers with farcical data and Disney’s determination that online readers didn’t want what FiveThirtyEight was peddling, the clear thought within Disney and its subsidiaries now is that if readers do want probabilities, they don’t care if those probabilities are serious. Publishing the Will Levis silliness days after gutting a pillar of modeling integrity and accountability sends exactly one message to consumers: “You are dumbasses, and we will treat you as such.”

There will always be a market for serious probabilities, especially with sports betting becoming more and more legal across the United States. Also, for as disappointing as these developments are, the public understanding of probability and its concepts has undoubtedly improved over the last ten or twenty years, with an argument to be made that Silver played a big role in this. Speaking personally, I can say that I have thought about probability more and understood it better because of Nate Silver. Speaking personally, I can say that The Barking Crow might not exist were it not for Nate Silver taking probabilities as seriously as he’s taken them. Much of our site’s content falls on the comedy side, but much of our site’s traffic (and therefore our case for existing) comes from our numbers, and FiveThirtyEight showed us the way on those. Every model we create owes a major debt to FiveThirtyEight, and many have been directly built with FiveThirtyEight’s endorsement of elo at their base.

If ESPN wants to treat consumers like idiots, ESPN can do that. It’s sad, and it’s disappointing, and it creates a lot of muck through which one must wade to find the truth, but ESPN can do what it wants and viewers can do what they want. For those of you that look at our models, though, we can promise a few things:

• We won’t publish models if we think they’re going to put egg on our face.
• When models put egg on our face, we will acknowledge it, explore what happened, and adjust to what went wrong.

ESPN might think you’re an idiot, but we do not. We respect our readers. We learned that from FiveThirtyEight.

The Other Will

Will McDonald IV was drafted last night, and was drafted much higher than just about anyone had him going. The Jets used the 15^th pick they landed with after the Packers trade to take McDonald nearly a full half-round before the consensus had him going. This might not be good for the Jets—of all the franchises, I wouldn’t rank the Jets highly among those to trust when they buck consensus—but it’s great for McDonald, who’ll make an extra four million dollars or so, and it’s great for Iowa State, who broke a 50-year drought in which the Cyclones didn’t produce a single first round pick.

McDonald will join Breece Hall and Allen Lazard in New York, making the Jets the clear leader as Iowa State’s NFL team. This might not sound great—again, not sure you want to be associated with the Jets—but compared to where Iowa State was a decade ago, this is a sensational development. The fact Iowa State has enough of a presence in the NFL for part of it to be concentrated on one team is a breakthrough for this football program, and it’s something that will remain true at least for a few years even in the sad scenarios where Matt Campbell moves on. Recruits can make the NFL from Iowa State. Iowa State players can be drafted in the first round. It’s possible, and it’s happening, and that’s something that changes a school. It’s one step on a big staircase, but it’s a step. And it is so hard to climb those stairs.

The Packers Trust Jordan Love

For years, the Packers have drafted defensive players in the first round. For years. They did it with Rodgers under center. They did it with Favre. For years, the Green Bay Packers have signaled that they believe defensive players are consistently the best value available in the first round of the NFL Draft. Last night, drafting Hawkeye edge rusher Lukas Van Ness, they signaled that this isn’t changing just because Aaron Rodgers is gone.

A subset of Packers fans are frustrated that Brian Gutekunst didn’t take someone who could catch passes, or at least an offensive tackle to make Love’s job easier in the pocket. This is the wrong way to think about it. If Love is bad enough that Gutekunst & Co. think Love needs that kind of help, that Love needs it badly enough to justify a wholesale pivot from a decades-long approach (which has worked quite well), then the Packers have a much bigger problem than who’s catching passes. They have a problem with who’s throwing them.

What the Van Ness selection signals, I’d argue, is that the Packers really do like the guy they have taking snaps, and that they respect him enough to not sacrifice value in order to prop him up.

Are the Knights the Favorites?

You can build a case a number of ways for who’s the championship favorite regarding any championship. Some are more accurate than others, but there are varied approaches and it’s good to see a range of them. ESPN, for example, would probably like to just say that the Bruins are 100% likely to win the Stanley Cup and then tweet a bunch of emojis of heads exploding if it doesn’t happen.

One approach to determining the Stanley Cup favorite is to look at who has the best path. After becoming the first team to advance out of the first round, with home ice awaiting in the second round and again in the Western Conference Finals should they get there, the Vegas Golden Knights have, by one conception, the best path.

This breaks down almost immediately. The Knights will probably play the Oilers next, who are probably better than the Knights and are probably among the four or five best teams in the playoffs, if not the second-best. They are also, crucially, not the Bruins, whom every piece of information continues to indicate are the best team in the National Hockey League.

Still, the Knights have done the most so far this postseason. If this were a regular season in which everyone had played five games, they’d be at the top of the standings. They’re quiet, but they’re a legitimate Cup contender. That’s a case we don’t need to stretch to make.

Other series, other thoughts, in both the NHL and the NBA:

• The Hawks really almost had the Celtics. Leading the game more than halfway through the fourth quarter? There was never so much doubt about the Celtics winning the series, but Game 7 was on the table. I’m curious, from this, about the invincibility of Boston. It’s reading like Milwaukee in some senses, where the foregone conclusion has picked up a head of steam and left the numbers in its wake.
• How do the Sacramento Kings respond after those three losses? Thursday’s and Sunday’s were the most gutting, because of Draymond Green’s absence in the first and the final-seconds nature of the second, but Game 5 is what turned the series firmly towards the Warriors. Surely, Sacramento puts up some fight tonight, right?
• Are the Lakers banged up? LeBron James, Anthony Davis, and Dennis Schröder are all on the injury report again, but all played in Game 5, which is one of those things that makes you question the nature of injury reports in general. When you think about them for a medium amount of time, they’re strange. Long enough, and it makes sense, short enough, and you don’t question, but in the middle? Odd little lists.
• The Nuggets appear to badly need to win Game 1, if we’re taking the odds at face value. Given Phoenix is favored to win the series and Denver’s favored to win tomorrow, it would stand to reason that Denver has few paths in bettors’ eyes with a Game 1 loss. I’m suspicious of the line, but I do think there’s unusually high leverage for a Game 1 on this. Part of that is because Game 1 should theoretically teach us things about the Suns, while the Nuggets are more of a known quantity.
• The Heat and Knicks play on the early side on Sunday, and one possibility is that one or the other will experience a letdown. It would fit more with Miami’s narrative arc, and in the NBA more than other sports, the narrative arc isn’t nothing, but it does exist more in our guts than those of the players. … I think.
• On the ice tonight, the Islanders are slight favorites to send that series back to Raleigh, which would be a huge turnaround after Games 1 and 2. The Islanders answered, then they answered again. Can they keep answering?
• Can the Panthers force a Game 7? It’s a lot like the Hawks Game 6 last night, but to a much scarier extent for Boston given hockey’s more uncertain nature in individual games.
• I can’t tell if either of the Wild or Stars are good. That series feels like it’s happened in a bubble.
• I hate the nature of this Game 6 for the Kraken, where it feels like they absolutely need to win even though they don’t. I know intellectually that they have a great chance of winning one of two as things stand right now, but the conditional probability piece of this—the one where if they do lose tonight, they’re highly likely by hockey standards to lose Game 7—rattles me.
• The Leafs don’t need to win Game 6, but Leafs fans might need the Leafs to win Game 6. It would be very Lightning to come back here. Last night was startling, but that’s what a good goaltending performance can do.
• The Devils were so dominant last night. The Rangers look like they don’t have any answers. Can New York flip this back? It flipped once before.
• Do the Knights get a rest advantage if the Oilers can’t put this series away in six? They’ve already got at least two days, plus one fewer game of bruising and straining. Something to ponder.

Starting Pitching Prospects and Sample Size

Caleb Kilian is getting another go at the MLB level tomorrow after posting a 4.97 FIP and a 4.37 xERA in three starts last summer. In those starts, he had an ERA of 10.32 and he walked more than a batter an inning.

In that context, the FIP and xERA are remarkable. It is very difficult to have numbers that normal with other numbers that disastrous. Still, nobody would call Caleb Kilian’s earliest tastes of the big leagues a success.

Kilian isn’t a top 100 prospect right now, but he isn’t far away from that. FanGraphs has him eighth in the Cubs system, and the Cubs system, though bottom-heavy, is one of the best in the business at the moment. In other words: This is a rather important call-up for Chicago’s National League team. Just as it is for Kilian himself, who’s struggled with the long ball so far this year in AAA.

It’s easy to overhype prospect debuts. At Kilian’s level of potential, they’re numbers games, each one a single roll of the dice among many. Still, if the righty can click this time around, it’d nudge those numbers upwards for the Cubs. The Cubs don’t need Caleb Kilian for the Cubs to be good again one day. But if Caleb Kilian can wind up good after all, that’s one fewer spot the Cubs will need to fill.