How Our Virtual NIT Game Simulator Worked

We’ve been asked a few times how our game simulator worked for the Virtual NIT (and the Illinois high school results we simulated for the Chicago Sun-Times). We’re normally hesitant to share the exact details of our models, in part out of a desire to keep them proprietary while we’re growing as a site, and in part because the models are growing rapidly year-to-year—making one year’s details untrue and possibly misleading the next season. However, for the game simulator, we don’t envision using it again, it’s very simple, and it seems to be of interest to some. So, we’re making this self-review of the system public.

We’ll go through this in three parts:

1. Our Purposes: Why we made this, what we wanted to use this to do.
2. The Methodology: How exactly the simulator worked.
3. Strengths and Weaknesses

Our Purposes

The purpose of the game simulator was to give reasonably realistic, objective results in a Virtual NIT, run for entertainment purposes on this site. We wanted each simulation to take little time to run. We wanted the simulator to be built quickly. We were willing to sacrifice quality in exchange for simplicity to achieve these two prior goals.

A note: There are some minor differences between this simulator and the one we used for the Illinois high school simulations. The methodology was the same, but the inputs were different in a few ways. We decided to share the methodology behind this one, rather than the Illinois high school one, because the Virtual NIT was more integral to our site and its simulator included fewer assumptions (we had tempo data for the Virtual NIT, and we didn’t have to adjust the efficiency margins for strength of schedule ourselves).

The Methodology

Before we get into this: We relied entirely on KenPom for our initial data, and some of the information comes from behind his paywall, so first, a more-than-deserved plug: if you enjoy college basketball, a KenPom subscription is worth the low cost. We hope he’s ok with us sharing a few minor details from his data here. Ken Pomeroy, if you’re somehow reading this, let us know if you want us to pull this post or alter it in any way. Also, thanks for making such a good site. You are our favorite.

Team Data

For each team, we assembled four pieces of data:

1. AdjO/Poss – Adjusted Offensive Efficiency: How many points a team would be expected to score, per possession, against an average defensive opponent. This was taken directly from KenPom.
2. AdjD/Poss – Adjusted Defensive Efficiency: How many points a team would be expected to allow, per possession, to an average offensive opponent. This was taken directly from KenPom.
3. AdjT-Inv – Inverse Adjusted Tempo: How long an average possession would be expected to take for each team, in seconds, against an opponent playing a perfectly average tempo. To calculate this, we divided 2400 (the number of seconds in a regulation college basketball game) by KenPom’s adjusted tempo number for the team, then divided the result by two to split it down to just one team’s turn in the possession, rather than the entirety of the possession, in which each team would possess the ball once (example of how this was done: Texas Tech, with an adjusted tempo of 67.3 possessions/game, had an AdjT-Inv of 17.8, which is equal to 2400 divided by 67.3 divided by 2).
4. HCA – Home Court Advantage: The number of points better, per game, that a team is estimated to play at home relative to on the road. Taken directly from KenPom, but from behind the paywall.

Offensive Scoring Frequencies

In each game, we built what was effectively a pie chart for each team breaking down how frequently they’d score zero, one, two, three, and four points in their respective possessions.

We started this by calculating out an expected points per possession (ExpP/Poss) for each team. For the home team, this calculation went as follows, with 1.024 being the average number of points per possession in Division I basketball this year (thanks, KenPom) and the HCA calculation spreading out home court advantage on a per-possession basis.

For the road team, the calculation was the same, except HCA became HCA_Opponent and became a negative value.

For neutral-site games, HCA became zero, eliminating that term from the equation.

Next, we turned ExpP/Poss into the aforementioned “pie chart.”

In nearly every possession, a basketball team scores either zero, one, two, three, or four points. There are ways to score more, but it is rare for those to happen—rare enough that in this simulator, we ignored them.

Four-point possessions are also rare. We didn’t research how rare, exactly, but we assigned these a one percent probability, regardless of the teams playing, so we’d average one or two four-point possessions per game.

We also decided to make the frequency of one-point possessions constant between teams. Our rationale for this was that one-point possessions must happen due to free throws, and while there are certainly correlations between free throw percentage, free throw attempt frequency, opponent free throw attempt frequency, and scoring/opponent scoring, those correlations aren’t particularly strong. Given the average free throw percentage in Division I basketball this year was 70.8%, and given teams shot, on average, 0.326 free throws for every field goal they attempted (again, thanks, KenPom), the probability of scoring one point per possession would be roughly ten percent if every free throw attempt was part of a two-free throw possession with no field goal made and the first and second free throws were independent events. However, a free throw is sometimes added onto a play after a field goal attempt is made; some free throws in college basketball come in bonus situations; and the first and second free throw attempts are not independent events—their results are correlated, because they’re shot by the same shooter. After considering all these things, we landed on an eight percent probability for every team of scoring one point in a possession. This is, as the four-point possession frequency assumption was, a very rough assumption.

We needed one final assumption to turn the frequency distribution into a simple calculation, and that was the ratio between two-point possessions and three-point possessions. The Division I ratio between two-point field goal attempts and three-point field goal attempts this season was roughly 1.67-to-1 (thanks, KenPom). Given differences in shooting percentages, the ratio would become roughly 2.5-to-1, so after doing some estimating regarding the frequency of free throws in different situations (bonus free throws, free throws on a two-point attempt, free throws on a three-point attempt), we decided on a 3-to-1 ratio (another very rough assumption). The result of this was the following point distribution breakdown for each team, calculated off their ExpP/Poss:

Four Point Possessions: 1%
Three Point Possessions: (Two-Point Possession Frequency)/3
Two Point Possessions: (ExpP/Poss – 0.12)/3
One Point Possessions: 8%
Zero Point Possessions: 100% – (Sum of all other frequencies)

For a team with an ExpP/Poss of 0.92 in a given game (this was Texas’s ExpP/Poss for the Virtual NIT Championship game against Texas Tech), these calculations work out as follows:

Four Point Possessions: 1%
Three Point Possessions: 9% (27%/3)
Two Point Possessions: 27% (0.92-0.12)/3
One Point Possessions: 8%
Zero Point Possessions: 55% (100% – [1%+9%+27%+8%])

Running the Simulation – Points Scored, Each Team, Each Possession

In our simulations, each possession consisted of each team having one chance to score. To determine how many points each scored, we generated two random values—one for each team—between zero and one, then plugged those values into the respective teams’ frequency distributions. As an example, here’s how many points a team with an ExpP/Poss of 0.92 in a given game (Texas’s distribution from above) would score in a possession if their random value fell within the following ranges:

0.00-0.55: Zero Points
0.55-0.63: One Point
0.63-0.90: Two Points
0.90-0.99: Three Points
0.99-1.00: Four Points

Running the Simulation – Length of Each Possession

With the points scored on each possession, listed above, we created a ledger of a scoreboard—a running tally of how many points each team had following each possession. The next step, then, was to determine how long each possession lasted, so we could cut games off at halftime and at the end of regulation (none of our games went to overtime—something we’ll address in the next section).

To determine possession length, we used each team’s AdjT-Inv and two more random values—one for each team’s portion of the possession, with each again between zero and one. We wanted to use a normal distribution for this, for simplicity’s sake, but that would have resulted in having some possessions with negative lengths. Because it didn’t strike us as feasible for two teams to both possess the ball and score in a span of fewer than ten seconds (the “and score” in there is important—our possession lengths were independent of scoring on the given possession, so possessions in which teams quickly trade turnovers couldn’t be baked in without making that independent relationship a dependent one), we set five seconds as the minimum amount of time each team could possess the ball. This skewed our distribution, so rather than using the true mean possession length we would expect between two teams as our mean in the normal distribution, we multiplied that number by 0.855 and used a standard deviation of 16 (we landed on both these numbers through a few hundred simulations with different coefficients—our goal was to keep 95% or more of games between 60 and 77 possessions).

How this worked, in practice, was that we took Team 1’s random time-of-possession value and calculated its inverse value in the normal distribution (mean = 0.855*(AdjT-Inv₁+ AdjT-Inv₂)/2 seconds, standard deviation = 16 seconds), did the same with Team 2’s random time-of-possession value, then added the two together.

Running the Simulation – Overall

In sum, what we did, each possession, was allow random values to assign each team a score for that possession and assign that possession a total length. We tracked the score and time on a ledger of a scoreboard. The final possession of each half was the first in which the time remaining was below 0:00.

Strengths, Weaknesses

The strengths of this simulator lie around its mean results and its simplicity:

Were a game run through this simulation 10,000 times (or a similar large number), its mean result would be a fairly accurate reflection of the true expected outcome of that game. It was also quick to build and execute.

The weaknesses of the simulator lie around its standard deviations, and—again—its simplicity:

Were a game run through this simulation 10,000 times (or a similar large number), the standard deviation of its results would likely not be accurate. We don’t know this with certainty, but there were more upsets in our Virtual NIT than we would have expected were the simulator calibrated to expected results (by “calibrated to expected results,” we mean calibrated such that a team expected to win a game 90% of the time would, in fact, win that game 90% of the time). Further deepening this suspicion is the fact no games went to overtime and nearly half of games were decided by double digits (one was decided by 59 points), implying a distribution of outcomes with a much larger standard deviation than a better-calibrated simulator would have. The first step to addressing this perceived problem would be to confirm its existence by testing the simulator on at least a few matchups (chosen to reflect different probabilities of victory—one in which the favorite was a 90% favorite, one with a 70% favorite, one with a 55% favorite, etc.) 10,000 times each, or a similar large number.

If the problem does exist, the simplest possible solution would be to alter some of the assumed numbers in the scoring frequencies and test those altered values against expected outcomes, iterating until we landed on a strong set of assumed numbers, but there are other areas in which the simulator can be improved:

The idea of using a normal distribution with a hard minimum to determine length of possession is dramatically simple. Further research into lengths of possession in Division I men’s basketball games would give us a better picture of the distribution, making that piece of the simulator more realistic.

A more advanced simulator would include data on team’s individual three-point attempt and free throw attempt frequencies, tailoring each team’s frequency distribution to those numbers.

A more advanced simulator could include tendency data—teams’ block rates, rebounding rates, etc.

Some exploration into the last few minutes of a large sample of games would yield a more accurate picture of what may be a very different basketball in terms of time of possession and scoring frequency distribution.

***

Overall, for our simplistic, entertainment-based purposes, the simulator worked well enough. It had major flaws, but those flaws were at least objective flaws, with the likely result being that more randomness surrounded the simulated games than would surround real games.

Hopefully, we don’t have any need to revisit this simulator for future tournaments and events. However, if you’re interested in running simulations on your own, especially during this time of social distancing, reach out and we’d be happy to send you the file, along with some instructions. It can be run in Microsoft Excel.