Evaluating Fangraphs Playoff Projections
Photo Credit: Andrew Dieb/USA TODAY Sports
As the September playoff race intensifies, key division battles are shaping up across the league: In the AL East, the Yankees and Orioles are going head-to-head, while it’s the Guardians and the Royals matching up in the AL Central. The Wild Card picture is just as thrilling, with the Twins and Tigers competing for the final spot in the AL and the Padres, Diamondbacks, Mets, and Braves all contending for Wild Card berth in the NL. Meanwhile, the Phillies, Dodgers, and Brewers are jockeying for the top two seeds in the National League, hoping to capture the first round bye.
In this post, I’ll introduce a win probability model that uses a team’s expected win percentage against a .500 opponent to determine its chances in a head-to-head matchup. I will refer to it as the Paraball Win Probability Model. I’ll then compare the results from my model with Fangraphs’ Playoff Simulation to highlight which playoff teams Fangraphs overvalues and undervalues.
How The Paraball Win Probability Model Works
If the Favorite has a Win % >= 50%, the output is
Favorite Win % / (Favorite Win % + Underdog Win %) + (Favorite Win % - Underdog Win %) * 0.6
If the Favorite has a Win % <= 50%, the output is
Favorite Win % / (Favorite Win % + Underdog Win %) + (Favorite Win % - Underdog Win %) * 0.37
The coefficients of 0.6 and 0.37 were derived empirically to minimize the model error when run on ten known cases (below). Table 1 shows how the model performs when tested on the several known cases.
Table 1: Model performance when tested on several known cases
As seen in Table 1 above, the model showcases a mean absolute error close to 0 in the known cases in which teams have win probabilities between 70% and 30% (which is the case for every team this year except for the White Sox).
How Does The Paraball Model’s Projections Compare to Fangraphs Projections?
Table 2 compares the Paraball Win Probability Model output with the Fangraphs output.
Table 2: Paraball Rest of Season (ROS) Win % compared to Fangraphs ROS Win %
The standout number from Table 2 is Fangraphs’ projection that the White Sox will win 42.5% of their remaining games, compared to my model’s much lower estimate of 23.2%. With a season win percentage of just 22.3% and their remaining opponents averaging a 46.8% win percentage, Fangraphs projects the White Sox to win the equivalent of 39% of their games against a .500 team. This would suggest they should have around 58 wins after 148 games, meaning their actual 33-115 record represents a 2.5 standard deviation underperformance—something that has just a 0.59% probability.
I find it hard to believe that the White Sox have been that unlucky this season. Their Pythagorean expected win-loss record, based on run differential, is 41-107, which aligns much more closely with their actual performance than Fangraphs' projection.
But enough about the White Sox. How does Fangraphs view this year’s playoff contenders?
Figure 1: A comparison of the Paraball ROS Model’s Win % (y-axis) versus the Fangraphs’ ROS Win % (x-axis) for each team. The 45 degree line is included to visualize the differences between the two models.
As we can see from Figure 1, Fangraphs is particularly bearish on the Brewers and Guardians, somewhat pessimistic about the Royals, Mets, and Orioles, while being bullish on the Astros and Braves.
It seems that recent playoff success plays a significant role, alongside each team's performance this season, in shaping Fangraphs' projections.
