The Beane Count in Baseball: Predicting Success on Exception Fallacy Makes this Cute Stat Pure Junk

If you have ever seen the 2011 box office hit Moneyball, or read the lesser-known, previously released book of the same name by Michael Lewis, then you may be keen to the field of sabermetrics. If you’re not, and you’re a baseball fan, then I’m about to introduce an exciting aspect of the game to you.

In the movie Moneyball, Brad Pitt plays Oakland Athletics general manager Billy Beane. At the time, and really to this day, the Oakland A’s remain one of baseball’s smallest market teams. This imposes some financial constraints on the team: because they don’t sell tickets and other amenities at hundreds of dollars or have unusually large TV contracts, they have a smaller cap on money to spend on players. This means that the A’s are less likely to bid on high profile players and usually lose out on bidding wars with other teams for players they’ve developed through their minor league system who become emerging superstars. This is the event that sets up the premise of Moneyball.

With strapped resources and skepticism about how the game is scouting valuable players, Billy Beane stumbles across a young assistant to the general manager of the Cleveland Indians, Paul DePodesta (portrayed as Peter Brand by Jonah Hill in the movie). DePodesta introduces Beane to sabermetrics, the statistical analysis of baseball pioneered by Bill James and a play on the acronym SABR (Society for American Baseball Research). With DePodesta and sabermetrics in his corner, Beane is able to build a team that recovers from the cellar of the American League West division and go on to hold the longest winning streak of a regular season and runner up in the 2002 American League Championship Series. Since Beane’s success swept across baseball, sabermetrics has been a common practice in both small and large market teams.

One of the hallmarks of sabermetrics is the eccentric, and often nonsensical statistical algorithms it uses to predict player success: for instance, WAR or wins above replacement, is marketed as an important, all-inclusive statistics that answers the question of a player’s value. Essentially, if his team were to lose him to injury and replace him with a free agent or AAA minor league call-up, how many wins would the club lose out on that year. Pretty weird, huh?

WAR is just one example of this, and the Beane Count is another. Aptly named in honor of Billy, the Beane Count was created by baseball writer Rob Neyer to rank teams based on summing a team’s homeruns, walks, homeruns allowed, and walks allowed. In this case, the lower a team’s Beane Count, the better the Beane Count. Neyer, who used to work for Bill James before moving to ESPN, is someone you’d expect to be meticulous about their creation of baseball metrics, but in all honest, the Beane Count appears to be a junk stat.

Let’s take the 2016 season to explore the claims of the Beane Count as a junk stat. First, the 2016 National League Beane Count as maintained by ESPN.com. Of the top 5 teams, 4 made the playoffs (80% of the sample), with 2 (Chicago and Washington) being division leaders, and 2 (San Francisco and New York) being wild card leaders. St. Louis missed this year’s playoff by only one game. So far, the Beane Count seems to be looking pretty good.

Let’s now look at the 2016 American League (AL) Beane Count. Remember, the Beane Count is the summation of the two “Batters” category ranks and the two “Pitchers” category ranks. So, Toronto would have a Beane Count of 13.0 from 3.0+1.0+4.3+4.7.  

Of the top 5 teams in the AL, only 2 teams (40% of the sample) made the playoffs: Toronto, who made it in the wild card, and Boston, who won the AL East division. If the Beane Count is meant to rank teams based to determine success in a season (i.e., wins), why then does it break down in the AL?

Well, as it turns out, it’s differential success patterns across leagues may just be by pure chance, but Neyer buys into a deadly inference when he calculates Beane Count, exception fallacy. Exception fallacy is when the statistician reaches a conclusion about a group based on exceptional cases. In context of the Beane Count, Neyer assumes that good pitchers give up less homers (probably true) and less walks (not necessarily true, as the top fivewalkers in MLB history are Hall of Famers and all-time strikeout leaders, seasonwin leaders, and earned run average leaders). In short, they probably just throw more pitches than other pitchers. Same goes for batters: do a ton of walks make your cadre of batters better and thus translate to more wins? No, and this can be proven by multiple losing teams usually being in the top walks taken in a season. Your team will get on base more, but that doesn’t necessary translate to runs plated. Toronto, the lofty rated Beane Count of the AL, had a mediocre 16^th-ranked offense when it came to runs scored with runners in scoring position. So although Toronto get on base a lot, Neyer’s stat predicts that because of this maybe Toronto should be the top of the AL this season. That’s an exception, and a victim of exception fallacy.

The use of exceptions to think up statistical meaning in baseball has run awry since sabermetrics went mainstream. It makes sense, as the whole point of sabermetrics is meant to expose untapped market, but Rob Neyer needs to be a bit more careful. His analysis of success is similar to trying to come up with a statistic to quantify the most beautiful color in the light spectrum: it’s subjective and pretty useless.