Pocket worthyStories to fuel your mind

An Analysis of Nearly 4 Million Pitches Shows Just How Many Mistakes Umpires Make

Umpires don’t need to be replaced by robots, but some troubling findings indicate that they could use a little help.

The Conversation

Read when you’ve got time to spare.

file-20190405-180052-84ayxm.jpgcrop.jpg

Umpire Quinn Wolcott signals a strike out during an at-bat by Toronto Blue Jays’ Josh Thole. Credit: AP Photo / Ted S. Warren.

Baseball is back, and fans can anticipate another season of amazing catches, overpowering pitching, tape-measure home runs – and, yes, controversial calls that lead to blow-ups between umpires and players.

Home plate umpires are at the heart of baseball; every single pitch can require a judgment call. Yet ask any fan or player, and they’ll tell you that many of these calls are incorrect – errors that can affect strategy, statistics and even game outcomes.

Just how many mistakes are made?

Comprehensive umpire performance statistics are not readily known, tracked or made available. Major League Baseball doesn’t seem interested in sharing the historical data.

Could it be because the numbers aren’t flattering?

Luckily, every MLB pitch is tracked and made available – numbers then have to be accessed, downloaded, sorted and evaluated. This takes time and computing power. In a study with support from a team of Boston University graduate students, we closely analyzed how many balls get called strikes and vice versa. The accuracy of all home plate umpires was ranked and age and experience taken into account.

While the human element of the game certainly adds color, our results show that it comes at a high cost: far too many mistakes.

Mining the Data

All 30 Major League Baseball stadiums are outfitted with triangulated tracking cameras that follow baseballs from the pitcher’s hand until it crosses home plate. Ball location can be tracked up to 50 times during each pitch, and accuracy is said to have a margin of error of one inch. This information is used to evaluate players, but MLB doesn’t share the results in a way that allows fans to easily evaluate the performance of umpires.

We analyzed nearly 4 million pitches over the course of the last 11 regular seasons. This data, which had been collected by MLB-owned Statcast and Pitch f/x, was sorted, formatted and superimposed on a standard strike zone map.

file-20190405-180010-j8muom.jpg

An example of balls and strikes superimposed over a strike zone from a 2010 game between the Boston Red Sox and Toronto Blue Jays. The red points were called strikes, and the green points were called balls. Credit: Pitch F/X.

Using this available technology, we measured ball and strike calls for accuracy. We then ranked the error rates for each active umpire, creating a “Bad Call Ratio.” The higher the ratio, the worse the umpire.

The findings were troubling.

Botched calls and high error rates are rampant. Between 2008 and 2018, MLB home plate umpires made incorrect calls over 12 percent of the time. In the 2018 season, MLB umpires made 34,246 incorrect ball and strike calls for an average of 14 per game, or 1.6 per inning. In the 2018 season, 55 games – 2.2 percent of the total played – ended with an incorrect call.

When batters had two strikes, the error rate for all umpires increased – incorrect calls happen 29 percent of the time, almost double the error rate when the batter had one or no strikes.

We also found that the highest error rates did not come from younger, less experienced umpires; they came from the older, veteran umpires. The average MLB umpire is 46 years old, with 13 years of experience. But the top performers between 2008 and 2018 had an average age of 33 years old and had less than three years of experience at the big league level. Like professional baseball players, professional umpires seem to peak at a certain age.

Despite years of data-driven evidence, MLB has notoriously resisted retiring poorly performing umpires and hiring better-performing ones. The league remains top heavy with aging umpires, making it difficult for fresh new talent to make impact.

Umpires Can Still Play a Role

For all of the ways MLB has incorporated technology into the game – the radar gun, instant replay, pitch graphics, Doppler radar – the league has resisted deploying this technology to assist with calling balls and strikes.

Umpires continue to call balls and strikes like they did a century ago when Babe Ruth played.

I’m not proposing that baseball bring in robots and fire the umpires; baseball has too many one-off situations and complexities to assume a bot could replace an umpire. But MLB does have a unique opportunity to use existing technology and strengthen human-software collaboration so umpires can do a better job.

file-20190405-180020-1ecdo1j.jpg

Atlanta Braves outfielder Ender Inciarte argues with home plate umpire Doug Eddings after striking out against the Arizona Diamondbacks. Credit: AP Photo / Ralph Freso.

Umpires could easily be fitted with ear pieces connecting them to a control center that conveys real-time ball and strike information. These tech-assisted umpires could then make calls correctly, quickly and effortlessly. Time-honored and much beloved behind-the-plate signs, signals and sounds would still exist. And umpires could remain the final arbiter, having override ability under certain circumstances, such as if a ball hits the ground before crossing the plate or if a system outage occurs.

Strong recruiting, hiring and retention of superior performing umpires coupled with tech aids would reduce error rates and also help dampen biased pitch calling. Strike zone subjectivity would be minimized, allowing batters and pitchers to focus more on their craft and less on guessing a specific umpire’s strike zone quirks. It would also reduce conflict between teams and umpires. And imagine how much the player and fan experience would improve if more than 34,000 annual incorrect calls vanished.

Mark T. Williams is the James E. Freeman Lecturer in Management at Boston University.

How was it? Save stories you love and never lose them.


Logo for The Conversation

This post originally appeared on The Conversation and was published April 8, 2019. This article is republished here with permission.

The Conversation brings you insights and analysis from academic experts, straight to your inbox.

Get the daily newsletter