Do umpires discriminate in favor of veterans?
At the SABR convention last month, some evidence was unveiled that suggests that umpires give more favorable ball/strike calls to veterans.
Pat Kilgo gave a presentation, "Do Umpires Give Favorable Treatment to Some Players?" He and his colleagues -- Hillary Superak, Lisa Elon, Mark Katz, Paul Weiss, Jeff Switchenko, Brian Schmotzer, and Lance Waller -- looked at called pitches from 2009-10. They compared the call to the PitchF/X data, and, from that, decided if it was a correct call, a "false strike," or a "false ball".
They then created a matrix classifying both the batter and pitcher by years of experience. There were 16 classifications for each, from "0-1 year experience" to "more than 15 years experience". So the matrix had 256 cells. Each cell contained the percentage of "false strikes" for that situation.
As it turned out, there were many, many more false strikes when the pitcher had a lot of experience but the batter did not. And there were many *fewer* false strikes when the situation was reversed, with an experienced batter and rookie-ish pitcher.
Pat was kind enough to give me permission to post his PowerPoint slides, which are here. If you turn to slide 16, Pat and his colleagues color coded the cells, from dark green (lots of false strikes) to beige (few false strikes). Most of the green are at the bottom-left; most of the beige are at the top-right. There is no doubt that the distribution of colors is statistically significant.
Here, take a look:
On slide 22, the authors repeat the analysis for "false balls". This time, the pitcher's experience is significant (veterans don't get cheated out of a strike very often), but the batter's is not.
To summarize the authors' slide 33:
-- Umpires absolutely favor veterans with respect to false strikes
-- Umpires most likely favor veteran pitchers with respect to false balls
-- No evidence of benefit to veteran hitters [on false balls]
There are a couple of possible criticisms to the study. One is that PitchF/X might not be the best way to classify missed calls (I believe Mike Fast made this argument, but I don't have the link handy).
Another -- and I think this was raised by a questioner at the original presentation -- is that not all pitches are created equal. If veteran pitchers tend to throw down the middle, instead of trying to paint the corners, that would reduce their number of false balls (since their strikes are more obvious). I suppose you could check that out by controlling for pitch location.
Still, it seems to me that there's a good chance that Pat and his colleagues have found a real effect. Part of the reason is that the "umpires favor veterans" theory doesn't come out of the blue -- a lot of observers have long believed it to be true. That's unlike the "umpires have racial bias" hypothesis, which was (and still is) generally doubted by players and sportswriters.
I look forward to hearing what everyone else thinks. Thanks again to Pat for permission to post and link.