Do umpires discriminate in favor of veterans?
At the SABR convention last month, some evidence was unveiled that suggests that umpires give more favorable ball/strike calls to veterans.
Pat Kilgo gave a presentation, "Do Umpires Give Favorable Treatment to Some Players?" He and his colleagues -- Hillary Superak, Lisa Elon, Mark Katz, Paul Weiss, Jeff Switchenko, Brian Schmotzer, and Lance Waller -- looked at called pitches from 2009-10. They compared the call to the PitchF/X data, and, from that, decided if it was a correct call, a "false strike," or a "false ball".
They then created a matrix classifying both the batter and pitcher by years of experience. There were 16 classifications for each, from "0-1 year experience" to "more than 15 years experience". So the matrix had 256 cells. Each cell contained the percentage of "false strikes" for that situation.
As it turned out, there were many, many more false strikes when the pitcher had a lot of experience but the batter did not. And there were many *fewer* false strikes when the situation was reversed, with an experienced batter and rookie-ish pitcher.
Pat was kind enough to give me permission to post his PowerPoint slides, which are here. If you turn to slide 16, Pat and his colleagues color coded the cells, from dark green (lots of false strikes) to beige (few false strikes). Most of the green are at the bottom-left; most of the beige are at the top-right. There is no doubt that the distribution of colors is statistically significant.
Here, take a look:
On slide 22, the authors repeat the analysis for "false balls". This time, the pitcher's experience is significant (veterans don't get cheated out of a strike very often), but the batter's is not.
To summarize the authors' slide 33:
-- Umpires absolutely favor veterans with respect to false strikes
-- Umpires most likely favor veteran pitchers with respect to false balls
-- No evidence of benefit to veteran hitters [on false balls]
There are a couple of possible criticisms to the study. One is that PitchF/X might not be the best way to classify missed calls (I believe Mike Fast made this argument, but I don't have the link handy).
Another -- and I think this was raised by a questioner at the original presentation -- is that not all pitches are created equal. If veteran pitchers tend to throw down the middle, instead of trying to paint the corners, that would reduce their number of false balls (since their strikes are more obvious). I suppose you could check that out by controlling for pitch location.
Still, it seems to me that there's a good chance that Pat and his colleagues have found a real effect. Part of the reason is that the "umpires favor veterans" theory doesn't come out of the blue -- a lot of observers have long believed it to be true. That's unlike the "umpires have racial bias" hypothesis, which was (and still is) generally doubted by players and sportswriters.
I look forward to hearing what everyone else thinks. Thanks again to Pat for permission to post and link.
Labels: baseball, referee bias, umpires
posted by Phil Birnbaum @ 8/01/2011 11:05:00 PM
7 comments
7 Comments:
Hey Phil,
I actually find similar results in my models even controlling for location on both the batter and pitcher side with respect to age and experience
Excellent! That's one objection out of the way.
Couldn't I argue that a pitcher will be more likely to become a veteran if there is something about his pitches which make them *look* like strikes to an ump?
I'd like to see Millsy's data before I believe it. I mean, I believe he has done good work on this topic, but there is a big effect due to location by age, and I'd want to be convinced that the effect that remains is not just due to not controlling well enough for it.
Mike 1,
Good point. One way to look at that will be to watch guys over time that have many years of fx data once we have all of that.
Hey mike,
Understandable. Because I'm writing this as a dull academic paper I am hesitant to put a lot out threre before a working version is ready. However, I think it would be advantageous to have your feedback specifically on the project. I know I have made claims and confirmations here and there on this site, at BtB, prospectus and baseball analysts and providing the methods and model is certainly something I'll need to defend those claims
Very interesting.
This looks significant regarding the issue of false strikes. The fact that something about the batter is predictive seems to suggest that features of the pitches can't explain all the data.
I'm a little more worried about the issue of false balls. The problem is that I'm betting veteran pitches throw more strikes (that's how they stuck around to become veterans.) But then you can wonder whether the false ball bias allegedly toward veterans is just a bias toward strike throwers, i.e., if you throw a lot of strikes, you're more likely to get the benefit of the doubt from the umpire. There are other possibilities as well; for example, some people speculate that a ball is more likely to be called when the pitch is not where the catcher sets up (regardless of whether it's in the zone.) Perhaps rookies false balls come from making the catcher move.
Philosofool,
Different types of batters definitely see different types/locations of pitches, and this does affect the size/location of their strike zones.
I haven't looked specifically at batters to see if/how pitch location varies by age. I know that it does for pitchers (see here: http://www.baseballprospectus.com/article.php?articleid=12965, end of section entitled "The Catcher Target Theory").
Post a Comment
<< Home