Saturday, November 07, 2009

Do younger umpires call a more accurate strike zone?

In a post on the economic incentives facing would-be umpires, J.C. Bradbury has an interesting study on how older umpires are more likely to have a larger or smaller strike zone.

Bradbury ran a regression, to predict an umpire's season strikeout-to-walk ratio based on his age and who he is. He found that the older the ump, the more different his strike zone size. He writes,

"It turns out that every year an umpire ages he increases his deviation from the league average by about 0.8% [.16 change in K/BB ratio]. That doesn’t seem like a lot—and it really isn’t a huge effect—but over a period of 12 years that pushes the umpire a full standard deviation (9.5%) [.19] above/below the average deviation. Thus, by the end of an umpire’s career, his calls are about two standard deviations from the typical deviation. This is evidence of a tenure effect or a loss of competency." [square brackets mine.]

How big is that effect? Well, 0.19 might be the difference between a K/BB ratio of 2 and a ratio of 2.19. Assuming the same number of total K+BB, that means that instead of (say) 20 strikeouts for every 10 walks, there would be 20.6 strikeouts for every 9.4 walks. That turns 0.6 walks into strikeouts per 30 K+BB events, which is about 0.35 runs.

In 2009, there were 10.33 such events per game (per team), not 30. That means a standard deviation is worth a bit over a third of .35, or .12 runs per game. So a pitcher's ERA might go up or down by .11. For two standard deviations, it would be .24 runs or .22 earned runs. Assuming both teams are equal, there would obviously be no effect on who wins the game (although it seems likely that pitchers may be affected differently).

What's that in terms of pitches? A study I did (.pdf, page 4) finds that the difference between a ball and a strike is about 0.14 runs. So, at 2 standard deviations, an umpire calls about 1.7 pitches differently per game, per team. That means that more than 95% of umpires are less than 1.7 pitches per game different from the mean.

But since the pitchers and batters are presumably aware of the differences between umpires, they would adjust accordingly. So the effect might be more than .12 runs per SD -- it could be .12 for the actual strikeouts and walks observed, but it might cost the batter another .12 (or some other number) in having to swing at bad pitchers (which would be called balls by another umpire).

And, of course, even umpires who call normal numbers of strikeouts and walks might have their own particular strike zone -- it might be the same size, but have a different shape or location.

Bradbury concludes that, as umpires gain experience, they get more confident and less reverent, and feel less of a need to stick to the league's interpretation of where the strike zone should be. That sounds reasonable, especially considering that Bradbury's regression controlled for calendar year.

It's worth reading Bradbury's entire post, which includes a list of umpires and their individual K/BB ratios.


(Note: post was updated in response to an e-mail from Guy, who pointed out I had misinterpreted Bradbury's percentages and got the effect being half of what it should be. I think it's correct now.)

Labels: ,


At Monday, November 09, 2009 7:33:00 AM, Anonymous Guy said...

Very interesting post by JC. His point that long-term job security can contribute to the quality of the workforce by attracting more talented people is important but rarely noted -- it's much easier to see the ways in which job security can be anti-meritocratic.

However, I'm not sold on his finding that increasing divergence from the league mean with age must signal "shirking" and "incompetence." There may be a substantial cohort effect that looks like an age effect in his regression. That is, an umpire learns a particular strikezone when he is young, and then tends to stick with it as the average zone in the league changes. And the zone has changed a lot: K/BB rose steadily from 1954 to 1968, dropped suddenly to a low level in the 70s, then resumed a steady upward climb from the early 80s to today. Not all of that change reflect changes in the strikezone, but a lot does. So an umpire who sticks close to the zone he learned as a young umpire will tend to see his deviation from the league mean grow (except those who actually started out extremely high, who will appear to improve if they don't change a thing).

JC provides some evidence this is true. He shows the career K/BB rate for umpires from 1998-2008. The top 7 (highest K/BB) started umpiring in 1992 on average. The bottom 7, however, have an average starting year of 1984 -- 8 years earlier. That's a big spread, and suggests umps who first learned a smaller strikezone tend to continue that zone.

This could just mean it's very hard to learn a new strikezone (especially since some changes haven't been official league policy, so presumably no training in the de facto new zone was ever provided). It may have nothing to do with older umps feeling free to flout league policy.

I hope JC follows this up and provides some more info on the umps.
* He should distinguish between any cohort effect and an aging effect.
* I also wondered if there was a general tendency for umps to narrow or expand their zone over a career.
* It would be interesting to see if pitchers' umps and hitters' umps have different HR rates and/or BABIP, as a result of hitters getting better/worse counts.
* Regression aside, it would be nice to see a simple table showing the average deviation from league mean by years of service (1-6, 7-12...).

At Monday, November 09, 2009 7:18:00 PM, Blogger Phil Birnbaum said...

Excellent points from Guy -- wish I had thought of them before posting. As you suggest, it could simply be that all umpires stay the same as when they break into the league, but the standards change from under them.

For instance, find the 85th percentile "liberal" on race issues in 1957 Montgomery, Alabama. Even if that person hasn't change his views from then to now, he'd today be one of the most racially "conservative" people around.

At Wednesday, November 18, 2009 9:33:00 PM, Anonymous Anonymous said...

three words: age. period. cohort.


Post a Comment

<< Home