The Hamermesh umpire/race study revisited -- part VIII

This is part 8 of a continuing series on the Hamermesh study on umpire racial bias. The previous parts can be found here.

------

One of the criticisms I had of the Hamermesh study is its implicit assumption that if there is same-race bias on the part of umpires, it must be the same for all three races. The model used in the paper assumes that if (for instance) white pitchers are 0.5 percentage points more likely to call a strike for a white pitcher, then hispanic umpires must also be 0.5 percentage points more likely to call a strike for a hispanic pitcher. I argued that racism usually goes more one way than the other, and it's usually the majority who's more biased against the minority.

But there's another implicit and incorrect, assumption – that the *individual umpires* themselves have identical bias. That obviously can't be right, can it? Grab any random group of white people, and some will be more racist than others. You might have some that are explicitly racist, and some that harbor racist biases but won't admit it. But, on the other hand, you'll have some people who favor affirmative action, and may even practice it in their daily lives. And you'll find others who strictly believe in color-blindness in all aspects of life, and oppose affirmative action and racism with equal fervor.

If that's how it works in everyday life, wouldn't it be reasonable to assume that's how it would work with umpires too? Even if some umpires would have an unconscious bias in favor of their own race, wouldn't there be others who didn't, and even some who were unconsciously biased the other way? It's not that hard to imagine a white umpire so concerned about racism (societal or baseballistic) that he bends over backwards to be fair to minority pitchers, with the effect that he becomes unconsciously biased in their favor.

In other words, the idea that *every* umpire is biased in favor of his own race is probably not very close to reality.

What if, say, only half of umpires were biased? Well, then, what we'd get is a bimodal distribution of strike calls. There's still random luck in strike calls, so we'd get the sum of two normal distributions: the race-biased umpires would be one normal distribution, and the non-biased umpires would be the other. Half the pitchers would form a bell curve around unbiased, and the other half would form a bell curve around biased. (If there was a substantial number of "affirmative-action-biased" umpires, that might be a third).

Now, I bet that the differences would probably be small enough that the sum of the two normals would still look bell-shaped if you drew a graph. However, you might expect more outliers, more umpires far from the mean. For instance, 2.5% of the biased umpires would be 2 SD from the biased mean – which means they'd be *more* than 2 SD from the overall mean. That is, the "half of umpires are biased" bell curve would be spread out more than you'd expect from the regular binomial distribution of balls and strikes.

To check, I looked at all the umpires on 0-0 counts, and how they judged hispanic pitchers relative to white pitchers. The results looked very normal. (The Anderson-Darling test, which checks for normality, gave a coefficient of 0.13, definitely not significant.)

Out of about 85 umpires with large enough sample sizes, there were two umpires more than 2 SDs in favor of hispanic pitchers (Welke at 2.2 SD, Dellinger at 2.0), and two umpires more than 2 SDs in favor of white pitchers (Nauert at 2.2, Carlson at 2.5). That's almost exactly what you'd expect if nobody were biased. The alleged bias is on the order of 0.5 percentage points, and the random SD for an individual umpire is about 2.5 points. That ratio, I think, is big enough that if bias actually existed, we'd notice something funny in the data. But everything looks very close to normal.

What if it were *more* than half of umpires that were biased? Well, you'd still expect some variation in how biased they are. Some might be *really* biased, and some might be only slightly biased. Some might be biased against blacks but without it affecting their ability to call strikes. And some might have prejudices that actually benefit the minorities. (For instance, suppose an umpire harbors an unconscious assumption that blacks are intrinsically better athletes than whites. Might that prejudice lead him to call more strikes for black pitchers? I think it might.)

In that case, if the extent of the bias varied substantially between umpires, the distribution of strikes would be more spread out than we observed: again, we'd expect more than 5% of umpires to wind up outside the +/- 2 SD range. I'm not sure how big the effect would be; it would depend on the extent of the variance in bias. But, again, the study is talking about half a percentage point overall, which would require a significantly higher amount of bias at the extremes to counteract the mildly-biased umpires. I can't prove it offhand, but I think if you added any reasonably large (but varying) bias, you'd see something non-normal in the distribution of strikes.

So I think we can reject the idea that most of the umpires are biased, or even that half the umpires are biased. And logic suggested earlier that we reject the idea that *all* or (almost all) the umpires are biased. What does that leave us? Only the possibility that *a few* umpires are biased.

And if only a few are biased ... well, isn't that suggestive of the minority umpires? From Part 7, here are the pictures of where the minority umpires fit in relation to pitchers of their own race:

---X--------X----------------------------------

X--------X---------X-------------------X-------

The top line is the hispanic pitchers, the bottom line is the black pitchers. In both cases, umpires of the same race (marked with an "x") lean towards the left side of the line (favoring their own race).

If you had to get the x's balanced around the center of the lines, how do you do it? You could move a couple of x's to the right. Or, you could move a whole bunch of hyphens (mostly the white umpires) to the left.

But moving 10 or 20 hyphens would imply widespread racism (20-40 umpires, since each hyphen represents two). We rejected that idea. That means we have to move Xs, which implies that it's the minority umpires who are biased (if, in fact, bias exists at all).

Which is why I disagree with the authors' conclusions on page 24 of the paper:

"In particular, non-White pitchers are at a significant disadvantage relative to their White peers ... the fact that nearly 90 percent of the umpires are White implies that the measured productivity of non-White pitchers may be downward biased."

What they're saying is this: black umpires favor black pitchers, and white umpires favor white pitchers. But since there are so many more white umpires than black umpires, the white pitchers get favored more often, and get a better deal.

But, again, that can only happen if racial bias is widespread. If only a few umpires are biased, it's probably the minorities. It could well be that it's the minority pitchers who are the beneficiaries of any bias.

If, in fact, there is any bias at all.