The Hamermesh umpire/race study revisited -- part I (addendum)

In the previous post, I ran some regressions on the Hamermesh data using a 1/100 sample. But thanks to a couple of commenters who suggested "gretl" regression software, I am now able to regress on the full dataset (a bit over a million pitches).

I thought the full data would give almost the same results as the smaller dataset -- after all, I cut down the data in exact proportion. But the rounding errors proved to be more important than I thought. When I divided by 100 and rounded to nearest strike, that was a maximum of an 0.5 strike error in the new, smaller sample, which is the equivalent of a maximum 50 strike error in the large sample. That's a fairly large difference, considering that we're dealing in hundredths of percentage points.

So I'm going to rerun the results for the larger sample, here, just to be as consistent as possible with the data in the study. If you're not interested in the details, I'll tell you the conclusions right now so you can skip the rest of this post. The full regression gives:

-- slightly different numbers;
-- slightly less evidence of same race bias;
-- and pretty much the same overall conclusions.

For those who want to see the updated regressions, keep reading.

-----

Here's the "expected" strikes matrix for the regression that did NOT include a variable for racial bias:

Pitcher ------ White Hspnc Black
--------------------------------
White Umpire-- 32.06 31.46 30.64
Hspnc Umpire-- 32.03 31.43 30.61
Black Umpire-- 31.83 31.23 30.41

Subtracting that from the real-life observations in the original Table 2 matrix :

Pitcher ------ White Hspnc Black
--------------------------------
White Umpire-- +0.00 +0.01 –0.03
Hspnc Umpire-- -0.12 +0.37 +0.18
Black Umpire-- +0.10 –0.36 +0.35

Converting that to pitches:

Pitcher ------ Wht Hsp Blk
--------------------------
White Umpire-- -15 +23 -08
Hspnc Umpire-- -29 +27 +01
Black Umpire-- +45 -50 +06

The results are a bit different than in the 1/100 sample. For instance, white umpires are even less biased in favor of their own race, by negative 15 pitches here vs. negative 4 pitches in the other regression. And the same-race cells add up to only +22, as compared to the +37 from before, which is also less suggestive of same-race bias.

-----

Now, here's the regression that includes the UPM variable for same-race bias:

Chance of a called strike equals:

30.4448%
---- plus .1906% if the ump is white
---- plus .1827% if the ump is hispanic
---- plus 1.377% if the pitcher is white
---- plus .8206% if the pitcher is hispanic
---- plus .0513% UPM (if the umpire matches the pitcher)

The old regression had the UPM term at .1169 – this one has it at less than half that, at .0513.

Now that we're using the full dataset, we can get a signficance level for the UPM parameter. It turns out it's not significant at all, with a p-value of about .83, far more than the .05 required for significance. In fact, the real-life data show *less* racial bias than if the data were random (which would give a p-value of 0.5).

Doing the calculation for baseball significance shows that the proportion of pitches affected by the presence of a same-race pair is somewhere between 1 pitch in 2855, and 1 pitch in 6140.

-----

It's interesting how a such small change in the observed percentages – caused just by rounding! – could bring on such a large difference in the estimate of racial bias. In part, that's because this regression is trying to reproduce the numbers in the nine cells, regardless of whether those cells contain 800 pitches or 700,000 pitches. While the number of observations certainly does affect the results of the regression, it seems that the raw numbers in the cells matter even more.

So the conclusions in the previous post still stand – no evidence of bias, no statistical significance, and no baseball significance either.

And, again, we haven't actually got to the Hamermesh study itself yet. We'll do that next.