Racial bias and NBA referees -- a follow-up study

In comments to one of my posts on the Hamermesh study, commenter Guy mentioned a study on a similar subject, NBA referees and race. I hadn't seen that paper before.

The article is called "Racial Bias in the NBA: Implications in Betting Markets." It's a follow-up to last year's famous Joe Price/Justin Wolfers study (which I reviewed back then in three parts). This one, now also co-authored by Tim Larsen (so I'll refer to the authors as "LPW"), examines the impact of race on NBA betting markets -- that is, the Vegas line.

The paper comes to similar conclusions to last year's study – that referees appear to be either biased in favor of players of their own race, or biased against players of a different race (or both – it's impossible to tell which). Also, it suggests a profitable betting strategy to take advantage of this bias.

However, I disagree that the study has found referee bias. The way I see it, there is only one finding in this paper that suggests such a bias, and it's significant only at the 10% level. I'll get to that finding in a bit, but I'm going to start by listing some of the paper's findings that do NOT have to do with racial bias. Some of these, actually, are very interesting findings, and I don't think I've seen them before.

------

1. Black referees favor the home team more than white referees.

In Panel C of Table 2 of the paper, the authors do a regression on the home team's winning margin, against the relative racial composition of the two teams. On average, the home team's winning margin is:

3.167 when there are three white referees
3.541 when there are two white referees [and one black]
3.631 when there is one white referee [and two black]
4.591 when there are no white referees [so three black].

The differences are not statistically significant, but it is interesting that each one is higher than the last, and that the extremes are significant in a basketball sense (1.4 points difference is a lot).


2. White players, on average, are better than black players.

Again, in the same table, the authors show the point differential between when one team has 100% black players, and the other team has 100% white players. Actually, this is extrapolated from real life – a more realistic explanation is that the study found what happened when one team has (say) one "extra" white player, and multiplied it by 5. In any case, here are the results, expressed in more points for the all-white team (relative to the all-black team):

6.128 more points when all three referees are white
5.140 more points when two referees are white
3.673 more points when one referee is white
0.903 more points when no referees are white.

Obviously, "whiter" teams, overall, are better than "blacker" teams. A 100% white team seems to be about 5 points better than a 100% black team, so the difference is about 1 point per "extra" white player.

Also, there's some evidence here of the racial bias the authors are searching for. It does look like white refs are easier on white teams (and vice-versa) This is indeed suggestive. However, none of the differences are statistically significant. The standard errors of the four estimates above are 1.1, 0.9, 1.3, and 3.2 points, respectively.


3. The Vegas line underestimates white teams' chances of winning.

We know that from Panel D of Table 2, where LPW show how the point spread varies by the race of referees. When one team has 100% white players, and the other has 100% black players, the spread in favor of the white team is:

2.976 points when all three referees are white
3.593 points when two referees are white
2.413 points when one referee is white
2.144 points when no referees are white.

Compare these spread numbers to the actual numbers above: except for the no-white-refs case (which is only 3% of games), the spread underestimates how much the white teams will win by. In real life, the difference was about 5 points. In the spread, it's about 3 points.

This appears to contradict the hypothesis that betting markets are efficient – they appear to be a couple of points off in these cases.

Since the Vegas line doesn't try to pick the correct score, but, rather, tries to pick the spread at which the betting on both sides will be even, the most likely explanation here is that bettors, as a group, are slightly biased in favor of teams with more black players. That could be because black players are more likely to be superstars (and have more fans who bet on their teams). It could be that cities with more basketball bettors happen to have more black players. It is also possible that basketball fans are biased against white players (although I prefer not to accuse anyone of racial bias, even bias on who to be a fan of, except when all other explanations have been considered and found wanting).

------

Okay, those are three statistical effects the study found that have nothing to do with racial bias among refs. I'll repeat them:

1. Black refs tend to slightly favor the home team.
2. White players tend to be slightly better than black players.
3. The betting line underrates whiter teams.

------

Now, let's go to the bias tests.

In Table 3 of the paper, LPW run a regression to predict various aspects of their sample of games. They correct for the race of the teams and the races of the referees. Then, after making those corrections, they look at what's left – specifically, what happens when the race of the refs matches the race of the players, and when it doesn't.

(Technical note: they define the "same-race" parameter as

% white refs * (% home black players - % visitor black players)

This reaches its minimum of -1.00 when (a) all the refs are white, (b) all the home players are white, and (c) all the road players are black. It's +1.00 when (b) and (c) are reversed.)

Here are the differences in this most extreme case (all white team, all black team, all white refs):

A 16% additional chance of beating the spread
An extra 3.3 points relative to the spread
An extra 4.1 points relative to the other team

(The difference between the 3.3 and 4.1 comes from the spread itself being 0.8 points lower.)

These seem like big differences. However, none of these numbers are highly significant. All three are a little less than 2 standard deviations away from zero, so they're only significant at the 10% level.

In terms of basketball, are they significant? Well, in the extreme case, yes. But in real life, you're not going to have these extreme conditions very often, if at all. No team is 100% white, and no team is 100% black. More typically, the home team might have one extra white player (out of 5, adjusted for expected playing time), and there might be two white refs on average. That would give a parameter of (66% * 20%), which is about .13. So the effects would be 13% of the ones above That means the extra chance of beating the spread would be closer to one percentage point, not 16.

Still, that's something, even though (as I said) statistically significant only at the 10% level.

------

Now, for Table 4, the authors suggest a betting strategy: bet on the team with the greatest racial similarity to the refereeing crew. If there are more white refs than blacks, bet on the whiter team. Otherwise, bet on the blacker team.

Here's the percentage of bets you'll win:

3 white refs: 51.37%
2 white refs: 50.99%
1 white refs: 50.39%
0 white refs: 52.53%

In all cases, you win more than half your bets!

However: remember finding number 3: the betting line underrates whiter teams. The "3 white refs" and "2 white refs" cases involve betting on the whiter team. So it's likely that what's happening here is not referee bias, but Vegas line inefficiency! Betting on the whiter team is generally a better than 50-50 shot.

The other two cases are not as easily explained – now, you're betting on the blacker team, and those should be slightly below-average bets. But they're not – they're better than even wagers. What's going on?

Probably statisical insignificance. The "0 white refs" sample is very small, and the 52.53% figure is only one standard error from 50%. As for the "1 white ref" sample, there's an interesting result: the more black players you're betting on, the lower your chance of winning the bet. If the blacker team is blacker by more than half a player (out of 5), your winning percentage is less than 50%. It's only when one team is *slightly* blacker – less than half a player – that the bet is a winning one. This is certainly not consistent with the idea that it's racial bias causing the effect.

In the 3-white-refs and 2-white-refs cases, the odds of winning go up the more white players there are on the team you're betting. Since this is consistent with white players being better, you can't really tell if there's referee bias happening there too.

------

Finally, we come to Table 5, where the authors consider various betting strategies.

They propose a simple rule: wait for an all-black or all-white refereeing crew. Then, bet on the team whose racial composition better matches the referees. Doing this, they find you win 51.48% of bets. This is significant at the 5% level (a tiny bit over 2 SD).

But, by this rule, one team might be only very, very slightly blacker (or whiter) than the other. Shouldn't you improve your odds if you also wait for a large discrepancy in team race? It turns out that you do. If you don't bet until you find a race difference is over 10% (one half a player), you win 51.82% of your bets.

If you wait for a 20% advantage (one player), you move up to 54.34%. If you wait for a 30% advantage, it's 56.30%. And if you wait for a 40% advantage, you'll only bet 160 times in 14 years. But you'll win 61.88% of those bets!

Here's all this in a table. Remember, these are games with three same-race refs only:

51.48% -- all games
51.82% -- 10% player race advantage
54.34% -- 20% player race advantage
56.30% -- 30% player race advantage
61.88% -- 40% player race advantage

The first three of these are statistically significant at 5%; the fourth one is significant at 1%. The last one is also significant at 1% (although the authors didn't mark it as such).

So does all this significance indicate racial bias? Nope.

Remember that there are many more white referees than black. Three white refs happens 28.1% of the time, but, because there are so few black referees, three black refs in one game happens only 3.0% of the time. So if you wait for three same-race refs, over 90% of the time, those three refs will be white.

And since you bet on teams that match the referees, that means that 90% of the time, you'll be betting on the whiter teams. And we saw, from number 3, that the Vegas line underrates white teams. So you're probably winning because your teams are whiter, not because of the referees.

To check for sure, you need to (for instance) find all the games, not just the ones with same-race refs, where the race difference is 30% or more. Then see what happens when you bet on the whiter team. I bet it would be not too far off from the 56.30% observed with only white refs. And I'd bet the difference between the two would not be statistically significant.

------

Bottom line: in this study, the only evidence of same-race bias came in Table 3. The difference one extra white player makes (assuming two white refs) is:

-- an extra .022 chance of beating the spread
-- an extra .54 points relative to the opponent

That has reasonable basketball significance – half a point. But even so, I hesitate to accuse anyone of racial bias on the basis of a significance level of only 10%.