How much does home-field advantage vary among teams?

What is the "reliability," in the statistical sense, of home field advantage? That is, if you had the opportunity to play a season over again, what would the correlation be between teams' HFA in the first season and their HFA in the second season?

One way to investigate the question might be to check real-life leagues from one season to the next. The problem, of course, is that things change over the winter – players move around, parks get modified, and so on.

There is indeed a better way to get the answer to the question, which I learned from a new JQAS article by Marshall B. Jones. It's called "A Note on Team-Specific Home Advantage in the NBA."

Jones shows us a method dating back to 1950, called the Spearman-Brown prophecy formula. What you do is start by finding the correlation between half the season and the other half. To avoid timeline bias (a team might have changed personnel between the first and second halves of the season), you divide the season into odd and even games, and check the correlation that way.

Once you have that odd/even correlation, you calculate the full-season correlation like this:

full season r = 2(half season r) /(1 + half season r)

(That's a specific case of the formula, where you want to double the sample size. If you want to triple the sample size, you replace the "2" by "3" and the "1" by "2". In general, if your sample size multiplies by X, replace the 2 by X, and the 1 by (X-1).)

Jones defined HFA as home winning percentage minus road winning percentage. With that definition, he found the NBA half-season correlations turn out to be:

2002-03: +.243
2003-04: +.147
2004-05: +.323
2005-06: -.051

The formula doesn't work for negative correlations, but plugging those numbers in for the other three seasons gives full season reliabilities of:

2002-03: +.391
2003-04: +.256
2004-05: +.488

These look pretty high, but you have to keep in mind that the fourth one is negative. So, overall, it's not as strong a correlation as it looks. The top list, which has all four seasons, looks a bit bigger than zero, but not much.

Anyway, Jones talks about how researchers shouldn't be careful when using team HFA in research, because it doesn't meet the "80% reliability" rule of thumb for measurements. He suggests that reliability can be enhanced by grouping teams together, to create a larger sample (thus increasing the coefficient in the Spearman-Brown formula). But I don't think that's right. The formula assumes that the additional observations are from an identical sample; but the whole point of team HFA is that the Cavaliers *don't* have the same HFA as the Spurs.

I do agree with Jones that there's way too much noise in individual team HFAs to take them at face value. For one thing, there's a lot of random variation involved. The SD of HFA in a single season is 1.4 times as high as the SD of wins. For a .500 team, that's .156. That's an overestimate, since NBA results are farther from binomial than other sports (since the better team is much more likely to win in basketball than, say, baseball); but still, it gives you an idea that the SD of the luck is more than half as big as the actual number you're measuring.

Second, HFA is dependent on how good the team is. It's higher for teams near .500, and lower for better or worse teams. Two points to show why that's the case:

1. Teams that never win have an HFA of zero. Teams that never lose have an HFA of zero. This suggests, intuitively, that HFA is highest in the middle.

2. If you consider that HFA is worth a certain number of points – say, 3 – then it only makes a difference when the home team would otherwise have lost by 3 points or less. That happens most often when the home team is just slightly worse than the visiting team, which is a point near .500.

The point of all this is that a substantial portion of the observed correlations aren't due to anything other than the team's overall skill. Good and bad teams will have lower HFAs in both odd and even games, and average teams will have higher HFAs in both odd and even games. That effect could be causing the entire correlation.

Has anyone seen any study that proves that a team has a specifically better home field advantage because of characteristics other than its overall skill level? I think I remember Bill James writing about how the Red Sox seemed to do better at home by tailoring the team to the park, but I don't remember a formal study.

In any case, I wouldn't be surprised if HFAs (after adjusting for overall talent) turned out to show very, very little difference between teams.