Friday, March 02, 2007

Home field advantage in speed skating

Most studies of home field advantage (HFA) deal with team sports. This has both benefits and drawbacks. On the one hand, team sports normally involve teams with schedules balanced between home and road games, which makes it easy to measure the effect. On the other hand, the effect can only be measured by the outcome of games, which makes it difficult to find where the effect is coming from.

In a study called Home advantage in speed skating," econometrician Ruud H. Koning tries to isolate HFA for individual speed skaters. To do that, he directly measures the skaters' performance -- that is, their speed.

Unfortunately, though, you can't just measure speed directly -- you have to make all sorts of adjustments. For one thing, there are "park" adjustments, since skaters post faster times in indoor tracks than outdoor tracks. They perform differently at high altitudes than at low altitudes. Skaters as a group perform better over time, dropping a huge 7% off their times between 1987 and 2002. (Some of the improvement is caused by changes in equipment, such as something called a klap skate, which was introduced in 1996.) Average speeds are different for different lengths of race. Competitors, for some reason, perform better at major events such as the Olympics.

And so, Koning had to adjust all the observed speeds, by running a regression of log(speed) on all the above factors. His estimates of HFA were:

Men are 0.2% faster at home than on the road;
Women are 0.3% faster at home than on the road.

I hoped to compare these numbers to the HFA in team sports, like baseball. To do that, we'd want to know the answer to the question "if two exactly equal skaters faced each other, how often would the home skater beat the visiting skater?" Alas, the study doesn't give us enough information to tell. If we knew the standard deviation of results for an individual skater, that would be enough. But we don't have that. (For instance, suppose the SD was 0.14%. Then, the SD of the difference between two equal skaters would be root-2 times that, or 0.2%. The observed HFA would then be exactly 1 SD. The chance of a normal variable exceeding 1 SD is about 1/3, and so the road skater would be about .333 against the home skater.)

What Koning does tell us is that the average difference between Olympic gold and silver in 2002 was 0.77% for men and 0.45% for women. In baseball, the 162-game difference between home and road is about 13 games, while, typically, the best team in MLB beats the second best team by only a game or two.

So in skating, the distance between first and second is three times the HFA. In baseball, the difference between first and second is maybe 1/5 the HFA. This suggests that HFA has a much, much smaller effect in skating than in baseball.

I would have expected HFA in skating to be much higher. For one thing, there's a lot of luck in baseball -- the inferior team wins a reasonably large proportion of games. But in speed sports like skating, you'd think there's a lot less luck, and individual times would be pretty consistent. In that case, you'd think that any advantage in ability would translate fairly directly into advantage in observed performance.

Put another way: home teams play .540 baseball. But if you compared only strikeout rates, I bet home teams would play higher than .540. And if you compared ball/strike ratios, home teams would be better still. Speed skating seems more like a pure skill, more like throwing strikes than scoring more runs. And so you'd expect a larger effect.

Second, it seems like the regression result is so small that changes in the method might significantly affect it. For instance, the Olympic Games effect is four times the size of the HFA. If there are other confounding factors that weren't included, those could significantly change the size of the observed HFA.

In fairness, I don't know if there are any of those, and Koning seems to know a lot about skating and has added a fair number of variables based on his understanding. (And I think this study illustrates the importance of knowing about the sport you're analyzing -- researchers with little knowledge of skating, such as myself, might have remained ignorant of the sudden introduction of the klap skate, and used a linear variable for year instead of separate dummies to capture the abrupt change.)

But still: why is the Olympic effect is so large? Koning makes the argument that the Olympics feature competitors who are at the top of their game. But wouldn't they also be at the top of their game in the years immediately before and after the Olympics? That's an important variable that isn't included -- stage in a skater's career. Would adding that variable change the conclusions?

Or, less importantly, what if the klap skate improved short distance skating more than long distance skating? If you added interaction variables for distance and year, how much would home advantage change then? Probably not much at all, but maybe a lot.

But perhaps I'm nitpicking. Still, given that the effect is so small, and there are so many variables to adjust for, to my mind this study constitutes fairly weak evidence of HFA in skating.

Labels: ,

At Tuesday, March 06, 2007 1:37:00 PM,  Anonymous said...

Interesting article. Thanks for pointing it out Phil. I hope there are more studies to come on hfa in individual sports. Koning does an admirable job at laying out a difficult regression problem to isolate effects. A couple comments on your comments, Phil:

1. I've always had a problem with the phrase "statistically significant, but small." To me, if it is at least a few standard dev away from 0, then it is in some sense big.

2. I agree that the olympic and world championships dummies seem artificially high. I would guess this is due to selection bias (only the best in the world qualify, and by nature raise the avg speed). Note that olympic and world championship dummies are almost the same - leading me to conclude that they let anyone skate in that world cup tour thing... This bias leads us to falsely believe hfa is only 1/5 the size of the olympic dummy.

3. I think there could be an interesting debate on how to compare hfa across sports. As Phil points out: which is bigger, 0.2% speed in skating or 8% win in baseball (home-away)? Phil suggests that since luck is involved more in baseball, the hfa effect should be diluted, so that the 8% win factor in baseball is even that much more impressive when compared against a "pure skill" event like skating. I'm not sure I agree. What is confusing the situation for me is how different sports can leverage differentials in skill into probabilities of winning.

Let me give a (contrived) example: How much better is Marvin Harrison as a wide receiver than a high school kid? It depends on the measurement of course. If you run them both in the 40, I bet Marvin's time is maybe 8-10% faster? If you throw them both a bunch of tough passes, Marvin catches 10% more? But if you have them both run routes against NFL corners in game situations, Marvin probably beats the corner for a completion 95% more often than the kid. All this to say that the leverage Marvin imposes with his 10% better raw skills lead to a greater than 10% final performance.

I guess all I am saying is hfa in a raw skill could have multiplicative impacts on win % depending on how much a sport processes that skill into something else (ball in basket, ball in goal, runner touching 4 different bases in order with a bunch of rules attached...etc) notwithstanding the mitigating effects of luck.

At Wednesday, March 07, 2007 9:40:00 AM,  Phil Birnbaum said...

Hi, Nate,

1. To me, statistical significance is a measure of whether the effect is *real*, not whether the effect is *big*.

2. Didn't the regression control for the ability of the skater? If that's the case, you'd expect no effect for the Olympics: why should skater X perform better in the Olympics than in other meets? The author suggests skaters put out more effort, if I recall correctly, which seems reasonable.

This is from memory, I should probably reread the study.

3. You're right, I never thought of that.

To extend your hypothetical leverage example, suppose Marvin Harrison makes 50% of catches in a neutral site. At home, he's 0.2% faster and his defender is 0.2% slower. This lets him gain half a step on the defender, and he leverages the difference to make 70% of those catches instead of 50%.

Absolutely right, I stand corrected. Thanks.

At Thursday, March 08, 2007 9:31:00 AM,  Anonymous said...

1. fair enough

2. you are right. results are already controlled for individual ability. my guess now is that olympics and championships are 1-time events that a skater can "taper" for and peak while the cup series is ongoing

3. i wonder if the majority of variability in hfa between sports depends on the degree to which that sport processes raw skills into final results... hard to come up with a measure of "processing" other than subjectively

At Thursday, March 08, 2007 9:43:00 AM,  Phil Birnbaum said...

I think that HFA in a sport is highly correlated to the probability that, in general, a better team will beat a worse team.

Tangotiger explained this well somewhere by citing the number of "confrontations" between the opponents. The more confrontations, the likelier luck will even out and the better competitor will prevail. If you assume the HFA makes the home team better and the visiting team worse, there you have it.

To go back to skating, maybe what's happening in there is that the HFA of 0.2% is, indeed large -- but the 0.77% difference between the first- and second-place skaters is even larger.

In team sports, with explicit or implicit salary caps, no team can dominate another -- it's hard to put together a team that's 100% superstars. But in individual sports, a player *can* dominate -- by definition, Tiger Woods and Roger Federer are 100% superstar.

So the difference between best and second-best is going to be big, and not a good comparison to HFA.

I'd still like to know the SD of individual performances; that would let us confirm whether 0.2% is a large HFA.