And old "icing the kicker" study

"Icing the kicker" is the tactic of calling a timeout just before the opposition is about to attempt a crucial field goal. The idea is that the interruption upsets the rhythm and mental balance of the kicker, who now as an extra couple of minutes in which to envision screwing it up.

Does the strategy work? In 2004, Scott Berry and Craig Wood decided to check. They published their article in Chance magazine, which I don't have – but Ivars Peterson summarized the study in an online column.

Barry and Wood looked at all "pressure kicks" (which would give the team a lead or tie with less than three minutes to go) in 2002 and 2003, normalizing them for weather, distance, and so forth. They found that icing the kicker did, in fact, work – reducing the chance of a successful kick from 76% down to 66%.

They say "the evidence is not overwhelming, but it is compelling."

The difference of 10 percentage points looks like a big effect, except that there were only 38 "icing" kicks overall. 29 of those kicks "should have" succeeded, but only 25 did. It's hard to believe that difference of only four kicks would be statistically significant.

Actually, the standard deviation for those 38 kicks is about 2.6 (the square root of 76% times (100 – 76)% times 38), so four kicks is only about 1.5 standard deviations away from expected. That's definitely not significant.

In fairness, the authors, who are statisticians, probably mentioned that in their article.

P.S. After writing this post, I found this, which points to more data from Stats, Inc. So I guess this is old news, but I wrote it, so I'm going to post it anyway.