Why does free throw percentage fluctuate so much?

Typically, we assume that a player's talent doesn't vary much from year to year, except perhaps because of injury and aging. When an established hitter drops, say, from .320 to .280, while in the prime of his career, we usually assume most of it is luck. It might be just randomness of outcome, perhaps in combination with random circumstances, like differences the opposition he faced. On occasion, we might consider the possibility that the player changed his approach or technique.

If you were to choose a skill that doesn't depend much on circumstances or technique, it might be NBA free throws. Every shot is from exactly the same place, and it's something the player has been doing since childhood. You wouldn't expect a player's foul shooting talent to jump around a lot.

But ... it seems that it does.

I took the 50 NBA shooters with the most attempts in 2004-2005, and checked how they did the next season, 2005-2006. I expected they'd be roughly the same. Specifically, if I expected that if I converted each difference to a Z-score, they'd form a bell curve with mean 0 and SD about 1. I say "about" 1 rather than exactly 1, because there are probably slight changes due to injuries, aging, etc. There's also a selective sampling issue, since I chose the players with the most attempts. (E-mail me for the spreadsheet -- I had to do it manually.)

But, the SD of the difference was much more than 1.00. It was 1.87.

The biggest changes were almost all declines. Desmond Mason went from .802 to .682 (4.8 SD). Tyson Chandler dropped from .673 to .503 (4.7 SD). Drew Gooden, Stephon Marbury, Jalen Rose, Mehmet Okur, and Pau Gasol also had big declines, 2.5 SD or more.

Only Yao Ming (.783 to .853, 2.7 SD) and Mike Bibby (2.5 SD) had improvements above 2.5.

Looking at these players' careers, you see that there's definitely some luck involved: many of them went from "too high" relative to the rest of their careers, to "too low". A couple of them changed (roughly) permanently -- Yao, for instance.

The scale for free throws isn't that much different than the scale for batting averages. For Mason, imagine a player who hits .302 in 420 AB one season, and then hits .182 in 524 AB the next. That can happen, but it doesn't happen that often (even taking into account that MLB players are unlikely to get 524 AB while hitting .182). Furthermore, whatever reasons you can think of that it would happen in MLB -- injury, conditioning, age -- you'd think wouldn't really apply to foul shooting.

None of the changes are that extreme, taken alone: the issue is that there are *too many of them* for it just to be binomial randomness. A Z-score of 3.0 or more should happen only one in 300 times, by chance. In this sample, it happened 4 times out of 50.

Let me make one more adjustment, for a "season effect". It turns out the overall league average FT% dropped by .011 between those two seasons. I don't know if that was cause or effect -- but, in any case, if I adjust every player by that .011, the scale is now balanced between improvers and decliners. But the overall SD stays the same (actually, it increases slightly, from 1.87 to 1.88). Also, the three biggest changes are still all decliners, and those three (Mason, Chandler, and Gooden) are at 4.4, 4.4, and 3.7 SD, respectively.

So, what's going on? Do players' free throw talents fluctuate that much? If so, what does that say about more complicated talents?

---

P.S. This post was inspired by a discussion about correlation at Sport Skeptic, here.