Which NBA talent is less stable: free throws or field goals?
Last post, I wondered about why NBA players' free-throw percentage seems to fluctuate so much. If you start with the hypothesis that changes are all just luck, you'd expect only one player in 50 to be more than 2.5 SD from zero. But, from 2004-05 to 2005-06, there were 10 such players. So, clearly, it can't be just luck. Talent must be changing too.
Since then, commenter "doc" was kind enough to e-mail me similar numbers for field goal attempts. My hypothesis was that the talent change for FG should be bigger than the talent change for FT, because FG requires a bigger set of skills.
The results surprised me.
Let me take you through the math of how I did this, just so you can make sure I got it right. If you don't care, skip to the bold parts and the last section.
Start, for instance, with Allen Iverson. In
2004-05 2005-06, Iverson shot .447 in 1,822 FG attempts. The binomial SD of that is the square root of (.447 * (1-.447) / 1822), which works out to .01165.
2005-06 2004-05, Iverson was .424 in 1,818 attempts. That's an SD of .01159.
The SD of the *difference* between the two seasons is the square root of (.01165 squared + .01159 squared), which works out to .0164.
Iverson's actual difference between the two seasons was .023. Divide .023 by .0164, and you get 1.400. So, Iverson's Z-score for the difference is 1.4.
Repeating this for all players in Doc's sample, we wind up with 120 separate Z-scores. If the differences were all luck, we'd expect that if we took the standard deviation of those 120 numbers, we'd get exactly 1.00. Instead, we get an SD of 2.18.
So, FG shooters change year-to-year by an SD of 2.18 "luck units". Since we know 1.00 of those units are actually luck, that leaves 1.94 units of talent change (since 2.18 squared minus 1.00 squared equals 1.94 squared).
What does 1.94 units mean in real life? Well, the typical player in Doc's sample went around 45% in 930 attempts. So, the SD from luck would have been .0231 (or 2.31 percentage points). Multiply that by 1.94, and you get .045. So:
-- An NBA player's FG% talent changes from year-to-year with an SD of 4.5 percentage points.
Now, let's do the same for FT%, so we can compare.
FT shooters change year-to-year by an SD of 1.87 "luck units". That leaves 1.59 units of talent change.
The typical foul shooter went 77% in 471 attempts. So, the SD from luck would have been .0274. Multiply that by 1.59, and you get .043. So:
-- An NBA player's FT% talent changes from year-to-year with an SD of 4.3 percentage points.
They're almost exactly the same!
Part of the reason I wouldn't have expected that is that for FG attempts, what we're calling "talent" isn't really just talent. It's "everything except binomial luck." So, it also includes changes in quality of opposition, quality of teammates, role on the team, ratio of 2-point and 3-point tries, and so on -- actually, quality of shot attempts. Since we're really measuring the sum of two variances -- talent, and shot quality -- we'd expect it to be higher than just for FT attempts.
Another reason I expected FG to be higher is that there might be some selective sampling involved in the FG case: a player who has a really bad year (perhaps by luck) might not play again next year. That would remove a bunch of outliers. But, the average player in the sample actually declined the second year, by 0.7 percentage points, so it doesn't look like that's it.
On the other hand, part of the reason could be that FG percentages are lower than FT percentages. For FG, we have 4.5 percentage points out of 45. For FT, we have 4.3 percentage points out of 77. So, looking at it that way, the talent change for FG is 10% either way, but for FT, it's less than 6% either way.
What do you guys think?
P.S. I ran the same numbers for batting average in MLB. I'll save that for a future post.
UPDATE: commenter bsball points out that FG% includes both 2-point and 3-point attempts. So, that's another way shot quality is affected.
Indeed, this might be a big effect. Iverson took 338 three-point tries in 2004-05, but only 223 of them in 2005-06.
I've updated the post. Also, I corrected where I had inadvertently reversed the two seasons.