How stable is a baseball player's talent? Part III

(Note: this is a continuation of a technical post, probably not of general interest.)

Following the pattern of what I did last post for the odd/even split, here are the calculations for confidence intervals for the real season-to-season cases. Skip the math and head right to the bold parts, if you like.

-----

For all 38 pairs of seasons from 1973/4 to 2010/11, I calculated the SD of the binomial Z-scores for all pairs of players with at least 400 PA both seasons.

The average of all 38 SDs was 1.118. The SD of the 38 SDs was .0755. To get the standard error of the average, we divide .0755 by the square root of 38, which gives .0122.

I'm not sure if the SD is normally distributed, but, even if it's not, 2 SE in both directions is a reasonable confidence interval. So, adding/subtracting twice .0122 from the average of 1.118 gives us an interval of (1.094, 1.142).

(Last post, I said this method was imprecise. I think I was wrong when I said that. The method is imprecise only when the effect you're looking for is very close to zero (like odd/even). For larger effects, the method works much better.)

Those are the total SDs. They comprise the variance caused by binomial randomness, and also the variance caused by talent (and circumstance) changes.

If the actual SD is 1.094, that means the SD attributable to talent (actually, anything other than binomal variance) is the square root of (1.094 squared minus 1 squared), which is 0.444. If the actual SD is 1.142, the SD attributable to talent is .552.

So the confidence interval of the SD of talent change is (.444, .552), in terms of binomial Z-scores.

We now need to convert that to OBP. The average player in the study had 574 PA the first season, and 570 the second season. That gives a binomial single-season SD of the square root of (.333 times .667 divided by 570), which works out to .0197. For the difference between two seasons, multiply that by the square root of 2, giving .0279 points of OBP.

Now, we just multiply the Z-score confidence interval by .0279 to convert to an OBP confidence interval. And the result:

The confidence interval for SD of talent (and circumstance) change between seasons, for a single player, runs from .012 to .015.

I did the same thing for strikeout rate, walk rate, and extra-base hit rate (all per PA). Results:

Strikeout rate: (.0145, .0163)

Walk rate: (.0136, .0151)

Extra base hit rate: (.006, .008)