A "Canseco Effect" study

Did Jose Canseco influence his teammates to use steroids? Canseco loudly claimed that he did, and the claim seems plausible. But do the numbers back him up?

A couple of years ago, in an academic paper, two Israeli economists said they do. Eric D. Gould and Todd R. Kaplan ran regressions on individual player-seasons, and claimed that after playing with Canseco, players in general hit for more power. Furthermore, the result didn't hold for other players. For instance, after playing with Ken Griffey Jr., Junior's teammates' home runs actually fell, instead of rising.

The paper was recently reviewed -- uncritically -- in Slate. In response, J.C. Bradbury reposted his original critique of the study. JC's response is right on the money -- he shows that the results aren't very robust, and wonders why the authors used raw home run totals instead of rates (which is especially egregious considering that their minimum was only 50 AB). I suggest reading JC's post before continuing here.

---

While I'm waiting for you to get back from reading JC's analysis, I'll type the title of the study here, so Google can find it. It's "Learning Unethical Practices from a Co-worker: The Peer Effect of Jose Canseco."

---

OK, here's my two cents.

The study's most important finding is that, after Canseco left a team, his average ex-teammate improved by almost three home runs a year. That's a large improvement (and statistically significant at 4.5 SDs).

But, for the most part, the reason they hit more home runs is just that they got more playing time! In Table 7, the study shows that those players who hit 3 more HR also had 53 more AB. You might have expected those guys to hit, what, maybe 2 HR in the extra 53 AB, instead of 3? So the difference Canseco makes is ... one home run. Two HR due to playing time, but only one that could possibly be attributed to a change in performance level.

What caused that extra home run? It's a long stretch to insist it's due to steroid use, and it's almost as long a stretch to even say that it's something unique about Canseco. Because, look at the numbers for some of the other players in the same chart.

Ken Griffey looks like the anti-Canseco. After playing with Griffey, his ex-teammates wound up with 46 fewer AB than before, and their homers dropped by 3.4. But with 46 fewer AB, their expected drop should only have been about 2 HR, not 3.4. So it looks like Griffey reduced his former teammates' output by about 1.4 home runs.

If you want to believe that the extra 1 HR is due Canseco pushing his teammates to try steroids, does that mean that Griffey's negative 1.4 HR must have been because he pushed his teammates to *quit* steroids, and he did it 30% better than Canseco did?

Or, look at Ryne Sandberg's teammates. Those guys also lost playing time after their Sandberg teammateship, by 24 AB. But even with fewer at-bats, they *gained* 1.6 HRs! So Sandberg's effect on his teammates was more than two-and-a-half times as large as Canseco's. So, did Sandberg push PEDs 250% as hard as Canseco did?

So far, I think it's clear that these numbers don't necessarily have anything to do with steroids. Especially because there's a better explanation, at least for the changes in AB.

Ryne Sandberg played almost his entire career for the Cubs. If you want to be an ex-teammate of Sandberg's, and wind up in this steroid study, you have to wait. Either you have to be traded, or you have to wait until Sandberg retires. If you wait until Sandberg retires, you're going to be an OLD ex-teammate. And old players tend to get fewer at-bats than they did when they were younger.

On the other hand, Jose Canseco played for eight teams in his career. For the most part, if you wanted to be an ex-teammate of Canseco's, you just had to wait for him to move to another team. In the 1990s, that meant a maximum wait of two years, and an average wait of probably one year. That means it's very easy for you to wind up a young ex-teammate of Canseco's than a young ex-teammate of Sandberg's. And young players tend to get more at-bats as they get older and better.

Canseco's ex-teammates are younger than Sandburg's ex-teammates. That would explain whey they get more playing time, and that would also explain why they get better in subsequent seasons -- because they're still young enough to be improving, rather than declining. I can't prove it, but I bet that almost all the effect would disappear if the study properly controlled for age.

What this study actually demonstrates, I think, is that Jose Canseco switched teams a lot.


UPDATE: The authors have released a new version of the study, dated August 4. I haven't looked at it in depth, but one difference is that it now includes years up to 2009, instead of stopping at 2003.

One thing you'll notice in Table 7 is that 19 out of the 77 entries are significant at the 5% level. You'd have expected 4, not 19. Of course, they're not all independent, but still.

Also, Rafael Palmiero and Ken Griffey Jr. show as more extreme than Canseco, but in the other direction. That, I think, casts doubt on the hypothesis that there's something special about Canseco's numbers in this regard.

Did Jose Canseco teach his teammates to use steroids?

Here's a study that believes it has evidence that Jose Canseco influenced his teammates to take steroids, by finding that those teammates hit more home runs after Canseco left the team.

It's called "Learning Unethical Practices from a Co-worker: The Peer Effect of Jose Canseco," by Eric D. Gould and Todd R. Kaplan.

Here's what Gould and Kaplan did. They took all hitters since 1970 in the "power positions" (C, DH, OF and 1B) and examined how they did before playing with Canseco, while playing with Canseco, and after (but no longer) playing with Canseco. They controlled for a bunch of factors: year, manager's record, etc., and, most importantly, they controlled for the player himself, so that they are comparing parts of his career to other parts of his own career.

They found that there was indeed an effect. While playing with Canseco, the average player hit almost 1.1 home runs more than before Jose was on his team. And later, once separated from Canseco, the average player hit an additional 2.9 home runs! The 1.1 figure is almost statistically significant (about 1.7 SD), while the 2.9 is extremely significant (4.5 SD).

However, those numbers don't control for playing time. It turns out that while playing with Canseco, players had an average 16 extra AB than before. 1.1 HR in 16 AB is about 35 HR in a full season of 500 AB – more than average, even for a player in a power position, but really not that big a deal. And in the "after Canseco" years, the players had 53 extra AB in which to hit their 2.9 HR, which is 27 HR per season – probably still a little more than what you'd expect.

So far, it's a bit more power than you would have figured, but nothing too serious, and certainly not enough to persuasively show a "peer effect" of Canseco on his teammates.

Also, keep in mind that the study didn't control for the age of the players. Obviously, they'd be older in their "after Canseco" years than in their "before Canseco" years, and power generally increases with age. Taking that into account, wouldn't you now think that these numbers are pretty much as expected? Players get older, and, as they age, they get more playing time and hit for a little more power. I don't really see what the big deal is.

There is one caveat: the authors repeated their study for players other than Jose Canseco -- Rafael Palmeiro, Jason Giambi, Mark McGwire, Juan Gonzalez, Ivan Rodriguez, Dave Martinez, Ken Caminiti, Ken Griffey Jr, Ryne Sandberg, and Cecil Fielder. (The authors say they checked 30 players, but they only show the AB results for these 10.) It turned out that of all these players, Canseco showed the strongest effect in raw AB numbers. In home runs, the most comparable player was Ryne Sandberg. After playing with Sandberg, players had 1.6 more home runs per season than before Sandburg. (Recall that after Canseco, they showed 2.9.) After Canseco, players had 53 more AB. But after Sandberg, they had 24 *fewer* at-bats. That's a bigger increase in home run percentage on Sandberg's part. From these numbers, it would seem to me that Sandberg would be a better candidate as a bad steroid influence than Canseco.

In any case, the authors do acknowledge that the raw home run numbers are not very meaningful without also taking into account the changes in AB. So they ran another Canseco regression, this time including a variable for AB. Now, instead of 3 extra home runs, these players hit only 1 extra home run. That still comes out statistically significant, at 2.2 SD.

But again, this regression doesn't include player age. For players in the sample, the mean number of AB was 310. Would an increase of 1 HR per 300 AB not be consistent with normal patterns of aging for players in "power positions"? And that's not a 1 HR increase every year – it's the average increase between the several years the hitter played with Canseco, and the several years following. Again, it seems reasonable to me.

We'd have a better idea if the authors repeated this analysis for the other 10 hitters, controlling HR for AB there also. But they didn't, so we don't know if Canseco is a special case in this regard as well.

Finally, Gould and Kaplan dropped six players from the sample, the players that Canseco claimed to have personally injected with steroids. (Those were Palmeiro, Giambi, McGwire, Gonzalez, Rodriguez, and Martinez.) Without those six, the "after Canseco" increase in HRs dropped from 2.9 to 1.5. That's still statistically significant. But there's a corresponding increase in strikeouts (6.8) and walks (4.7), while batting average and slugging percentage barely change (.002 and .003 respectively). That suggests that, again, it's an increase in AB that's responsible for all these increases.

The question we're left with, then, is: why do Canseco's teammates increase their AB once he's gone? It could just be the situations, or the managers. Suppose Canseco tended to be signed by teams that were trying to win it all this year. Those teams wouldn't be playing a lot of rookies. But once they dropped back in the standings, they would tend to trade Canseco to a contender, and give their young players more at-bats.

Does that sound plausible? Canseco played on a lot more teams than most of the other ten players in the comparison set. That would suggest that he would have left a lot of teams in a rebuilding stage, wouldn't it?

Or maybe the types of managers who pushed for Canseco to be signed are the same types of managers that like to keep lots of guys on the bench. That means they get 75 AB when Canseco's on the team, and when they wind up as regulars a couple of years later, those 75 at-bats give them a data point in this study (the authors used a minimum of 50 AB).

Anyway, I'm mostly just thinking out loud here; I have no idea if this is the correct answer or not. The fact remains that "post-Canseco" players did tend to increase their AB. But the significance level in the regression has a hidden assumption, that the players on Canseco's teams were random. And they're not – the patterns of player AB have a lot to do with manager and GM tendencies, which means the variances of the observations are underestimated, and the significance overestimated. That's probably why, of the 11 regressions, three of their "post" AB numbers are statistically significant at the 5% level.

Indeed, look at the "change in AB" numbers for the other 10 players:

-33, 10, 20, 1, 32, -15, -22, -46, -24, 0

Compared to these, +53 isn't *that* out of place, is it? And this hardly looks like a normal distribution, which again suggests team tendencies are at work.

My best guess is the fact that players had more AB after Canseco left has more to do with circumstances than with Canseco. And the fact that players had more HR after Canseco left is almost perfectly explained by the increase in AB, and normal aging.

Canseco may very well have influenced his teammates to use steroids -- but the evidence contained in this paper does not substantiate that hypothesis very much, if at all.

(Thanks to John Matthew for the link.)

------

UPDATE: while it is correct that the regression didn't control for age, I just noticed it *did* control for years of experience. That means that the one-home-run-per-300-AB figure is more significant than I originally thought.

However, the regression assumes that power increases over the years are constant and linear, which might not be the case. For instance, if power increases faster when the player is young, the regression might underestimate power for (say) 29-year-olds, and overestimate it for (say) 39-year-olds.

My gut still says that it's still very possible that a long-term increase of 1 HR per 300AB, even after taking tenure into account, could still have many other causes, such as management decisions on one or more of Canseco's teams. And it still could be that the entire effect is caused by the players (of Canseco's six) that are generally acknowledged to have juiced.

I wish the authors had run more regressions that controlled for playing time, so that we'd have more useful data.