New York Times article on steroids and performance

There's another statistical follow-up to the Mitchell Report, this one in today's New York Times. There, Jonathan R. Cole and Stephen M. Stigler – a sociologist and statistics professor, respectively – look at whether there is evidence that steroids improved players' performances.

For the players named by Mitchell, the authors looked at their career performance (ERA, HR, BA, SLG) before their steroid year, and their career performance after:

"After excluding those with insufficient information for a comparison, we were left with 48 batters and 23 pitchers. ... For pitchers there was no net gain in performance and, indeed, some loss. Of the 23, seven showed improvement after they supposedly began taking drugs (lower E.R.A.’s), but 16 showed deterioration (higher E.R.A.’s). ... Hitters didn’t fare much better. For the 48 batters we studied, the average change in home runs per year “before” and “after” was a decrease of 0.246. The average batting average decreased by 0.004. The average slugging percentage increased by 0.019 — only a marginal difference."

But the authors didn't adjust for age. It could be, as the authors conclude, that the steroids were ineffective. Or, it could be that the players who did take PEDs tended to be older, and the increased performance worked to offset age-related decline. For instance, the authors write,

"Roger Clemens is a case in point: a great pitcher before 1998, a great (if increasingly fragile) pitcher after he is supposed to have received treatment. But when we compared Clemens’s E.R.A. through 1997 with his E.R.A. from 1998 on, it was worse by 0.32 in the later period."

Of course, in 1998, Roger was 35 years old. For a pitcher to lose only 0.32 in ERA from age 35 to 44 ... well, that's remarkable. Adjusting for park -- which the article did not do -- would make the decrease worse. But the gradualness of the decline would still be impressive. I'd say that if Clemens took PEDs, it would be a reasonable presumption that they worked.

Also, what's with the "after excluding those with insufficient information?" If it's information on steroid use, that's one thing. But if the information is "insufficient" because the player dropped out of the major leagues or got injured, that's an important data point.

Finally, the authors say,

Our results run contrary to the prevailing wisdom. One reason might be that most baseball skills depend primarily upon reaction times and judgments, factors unaffected (or even degraded) by these drugs.

In that case, wouldn't steroids be of more help those players whose skills depend on muscle strength, like power hitters and strikeout pitchers? Say, Barry Bonds or Roger Clemens?

HT: J.C. Bradbury, who is more impressed than I am with the article and its authors. Bradbury in turn links to a BTF post, which has a couple of good rebuttal points in its comments.

UPDATE: here's a better, more thorough, more sarcastic analysis.

UPDATE: Isn't it annoying how the New York Times insists on writing "E. R. A." instead of just "ERA" like the rest of the world does it? I saw one article where they kept referring to the "N. F. L.", except the TV channel was the "NFL Network." I'm sure they have their own internal logic in their style book about how "N. F. L." is actually an abbreviation, but "NFL Network" is its proper copyrighted name. But I don't care. They still wind up looking like a bunch of pedantic dorks.