How do coaches improve their teams?

How much do coaches improve player performance? According to Dave Berri, of "The Wages of Wins," not much at all. According to a study coauthored by Berri (as quoted in this Ryan McCarthy article in Slate), only eight out of 19 coaches studied had any "statistically discernible" effect on performance. And only the difference between the very best and very worst coaches was statistically significant.

I haven't seen the numbers, because the study is unpublished. However, at a 5% significance level, you'd expect about one out of the nineteen coaches to be significantly different from zero, so the fact that the top coach was significantly better than the bottom coach is to be expected.

Anyway, the study quotes a Dan Oliver study as coming to a different conclusion. That analysis is in Oliver's book "Basketball on Paper"; I, um, haven't got to that chapter yet, but I really should do that soon. McCarthy says that Oliver shows

Coaches like Phil Jackson can be worth up to an additional 12 wins per year.

I gotta read that chapter: 12 wins sounds like a huge number, and I'm now very curious.

In any case, my gut leans to Berri's finding, that coaches don't affect a player's performance all that much. But, then again, I've always thought that there's no way a baseball manager could consistently cause a team to beat its Runs Created estimate, but some managers do appear to be able to do that. So I may be wrong.

My thinking is that coaches and managers *can* make a big difference, but that comes about in the choice of who to play. I remember back in the early 1980s, when Tony Fernandez, by all accounts ready for the major leagues, sat in the minors while the Blue Jays stuck with the mediocre Alfredo Griffin. Glancing at the Baseball Reference pages for those two guys, we find that, in 1984, Griffin had –3.2 batting wins, while Fernandez had –0.6. So if manager Bobby Cox had played Tony and benched Alfredo, the Jays would have won almost three extra games – which is huge. (It wouldn't have affected that particular pennant race – the Jays won 89 games in 1984, but that was the year the Tigers ran away with it at 104-58.)

I'm sure you can think of other examples ... I remember Bill James once writing that Dick Williams had a perverse fascination with Rodney Scott, keeping him on the roster when he obviously wasn't of major-league caliber. And in his Managers book, James also called Cito Gaston "inert," because he used his bench so little. However, Cito did win two consecutive World Series. And when you think about it, why would you bench Roberto Alomar, even for a few games a year, to play Luis Sojo? Unless you think the days off would *really* help Alomar, you're just throwing away runs.

What I'd really like to see is a study that somehow figures out what managers made the best use of the talent they had available. I'm sure every manager has, at some point, played a mediocre established player instead of an unknown but more-talented one. But which ones do it the most? Who has a penchant for leaving Tony Fernandezes languishing while Alfredo Griffins get playing time? Who doesn't bother giving 150 plate appearances to some crappy 35-year-old first baseman with no power? Who knows which players are good enough to be given a shot?

It would be a tough study to do: you'd have to estimate what all the bench players were capable of, figure out who was in the minors and how good they were, and compare all those possibilities to the guys who actually played. But I'd bet that if you found a way to do that, you'd discover some managers are legitimately worth an extra game or two a year, just from their personnel decisions.

Hat tip: Carl Bialik