Can managers induce "career years" from their players?
Over at the "Ask Bill" section of Bill James' website, there was some discussion last week about the 1980 Yankees (subscription required; start at April 7). They finished 103-59 despite a team that didn't look that great on paper. Was it that manager Dick Howser somehow got more out of the players than expected?
A few years ago, I did a study that tried to estimate how much a team was affected by the "career years" or "slump years" of their players. (Go here, look for "1994 Expos".) What I did, basically, was take a weighted average of a player's stats the two years before and two years after, regress it to the mean a bit, and use that as an estimate of what the guy "should have" done that year. Any difference, I attributed to luck. In the 1980 Yankees case, it was 12 games of "career years" from their hitters, and effectively zero for their pitchers.
A bit of discussion followed; Bill James wrote that he wasn't convinced:
"I am leery of describing as luck things that we don't understand. It may well be that players had good years because Howser or someone else was able to help them have good years."
Fair enough. In response, I posted a short statistical argument that if it *was* the manager, it couldn't happen very often, and another reader (Chris DeRosa) disputed what I said (partly, I think, because I didn't say it very well).
Since "Ask Bill" is not a good place for a long explanation, I thought I start again here and better explain what I'm talking about.
----------
Suppose we knew the exact talent level of every team in the majors. That is: for every single game, between any two teams, we know the exact chance either team will win. If both teams have an equal chance, it's exactly like flipping a fair coin. If the favorite has a 64 percent chance of winning, it's like flipping a coin that has a 64 percent chance of landing heads.
In real life, this pretty much the way it works. If not, the Vegas odds on baseball games wouldn't be so close to even. If you could look at the specifics of a game and have a 90% idea of who would win that day, Vegas would routinely offer 9:1 odds on underdogs. And they don't. That means that a huge part of who wins a baseball game is unpredictable.
So, a team's season record is like a series of 162 coin tosses -- heads is a win, tails is a loss. Mathematically, using the binomial approximation to the normal distribution, you can show that the SD of team wins over a season, for a .500 team is about 6.3 wins. That is, you expect 81-81, but you could easily wind up 87-75, or even 69-93, just due to luck.
The SD drops as the team gets better or worse than .500, but it doesn't drop much. If it's a .600 team, rather than a .500 team, the SD due to "coin tossing" is still 6.2 wins. Even for a .700 team, the SD is still about six games a season -- 5.8, to be exact.
Also, there's no need to keep the assumption that all games are the same. Suppose, before every game starts, you know the exact talent of both teams, and even the exact home field advantage for that game. You can even be omniscient enough to adjust for the weather, and injuries, and the fact that the starting pitcher had a big fight with his wife last night. Before the game starts, you'll have an extremely accurate estimate of the chance of the home team winning.
Still, that chance will be substantially less than 100%. You'll still have a huge amount of luck happening. Your estimate is almost always going to be less than, say, .700. It is absolutely impossible to get much better than that, for the same reason it's impossible to predict what the temperature will be exactly one year from now.
In theory, it could be predictable -- but the predictability is over uncountable numbers of molecules, beyond any possible computing capability humans could ever devise. So what is left is essentially random.
That means that, when we total up your wins and losses for the season compared to talent, no matter how accurate your talent estimates are, you're going to find that your SD is *still* around 6.2. That's a unalterable, natural limit of the universe, like the speed of light.
--------
If you have a model for estimating team talent, a good test of that model is how close your error can get to the natural lower bound of 6.2 wins.
The most naive model is when you predict that every team will wind up 81-81. If you check that, you'll find that the standard error of your estimates is around 11 wins. If you use a prediction method like Tom Tango's "Marcel", you'll get substantially closer. You could also check any other predictions, like the Vegas over/under line. I don't actually know what those are, but I'm guessing they'd be around 8 or 9 wins.
My model is at 7.2 wins. I'm pretty sure it's better than Marcels and Vegas, but that's only because it uses more data. Oddsmakers are predicting the team's talent *before* it happens; I'm predicting it after. Obviously, I have a huge amount more information to work with. From looking at the rest of Norm Cash's career, I know that Norm Cash wasn't as good a player in 1962 as his 1961 suggested, and I can adjust accordingly. Marcel looks only backwards, so it doesn't know that.
If that seems like I'm cheating, well, not really. I'm not using the method to show how good a predictor I am. I'm using it to try to figure out, after the fact, how good a team actually was. I'm not trying to predict the future; I'm trying to explain the past.
---------
My method works like this. Suppose you have a team that talent of X wins, but, instead, it got Y wins. The difference between Y and X is, by definition, luck. How might we measure that luck?
I think that these five measurements completely add up to the amount of luck, without overlapping:
-- how much the team's hitters got lucky and had a career year;
-- how much the team's pitchers got lucky and had a career year;
-- how much the team differed from its Runs Created estimate;
-- how much the team's opponents differed from their Runs Created estimate; and
-- how much the team's wins differed from its Pythagorean Projection.
The first two items deal with the raw batting and pitching lines. The second two items deal with converting those lines to runs. And the last item deals with converting those runs to wins. (You don't have to consider the opposition's "career year", because the opposition's career year in hitting is your career year in pitching, and vice-versa.)
Any source of luck you can think of winds up in one of those five categories. A pitcher has a lucky BABIP? That shows up as a career year. Team gets lucky and hits unusually well in the clutch? Partly career years, partly beating their Runs Created estimate. Team gets lucky and goes 15-6 in extra inning games? Shows up in their Pythagorean discrepancy. Your shortstop has a lucky defensive year? That shows up in a pitcher's career year (which is based on opposition batting outcomes, and therefore includes defense).
It's all there.
--------
So, for every team since 1961, I figured out their luck in each of the five categories. As I said earlier, the "career year" luck was by players' talent estimates based on the four surrounding years. The Runs Created and Pythagorean estimates were straightforward.
After all that, the unexplained discrepancy, as I said above, was 7.2 games.
That seems very close to the law-of-the-universe binomial limit of 6.2 games. The difference, however, is substantial: it's 3.7 games. (It works that way because 7.2 squared minus 6.2 squared equals 3.7 squared).
What does that 3.7 represent? It's not luck we haven't accounted for, because, I think, we've accounted for all the luck. We haven't accounted for it perfectly -- Pythagoras and Runs Created aren't exact. And, of course, the way I estimated a player's talent isn't perfect either.
So, here's what accounts for that extra 3.7 game standard deviation:
1. imperfections in Pythagoras and Runs Created
2. the fact that my method of estimating talent for "career years" is probably not that great
3. managerial influence in temporarily making players better or worse for a single season (Billy Martin's 1980 pitchers?)
4. injury patterns that make players look better or worse (but not injuries affecting playing time; that's reflected in the estimates already)
5. other sources of good or bad single years that aren't luck or injuries (steroids? Steve Blass disease?)
6. other things I'm forgetting (let me know in the comments and I'll add them here).
If I had to guess, I'd say that #2 is the biggest of all these things. My method just looks at four years. It may not be regressing to the mean properly. It doesn't distinguish between starters and relievers. It doesn't consider age (which is fine for most ages, but not for, say, 27, when it should give an extra boost over the average of 25, 26, 28, and 29). It takes previous or future career years at face value, so that, for instance, it predicts Brady Anderson's 1997 expectation based significantly on his 1996. (If you showed a human Brady's entire career, he probably wouldn't weight 1996 quite so high.)
UPDATE: Tango describes it better than I do:
"As for the reason for that 3.7, a large portion of that is almost certainly the uncertainty of the true talent for each player. There’s only so much we can know about a player, given such a small sample as 3000 plate appearances, combined with such a narrow talent base that is MLB."----------
In light of all that, my point about Dick Howser is this: since the entire unexplained residual SD is only 3.7 games, then there can't be a whole lot of manager influence in temporarily increasing a player's talent. It's certainly possible that Dick Howser managed his team into an extra 12 games of extra talent, but things like that certainly can't happen very often.
If you square the unexplained SD of 3.7, you get an unexplained variance of about 14. Multiply that by the 26 teams that existed in 1980, and you get about 356 total units of unexplained variance.
If Dick Howsers are routine, and there's typically one every season creating a discrepancy SD of 12, that Dick Howser singlehandedly contributes a variance of 144. That's about 40 percent of the total unexplained variance for a typical league. That's a lot.
Furthermore, it's absolutely impossible for there to be an average of two and a half Dick Howsers in MLB per year, each boosting his team by 12 wins worth of talent. If that were the case, then that would account for the entire 356 units of variance, which means all the other sources of error would have to be zero. That's obviously impossible.
Even if there were only half a Dick Howser every year, that would still be 21 Howsers in the period I studied. In that case, instead of seeing the discrepancies normally distributed, we'd see a normal distribution with 21 outliers.
But we don't.
If "batting career year discrepancy" is normally distributed, we should expect about 24 teams out of 1042 to have discrepancies of 2 SD or more. The actual number of teams at 2 SD or more in the study: 25, almost exactly as expected.
We should also expect 24 teams to have discrepancies of 2 SD or more going the other way. Actual number: 22.
So there is no evidence at all that there's anything more than luck going on. That still doesn't mean that Dick Howser can't be a special case ... it could be that career years are just random, *except for 1980 Dick Howser.* But, obviously, the number alone doesn't give us any reason to believe he is. A certain number of managers are going to have as big an effect as the 1980 Yankees, regardless. (And, in fact, three other teams beat them; the 1993 Phillies led the study with a "career year hitting" effect of 13.1 games.)
So, if you think Dick Howser is something other than a random point on the tail of the normal distribution, you have to explain why. It's like when Daphne Weedington, from Anytown, Iowa, wins the $200 million lottery jackpot. You don't know *for sure* that Daphne doesn't have some kind of supernatural power. But, after all, *someone* had to win. Why not Daphne?
Labels: baseball, luck, managers, statistics
posted by Phil Birnbaum @ 4/15/2011 02:18:00 PM
13 comments
13 Comments:
One thing I'm not following here. Basically it's this sentence: "since the entire unexplained residual SD is only 3.7 games, then there can't be a whole lot of manager influence in temporarily increasing a player's talent."
Wouldn't any manager influence on player performance already have been accounted for when you figured career years?
Hi, Matt,
We know the binomial variance must be 6.2. And we know that 6.2 must include the sum of the parts of career years that are *caused by luck*.
So the part of career years that are caused by other things, like managerial influence, must lie outside the 6.2, which means they're in the 3.7.
Yes, the manager influence on player performance was already accounted for in career years. That's perhaps [one of the reasons] why we got a 7.2 instead of 6.2.
Does that make sense?
I'll preface this by saying that, while I like to think of myself as statistically able and competent, I don't have nearly as good a grip on the abstract concepts as you (or TangoTiger) clearly do (does). So a lot of the details of what you wrote I only have a loose handle on.
But it was the underlying assumptions - rather than the details - that I disagree with.
The frequent citing (overuse, in my opinion) of luck to explain variance in baseball (or other areas) is an assumption - an assumption taken on faith.
So, when you say that, for instance, teams with a true talent level of .500 win 88 games, that that's just luck because it fits in with the binomial probability distribution layout of things, I would tend to disagree.
I'll take one common example: 1-run games. It used to always be said (maybe it still is) that, because teams generally play .500 in one-run games, and because teams don't show noteworthy year-after-year persistence in playing above- or below-.500 in 1-run games, that one-run games are luck. (This used to be a daily comment in Rob Neyer's comments, back around 2000 and 2001, when I think the sabermetric community was nearing its peak of arrogance and snarkiness.)
But that makes no more sense than saying that an MMA fighter with a .500 record was just lucky in his wins and unlucky in his losses. Or saying that any of the 50,000 Americans who die in a car accident were merely unlucky.
In fact, we can all see through casual empiricism that, unlike with flipping a coin, there is a direct cause-and-effect basis for these events. The same thing with 1-run games. If Roy Halladay outduels Ricky Nolasco in a 3-2 game, that doesn't mean that Halladay was lucky and Nolasco was just unlucky. If you go back and watch the game, you can usually see precisely the decisions and actions that were taken that lead to that outcome. (Even things like "ground balls with eyes" aren't the "luck" that many people think, when you consider all the hitters - Ichiro, Tony Gwynn, Derek Jeter - who, as far as I can tell, get an inordinate number of hits this way.)
So that principle of "the events follow a statistical pattern, and therefore they were caused by the statistical pattern" is just totally mistaken. To me, it's just a way of saying, "I don't know, so I'm going to pretend that there's no rhyme or reason to anything."
Just the same as you echoed Bill James's sentiment that attributing things to psychology (which, like luck, is similarly nebulous and foggy) should be a last resort, and should be said with great qualification and modesty, so, too, should attributing things to luck.
Don't get me wrong - I think that "luck" is often a better placeholder for sequences of events (such as an extreme number of players having career years, as in this Yankees example) - than some of the other theories that get proposed, like saying the manager caused everything. But citing luck is still just that: an explanation for things that we don't really know for certain.
So your previous article was, "Psychology should be your last resort". I think that a corollary to that is that, "Luck should be your second-to-last resort".
I know I'm dodging the specifics of your arguments - which were obviously reasoned and explained thoughtfully - but I hope that the overall point I was making still seems relevant.
Hi, David,
I think it's just a matter of definition. When we say a player is "lucky" because (for instance) he goes 4-for-4 one day, we agree with you that it's still a result of the players' decisions, and the laws of physics and biology.
What we mean by "luck" is the part of the cause-and-effect relationship that humans will never be able to calculate because the causes go down to such a deep level that we can't measure them.
The results of a coin flip are considered random and lucky, even though there's just as much biology and physics involved as a baseball swing. That's because we know humans can't control their muscles and brain enough to flip a head or a tail at will.
The same is true in baseball. Your example of Ichiro, for instance ... he's good enough to get a hit 1 out of every three times he puts a ball in play. He has good skills, but not perfect. If he goes 3-for-3 instead of 1-for-3, we call that luck, just as if we asked him to flip heads and he flipped 3-for-3, we call *that* luck.
In short: luck is, by definition, the part of "cause and effect" that humans can't control or predict. It doesn't have to be God that flips the coin to call it luck. Ichiro can flip the coin, too, instead, so long as there is some part of each of Ichiro's coin flip that is, for practical purposes, just as unpredictable as if God did it.
Sorry if this is a bit murky. I'm sure others have described it better, but I don't know where.
You wrote:
"What we mean by "luck" is the part of the cause-and-effect relationship that humans will never be able to calculate because the causes go down to such a deep level that we can't measure them."
This is pretty much my point: instead of saying that it's luck, say, "It could be luck. It could be psychology. It could be a great managerial season. I don't know how to prove what caused this."
As for your coin flip example, that is totally off because (a) I'm sure that lots of physicists can explain the gravitational effects and the ricochet and all that of a coin, and (b) there is no intent of partiality in a coin flip. Flipping a coin (or rolling dice) is supposed to be, effectively, a tangible version of a random number generator. So, to the extent that there are any factors besides luck at play, that's a flaw in the coin flip system.
As opposed to coins and dice which are random luck by intent, baseball games - like most human endeavors - are the result of intelligence and intent. Again, if the Phillies beat the Marlins 4-3 (or the opposite, like last night), that's the result of dozens and dozens of human beings all making decisions and striving to create one specific outcome.
In other words, randomness (like rolling dice or flipping a coin) is purposefully dumb; most human endeavors (like playing a baseball game or driving a car) are the product of intent and intelligence.
Therefore, I believe that the default explanation for the events on a baseball field should be something to do with the humans making the decisions, not randomness. It might follow similar patterns of randomness (or dumbness, to put it another way), but it's not dumb. An American might have a 1/6,000 chance of dying in a car crash, but it's obviously inaccurate to say that everybody who died in a car crash was merely unlucky. Most were victimized by their own poor driving, the poor driving another driver, bad road construction, or some other factor spawned from intent and human intelligence.
Obviously, I do think that pure, dumb luck definitely has to play some measure of significance in baseball, but I reject the assumption that, anything that false into statistical models is the result of dumb luck.
(And then there's age-old statistical issue of what's "statistical significance", anyway? If the binomial probability formula says that something has a 20% chance of happening, some people might point to the 5% threshold of statistical significance and say that it was all dumb luck. But others could say that that 5% threshold is totally arbitrary. But I guess that that's another layer of complexity that pulls the discourse too far off track.)
David,
I think you're defining "luck" out of existence.
If I pick six numbers for a lottery ticket and win, most people would say I was lucky. The fact that physicists could explain why those particular six balls came out of the machine ... well, that has nothing to do with it.
Luck has nothing to do with intent, either. I hear that there are some people, who, with practice, can flip coins and get heads more than 50% of the time. Suppose they can do it 55% of the time. Still, even though they WANT to do it every time, 45% of the time they'll fail. Which 45%? That can't be calculated. It's "luck".
Overall, the coin toss is a combination of skill and luck. The skill is what makes the tosser a 55% tosser instead of a 50% tosser. The luck is whether this particular toss will land on the 55% side or the 45% side.
Still, if you want to call it something else, feel free. But it will still follow exactly the same laws of statistics and mathematics.
You wrote:
"Still, if you want to call it something else, feel free. But it will still follow exactly the same laws of statistics and mathematics."
Now you're trying to make this a semantics argument. I believe that I was providing concrete reasonings for the difference between a random number generator (a lottery ball, dice, or a coin flip) and a system founded upon intelligence like baseball (and, to add to that, complexity).
A statistician could make the case that, in every evenly-matched MMA fight, the winner was just lucky, while the loser was unlucky. Obviously, this is not true: each of the fighters train for many months and make many decisions (in the match and before) which determine the outcome, not randomness. Similarly, you could make the case that every 0-10 slump or 7-10 hot streak by a hitter is pure luck. In fact, it's very possibly the result of intelligence and circumstance.
It's not that I'm defining luck out of existence, it's that you're overusing it and just defaulting to it in as the explanation for every event that you don't understand. We don't have access to all the information (nobody does) that goes into every event, and so therefore every model is incomplete, by definition.
And to address your (total non sequitur) situation where somebody has a skill at flipping a coin 10% above average, you've now injected intelligence (a skilled coin-flipper) into a scenario which was previously dumb, and now you don't know which of the 45% coin-flip failures were the result of luck. Maybe the guy's wrist gets sore after a number of flips and so his technique is off. Maybe he's doing it in Miami where the air is thicker which throws off his estimates of the number of rotations the coin flips into the air. The point is, once intelligence comes into play, things get incredibly complicated. And just because you're incapable of diagnosing the full complexity in the situation (a baseball season or a strategic coin-flipper) doesn't mean that all that complexity is just dumbness.
You're reasoning is this: All coin flips are 50/50, and, therefore, every coin flip is luck.
This is accurate (excepting the non sequitur example of a skilled coin flipper). But the follow up....
....And everything else that is 50/50 is luck, too. (I.e., the claim that all 1-run games are luck because, historically, all teams are basically 50/50 in one-run games.)
This is inaccurate. Not all 50/50 scenarios are the same, nor 60/40 or whatever else.
So, just because an exceptionally good season from the Yankees fell within the historical parameters doesn't mean that it's luck. That is what I believe.
David: OK, I think I understand better what you're saying now.
>"A statistician could make the case that, in every evenly-matched MMA fight, the winner was just lucky, while the loser was unlucky. Obviously, this is not true: each of the fighters train for many months and make many decisions (in the match and before) which determine the outcome, not randomness."
Right, I would disagree with you here ... it probably IS true that the winning fighter was lucky.
Still, the importance of training and decision making is absolutely right. If either fighter DIDN'T train and make exceptionally good decisions, then it wouldn't be 50-50. It would be 90-10, or 100-0 (if GSP were fighting me).
Humans don't have complete control over their own actions, down to the thousandth of a second or thousandth of an inch. The difference between team A winning a game and an equally talented team B winning a game usually comes down to thousandths of a second in swing time, or thousandths of an inch in bat placement.
The difference between an Ichiro and someone else is that Ichiro may be random with an SD of 1/1000 of a second, and someone else might be random with an SD of 1/500 of a second.
If you don't want to call the uncontrollable limitations of the human brain "luck", what do you want to call it?
"If you don't want to call the uncontrollable limitations of the human brain "luck", what do you want to call it?"
I call them challenges which need to be identified and quantified. Or, alternatively, "We don't know yets".
Your position seems to be comparable to the position of medicine in Medieval times. There, when something was unknown, they'd (supposedly) just go ahead and decree that it was Divine influence: a person collapsed plowing crops because they were a sinner rather than because it was a 110-degrees and his heart gave out. You're doing the same thing, but we're substituting "Luck" for "Divine influence".
You would call it luck when Roy Halladay outduels (or gets outdueled by) Ricky Nolasco. I would, if inspired to try and identify the cause, watch the game or survey the Pitch F/X and see whether Nolasco hung an exceptional number of curveballs which were hit for line drives at a higher clip than his well-placed curve balls. Or maybe Nolasco allowed too many fast runners on base who, in turn, the first baseman had to guard, which then opened up the right side of the infield for the following hitters, which allowed them to punch an extra pair of weak grounders through that right side. Etc., etc., etc.
Frequently, we won't be able to determine how Halladay outdueled Nolasco. But you would declare that you do have the answer: Luck. I would say that we don't have the answer, but it could possibly be luck.
Same thing applies to the Yankees having an unusually great season. It might be luck. But I think that there's a very good chance that there's a causal explanation instead of (or in addition to) luck.
But you don't know. And by pretending like you do and that the matter is settled, you're (a) wrong and (b) discouraging further study and analysis which could yield knowledge and illumination.
That's my opinion.
"We don't know yets".
But you could argue that EVERYTHING is a "we don't know yet," so nothing is luck.
Are lottery numbers random? We both say yes, even though "we don't know yet" is just as true here. If we just analyze the balls and the initial state, we could, in theory, know in advance which balls would fall, and so lottery numbers aren't luck. We just "don't know yet."
Why wouldn't you say the same for human performance? Sure, if we analyze Ichiro's brain and the pitcher's brain enough, we could, in theory, predict whether he'll go 0-for-4 or 3-for-3 with a walk.
You seem willing to admit that coins and lottery balls are "luck", but Ichiro's brain, which is much, much more complex, is not. How come?
I appreciate the discourse and I love your blog, but we're going around in circles here.
You wrote:
"You seem willing to admit that coins and lottery balls are "luck", but Ichiro's brain, which is much, much more complex, is not. How come?"
There is a deliberate intent for a specific outcome in one scenario (baseball), and there is the exact opposite - designed randomness - in the other system.
Also, you're now citing psychology. I haven't cited that as an explanation for anything (in fact, I specifically called it foggy and nebulous), but you're implying that I did. In the examples I provided (for 1-run games, for instance), I just mentioned just a few (of many) potential strategic causes that all baseball players would point to.
As for the Yankees 1980 season, by declaring that you have the answer - luck - (or, alternatively, something psychological that's too complex for us to ever know), you're missing out on tons of other interesting prospective causes. Maybe the manager was great at stealing signs (after all, it was only the hitters who had great years) but that, in the following year, teams picked up on it and adjusted accordingly. Maybe the manager came up with some awesome new brew of amphetamines that were giving his players extra energy, but that every team had it the following year. Maybe, at home games, the Yankees were doing something funny with the baseballs (see: last year's juiced balls in Colorado, which is the probable explanation for their insane home field performance in the final months of past seasons) and, just as they did with the Rockied, MLB effectively granted them a pass for their cheating and just quietly swept it under the rug (and, hopefully, put a stop to it).
We could go on and on and on. It's actually kind of fun.
Once again, the point is is that you don't know that it's luck. Saying you do is patently false. And because you don't know what the cause is, saying that the cause is way too complex is also false.
And please stop with the dice rolls/lottery balls/coin flips. There is no intelligence involved with those systems. A physicist can quantify and explain them, but the outcomes are still random, by design. Baseball games are the antithesis of that: you have two sides actively in control of an outcome.
OK, we may have to quit. I'm really trying to get a feel for what your point is.
Here, let me show you where I outlined my version of "luck" in more detail:
http://sabermetricresearch.blogspot.com/2010/07/poor-play-can-be-caused-by-bad-luck.html
Maybe that will make my argument clearer?
I spent five years as a professional gambler specializing in baseball futures bets. My calculations showed that the best humans/computers and the Vegas over/under lines* both manage a standard deviation of about 8.5 games, as you estimated.
*-i.e. the closing lines, which have moved in response to one-sided action. For example, the 2008 Rays opened at 68.5 wins and closed at 76.5.
On propositions such as division winners, most of the profitable bets are on longshots. I have speculated that the bookie is calculating odds based on fixed team winning percentages and the platonic 6.2 game standard deviation. If you screw up this step, it's easy to mistake what is really a 20-1 longshot for a 100-1 and set the odds accordingly. You won't cash this bet often, but your edge over the house is tremendous.
Good post. Can we all agree not to get upset by the use of 'luck' as a shorthand for 'unpredictable event'?
Post a Comment
<< Home