Sunday, November 19, 2006

Can money buy wins? Team correlation alone can't tell you

In the previous post, I linked to the Wages of Wins study that found a correlation of less than 0.4 between single-season NBA team payroll and wins. The authors have argued that because the correlation is low, we can conclude that money can't buy wins.

That got me thinking ... is that really true? It occurred to me that if the team payrolls vary in a narrow range, that should reduce the correlation, because there's less room for the relationship to make itself evident.

To check that, I ran an experiment. I set up a situation where payroll was 100% correlated with talent. Then, I simulated 30 independent seasons of 82 games, first, where the salary distribution was wide, and, then, where the salary distribution was narrow.

First, the wide distribution. Teams vary between .300 and .700 in a distribution shaped like the roof of a house. (Technically, talent was taken as .300 + (rnd/5) + (rnd/5), where "rnd" is uniform in (0,1).). Here are five correlation coefficients from successive runs. (These are r's; square them to get r-squareds.)

.78 .76 .86 .90 .84

Now here's the narrow distribution. Teams vary from .450 to .550:

.36 .32 .35 .07 .11

Clearly, the variability of payroll makes a huge difference in the correlation.

We can make the correlation come out as low as we want, just by reducing the teams' spending variance. If a salary cap and floor forced every team to spend within, say, 1% of each other, the correlation would probably be very close to zero.

And, therefore, the conclusion that low correlation implies inability to price talent is just not true. Here, payroll buys talent with 100% correlation, and there is no doubt that a team that chooses to spend more will win more games. And that's true whether the payroll/wins correlation is .7, or .3, or .1, or even zero.

It sounds illogical, but it's true: the correlation between team spending and wins, taken alone, is not enough to tell you anything about whether team spending can buy wins.

At Sunday, November 19, 2006 2:58:00 PM,  Guy said...

1) player performance is consistent, yet
2) payroll-win correlation is low, therefore:
3) salary must not be tied to player performance/talent (i.e. basketball GMs pay for the wrong skill).
If you're going to conclude #3, shouldn't you look at the R between salary and performance at the player level? Maybe they do this in the book and just don't reference it in their blog posts. But since they've gone to the trouble of developing player ratings, why infer a weak relationship between salary and performance when you can just tell us what it is?

* *

I assume one reason the payroll-win link is weaker in basketball is that weak teams get good draft picks, and it is far easier to predict which draftees will succeed in the NBA than it is in MLB. So this probably offsets the financial advantage of wealthy teams in a major way. I'd guess that and other economic factors are at least as important as GMs' misvaluing player skills.

At Sunday, November 19, 2006 4:02:00 PM,  Phil Birnbaum said...

Excellent point ... you'd expect Berri et al to use only team-level correlations if they didn't have a measure of player performance they thought was reliable. But then they go through pages and pages of their own player performance measure, and never revisit the salary question!

At Monday, November 20, 2006 12:54:00 AM,  Phil Birnbaum said...

Also agree on the second point in your comment, Guy.

I would add that since basketball has only five players on the court, and a player can play almost the whole game, the impact of a key draft choice is quite large.

Albert Pujols can only come up, say, 12% of his team's plate appearances, and has a minimal effect on defense. But LeBron James can be 20% of his team's defense, and even more than 20% of his team's offense.

At Monday, November 20, 2006 9:08:00 AM,  Beamer said...

Great point Guy -- I think you have hit the nail on the head. Is there a study that links wins (or OPS, or somesuch performance measure) to salary. If not it'd be fairly easy to create -- I'll have a go for BTB.

Phil -- you really have a bee in your bonnet about this! But rightly so. Anyway, just on your little simulation you are quite correct, of course, but it isn't a surprise. It is a well know attribute of correlation that the greater the variance between the datasets then the larger the correlation. Tango did a good empirical study of that for his DIPS solved study. And if you think about the correlation formula it is covariation (x,y) / var(x)*var(y). If one of var(x) or var(y) have little variance in it then the cov(x,y) will also be small, which will outweigh the two variance terms.

Good stuff as always, Phil. Keep it up, I love these posts