Tuesday, October 14, 2014

Corsi, shot quality, and the Toronto Maple Leafs, part VI

A year ago, I wrote about how I wasn't completely sold on Corsi and Fenwick as unbiased indicators of future NHL success. In a series of five posts (one two three four five), I argued that it did appear that "shot quality" issues could be a big factor -- if not for all teams, then maybe at least for some of them, like, perhaps, the Toronto Maple Leafs.

I haven't kept up with hockey sabermetrics as much as I should have, but, as far as I know, the issue of how much shot quality impacts Corsi remains unresolved.

In that light, and in hopes that I haven't rediscovered the wheel, here's some more evidence I came across that seems to suggest shot quality might be a bigger issue than even I had suspected.

It's from a post at Hockey-Graphs, where Garret Hohl looked at some shot quality statistics for every NHL team, for approximately the first 30 road games of last season (2013-14). 

His data came from Greg Sinclair's "Super Shot Search," which plots every shot on goal by plotting it on the ice surface. Sinclair's site allows you restrict your search to what he calls "scoring chances," which are shots taken from closer in. Specifically, a "scoring chance" is defined as a shot on goal taken from within the pentagon formed by the midpoint of the goal line, the faceoff dots, and the tops of the two circles. 

Hohl calculated, for every team, what percentage of opposing shots were close-in shots. (He limited the count to 5-on-5 situations in road games, in order to reduce power-play and home-scorer biases.)  Data in hand, he then ran a regression to see how well a team's "regular" Fenwick corresponded to its "scoring chances only" Fenwick. His chart shows what appears to be a strong relationship, with a correlation of 0.83. 

However: the biggest outlier was ... Toronto. 

Just as in the previous two seasons, the Leafs continued to outperform their Fenwick in 2013-14. What Hohl has done is to produce some data that shows that the effect resulted, at least in part, by their opposition taking lower quality shots. 

----

Anyway, the Leafs are really just a side point. What struck me as much more important are some of the other implications of the data Hohl unearthed. Specifically, how teams varied so much in those opponent scoring chances. The differences were much, much larger than I expected.

I'll steal Hohl's chart:




The Minnesota Wild defense was the best at limiting their opponents to weaker shots: only 32.3 percent of their shots allowed were from in close (206 of 637). The New York Islanders were the worst, at 61.4 percent (475 of 773). 

Shot for shot, the Islanders gave up twice as many close-in chances as the Wild. 

Could this be luck?  No way. The average number of shots in Hohl's table is around 750. If the average scoring-chance ratio is 44 percent, the SD from binomial luck should be around 1.8 percentage points. That would put the Islanders around 9 SD from the mean, and the Wild 7 SD from the mean. 

The observed SD in the chart is 5.6 percentage points. That means the breakdown is:

1.8 SD of theoretical luck
5.3 SD of real differences
--------------------------
5.6 SD as observed

Now, the "real" differences might be score effects: shooting percentages rise when a team is ahead, presumably because they take more chances and give up more odd-man rushes, and such. Those effects are large enough that they screw up a lot of analyses, and I wish more of those little studies you find on the web would limit themselves to 5-on-5 tied to avoid those biases.

But, in this case, the differences are too big to just be caused by score effects.

In 5-on-5 situations from 2007-2013, the league shooting percentage was 7.52 percent when teams were tied, but 9.19 percent for teams ahead by 2 goals or more. As big an difference as that is, it can't be that the Islanders were behind 2+ goals that much that it could make such a huge difference in scoring chances.

From my calculations, the difference between the Islanders and Wild is something that would happen naturally only if the Islanders were *always* down 2+ goals, and the Wild were *always* up 2+ goals.** But that obviously isn't the case. In fact, the Islanders were down 2+ goals only about 10 percent more often than the Wild last year, and up 2+ goals only 21 percent less often. The total of the two differences is about eight periods total out of a full 5-on-5 road season.

(** How did I figure that?  Suppose the shooting percentage on close shots is 13%, and 4% on far shots. At 45 percent close and 55 percent far, you get a shooting percentage of 8.1% percent. At 65 percent close, and 35 percent far, shooting percentage rises to 9.8%. That's a little bigger than the difference between up 2+ and tied.

So, it seems like, when you're up 2+ goals, 60 to 65 percent of your shots are scoring chances, compared to 35 to 40 percent when you're down 2+ goals.)

------

As for the Leafs: they were fourth-best in the league in percentage of shots that were scoring chances, at 38.2%. That's despite -- or because of? -- allowing the most shots, by far, of any team in the sample, at 926. (The second highest was Washington, at 843.)

It seems to me like this is significant evidence that teams vary in the quality of shots they allow -- in a huge way. The score effects can't be THAT large.

The only possibility that I can think of is biased scorers. But Hohl confirms that each team had an assortment of opposition home team scorers and rinks, so that shouldn't be happening.

-----

Here's some additional evidence that the scoring chance data is meaningful. 

I ran a correlation between team scoring chance percentage and goalie save percentage. If scoring chance percentage didn't matter, the correlation would be low. If it did matter, it would be high. (For save percentage, I used 5-on-5, tie score, both home and road.)

The correlation turned out to be ... -0.44. That's pretty high. (Especially considering that the scoring chance percentage was based on only 30 road games per team.)  

The SD of save percentage was 0.96 percentage points. The SD of scoring chance percentage (after 3/4 of the season) was 5.6 points. 

That means for every excess percentage point of scoring chance percentage, you have to adjust save percentage by 0.075 percentage points. 

The Los Angeles Kings gave up a bit more than 3 percentage points weaker shots than normal. That had the effect of inflating their goalies' save percentage by about 0.25 points. So, we can estimate that their "true talent" was closer to 93.45 than 93.7. 

If you like, think of it as two or three points of PDO: the Kings move from 1000 to 997.5 on this adjustment. 

For Toronto, it's five points: they drop from 1019 to 1014. 

The Rangers, for one more example, went the other way -- they gave their opponents 8 percentage points more close-in shots than average. Adjusting for that would boost their adjusted save percentage from 91.6 to 92.2, and their PDO from 974 to 980.

-----

OK, one more bit of evidence, this time subjective.

Recently, a survey from nhl.com ranked the best goalies in the league, from 1 to 14, with 15-18 mentioned in the footnotes. (I'm leaving out John Gibson, who only played one regular-season game, and I'm considering goalies not mentioned to have a ranking of 19.)

I checked the correlation between team goalie ranking and save percentage. It was -0.45. Again, that's pretty strong, considering how subjective the rankings are. 

Of course, some goalies were probably ranked high *because* of their the save percentage. So cause and effect are partly mixed up here (but I think that will actually strengthen this argument).

For the next step, I adjusted each goalie's save percentages to give credit for the quality of the shots their team faced. That is, I raised or lowered their SV% for the shot quality percentages listed in Hohl's post, at the rate of 0.075 points we discovered earlier. 

What happened?  The correlation between ranking and SV% got *stronger* -- moving from -0.45 to -0.50. 

It looks like the voters "saw through" the illusion in save percentage caused by differing shot quality. Well, that might be giving them too much credit: they might have ignored save percentage entirely, and just concentrated on what they saw with their eyes. Actually, I'm probably giving them too *little* credit: they're no doubt basing their evaluations on a full career, not just one season, and maybe team shot quality evens out somewhat in the long run.

Either way, when the voters differed from SV%, it was in the direction of the goalies who faced tougher tasks.  I think that's reasonable evidence that differences in shot quality are real. 

Oh, and one more thing: the highest correlation seems to occur almost exactly at the theoretical adjustment the regression picked out, 0.075. When I drop the adjustment in half (to 0.0375), the correlation drops a bit (-0.48, I think). When I double the adjustment to 0.15, the correlation drops to -0.44. 

Now, that *has* to be coincidence; the voters can't be that well calibrated, can they? And ranking numbers of 1 to 19 are kind of arbitrary.

Still, it does work out nicely, that the voters do seem to agree with the regression.

------

I think all this casts serious doubt on the idea that PDO (the sum of team shooting percentage and save percentage) is essentially random. The Islanders had a league-worst PDO of 982, but that's probably because their opponents took 61.4% of their shots from close-in, compared to the Islanders' own 42.8%. In other words, if you calculate a "shot quality PDO", the Islanders come in at 814. (That's calculated as 428 + (1000-614).)

The Leafs had the league's fourth best PDO, at 1019. But their shots were much higher quality than their opponents', 47.2% to 38.2%. So their "shot quality PDO" was 1090. 

For all 30 teams, the correlation between PDO and "shot quality PDO" was 0.43 -- signficantly high. The coefficient works out to approximately a 1:10 ratio. The Islanders' -186 point "shot quality PDO" difference translates to around -19 points of PDO. The Leafs' +90 works out to about +9.

I'll show data and work out more details in a future post (probably next week, I'm out of town for a few days starting tomorrow). 

(One thing that's interesting, that I want to look into, is that the SD of team quality shot percentage *for* is only about half of the SD of quality shot percentage *against* (2.7 versus 5.6). Does that mean that defenses vary more than offenses? Hmmm...)

------

So I think all of this comprises strong evidence that teams differ non-randomly in the quality of shots they allow. That doesn't invalidate the hypothesis that Corsi is still a better predictor of future success than goals scored. But it *does* suggest that you can likely improve Corsi by adjusting it for shot quality. And it *does* suggest that PDO isn't random after all.

In other words: Corsi might be misleading for teams with extreme shot quality differences.

A baseball analogy: using Corsi to evaluate NHL teams is like using on-base percentage average to evaluate MLB teams. Some baseball teams will do much better or worse than their "OBP Corsi", for non-random reasons -- specifically, if they have high "hit quality" by hitting lots of home runs, or low "hit quality" by building their "OBP Corsi" on "lower quality" walks.

In 2014, the Orioles were fifth-worst in the American League with an OBP of only .311. But they were above average in runs scored. Why?  Mostly because they hit more home runs than any other team, by a wide margin.

Might the Toronto Maple Leafs be the Baltimore Orioles of the NHL?


Labels: , , , , ,

Sunday, September 28, 2014

Experts

Bill James doesn't like to be called an "expert." In the "Hey Bill" column of his website, he occasionally corrects readers who refer to him that way. And, often, Bill will argue against the special status given to "experts" and "expertise."

This, perhaps understandably, puzzles some of us readers. After all, isn't Bill's expertise the reason we buy his books and pay for his website?  In other fields, too, most of what we know has been told to us by "experts" -- teachers, professors, noted authors. Do we want to give quacks and ignoramuses the same respect as Ph.Ds?

What Bill is actually arguing, I think, is not that expertise is useless -- it's that in practice, it's used to fend off argument about what the "expert" is saying.  In other words, titles like "expert" are a gateway to the fallacy of "argument from authority."

On November 8, 2011 (subscription required), Bill replied to a reader's question this way:


"I've devoted my whole career to battling AGAINST the concept of expertise. The first point of my work is that it DOESN'T depend on expertise. I am constantly reminding the readers not to regard me an expert, because that doesn't have anything to do with whether what I have to say is true or is not true."

In other words: don't believe something because an "expert" is saying it. Believe it because of the evidence. 

(It's worth reading Bill's other comments on the subject; I wasn't able to find links to everything I remember, but check out the "Hey Bill" pages for November, 2011; April 18, 2012; and August/September, 2014.)

Anyway, I'd been thinking about this stuff lately, for my "second-hand knowledge" post, and Bill's responses got me thinking again. Some of my thoughts on the subject echo Bill's, but all opinions here are mine.

-------

I think believing "experts" is useful when you're looking for the standard, established scientific answer.  If you want to know how far it is from the earth to the sun, an astronomer has the kind of "expertise" you can probably accept.

We grow up constantly learning things from "experts," people who know more than we do -- namely, parents and teachers. Then, as adults, if we go to college, we learn from Ph.D. professors. 

Almost all of our formal education comes from learning from experts. Maybe that's why it seems weird to hear that you shouldn't believe them. How else are you going to figure out the earth/sun distance if you're not willing to rely on the people who have studied astronomy?

As I wrote in that previous post, it's nice to be able to know things on your own, directly from the evidence. But there's a limit to how much we can know that way. For most factual questions, we have to rely on other people who have done the science that we can't do.

-------

The problem is: in our adult, non-academic lives, the people we call "experts" are rarely used that way, to resolve issues of fact. Few of the questions in "Ask Bill" are about basic information like that. Most of them are asking for opinion, or understanding, or analysis. They want to pick Bill's brain.

From 1/31/2011: "Would you have any problem going with a 4-man rotation today?"

From 10/7/2013: "Bill, you wrote in an early Abstract that no one can learn to hit at the major league level. Do you still believe that?"

From 10/29/2012: "Do you think baseball teams sacrifice bunt too much?"

In those cases, sure, you're better off asking Bill than asking almost anyone else, in my opinion. Even so, you shouldn't be arguing that Bill is right because he's an "expert."  

Why?  Because those are questions that don't have an established, scientific answer based on evidence. In all three cases, you're just getting Bill's opinion. 

Moreover: all three of those issues have been debated forever, and there's *still* no established answer. That means there are opinions on both sides. What makes you think the expert you're currently asking is on the correct side? Bill James doesn't think a four-man rotation is a bad idea, but any number of other "experts" believe the opposite. 

Subject-matter experts should agree on the basic canon, sure. It should be rare that a physics "expert" picks up a textbook and has serious disagreements with anything inside.

But: they can only agree on answers that are known. In real life, most interesting questions don't have an answer yet. That's what makes them so interesting!

When will we cure cancer? What's the best way to fight crime? When should baseball teams bunt? Will the Seahawks beat the spread?

Even the expertest expert doesn't know the answer to those questions. Some of them are unknowable. If anyone was "expert" enough to predict the outcome of football games, he'd be the world's richest gambler. 

-----

All you can really expect from an expert is that he or she knows the state of the science.  An expert is an encyclopedia of established knowledge, with enough understanding and experience to draw inferences from it in established ways.

Expertise is not the same as intelligence. It is not the same as wisdom. It is not the same as insight, or freedom from bias, or prescience, or rationality.

And that's why you can get different "experts" with completely different views on the exact same question, each of them thinking the other is a complete moron. That's especially true on controversial issues. (Maybe it's not that controversial issues are less likely to have real answers, but that issues that have real answers are no longer controversial.)

On those kinds of issues, where you know there are experts on both sides, you might as well flip a coin as rely on any given expert.

And hot-button issues are where you find most of the "experts" in the media or on the internet, aren't they?  I mean, you don't hear experts on the radio talking how many neutrons are in an atom of vanadium. You hear them talking about what should be done to revive the sagging economy. Well, there's no consensus answer for that. If there were, the Fed would have implemented it long ago, and the economy would no longer be sagging. 

Indeed, the fact that nobody is taking the expert's advice is proof that there must be other experts that think he's wrong.

Sometimes, still, I find myself reading something an expert says, and nodding my head and absorbing it without realizing that I'm only hearing one side. We don't always conciously notice the difference, in real time, between consensus knowledge and the "expert's" own assertions. 

Part of the reason is that they're said in the same, authoritative tone, most of the time. Listen to baseball commentators. "Jeter is hitting .302." "Pitching is 75 percent of baseball." You really have to be paying attention to notice the difference. And, if you don't know baseball, you have no way of knowing that "75 percent of baseball" isn't established fact! At least, until you hear someone dispute it.

Also, I think we're just not used to the idea that "experts" are so often wrong. For our entire formal education, we absorb what they teach us about science as unquestionably true. Even though we understand, in theory, that knowledge comes from the scientific method ... well, in practice, we have found that knowledge comes from experts telling us things and punishing us for not absorbing them.  It's a hard habit to break.

------

The fact is: for every expert opinion, you can find an equal and opposite expert opinion. 

In that case, if you can't just assume someone's right just because he's an expert, can you maybe figure out who's right by *counting* experts?  

Maybe, but not necessarily. As Bill James wrote (9/8/14),


"An endless list of experts testifying to falsehood is no more impressive than one."

It used to be that an "endless list" of experts believed that W-L record was the best indication of a pitcher's performance. It used to be that almost all experts believed homosexuality was a disease. It used to be that almost no experts believed that gastritis was caused by bacteria -- until a dissenting researcher proved it by drinking a beaker of the offending strain. 

Each of those examples (they're mine, not Bill's) illustrates a different way experts can be wrong. 

In the first case, pitcher wins, the expert conventional wisdom never had any scientific basis -- it just evolved, somehow, and the "experts" resisted efforts to test it. 

In the second case, homosexuality, I suspect a big part of it was the experts interpreting the evidence to conform to their pre-existing bias, knowing that it would hurt their reputations to challenge it. 

In the third case ... that's just the scientific method working as promised. The existing hypothesis about gastritis was refuted by new evidence, so the experts changed their minds. 

Bill has a fourth case, the case of psychiatric "expert witnesses" who just made stuff up, and it was accepted because of their credentials. From "Hey Bill," 11/10/2011 and 11/11/2011:


"Whenever and wherever someone is convicted of a crime he did not commit, there's an expert witness in the middle of it, testifying to something that he doesn't actually know a damned thing about.  In the 1970s expert witnesses would testify to the insanity of anybody who could afford to pay them to do so."

"Expert witnesses are PRAISED by professional expert witnesses for the cleverness with which they discuss psychological concepts that simply don't exist."

In none of those cases would you have got the right answer by counting experts. (Well, maybe in the third case, if you counted after the evidence came out.)  

Actually, I'm cheating here. I haven't actually shown that the majority isn't USUALLY right. I've just shown that the majority isn't ALWAYS right. 

It's quite possible that those four cases were rare exceptions: that, most of the time, when the majority of experts agree, they're generally right. Actually, I think that's true, that the majority is usually right -- but I'm only willing to grant that for the "established knowledge" cases, the "distance from the earth to the sun" issues. 

For issues that are legitimately in dispute, does a majority matter?  And does the size matter?  Does a 80/20 split among experts really mean significantly more reliability than a 70/30 split?  

Maybe. But if you go by that, it's not *knowing*, right?  It's just handicapping. 

Suppose 70% of doctors believe X, and, if you look at all times that seventy percent of doctors believed something else, 9 out of 10 of those beliefs turned out to be true. In that case, you can't say, "you must trust the majority of experts."  You have to say, at best, "there's a 9 out of 10 chance that X is true."

But maybe I can say more, if I actually examine the arguments and evidence.

I can say, "well, I've examined the data, and I've looked at the studies, and I have to conclude that this is the 1 time out of 10 that the majority is dead wrong, and here is the evidence that shows why."  

And you have no reply to that, because you're just quoting odds.

And that's why evidence trumps experts. 

Here's Bill James on climate scientists, 9/9/2014 and 9/10/2014:


"[You should not believe climate scientists] because they are experts, no. You should believe them if they produce information or arguments that you find persuasive. But to believe them BECAUSE THEY ARE EXPERTS -- absolutely not.

"It isn't "consensus" that settles scientific disputes; it is clear and convincing evidence. An issue is settled in science when evidence is brought forward which is so clear and compelling that everyone who looks at the evidence comes to the same conclusion. ... The issue is NOT whether scientists agree; it is whether the evidence is compelling."

If you want to argue that something is true, you have two choices. You can argue from the evidence. Or, you can argue from the secondhand evidence of what the experts believe. 

But: the firsthand evidence ALWAYS trumps the secondhand evidence. Always. That's the basis of the entire scientific method, that new evidence can drive out an old theory, no matter how many experts and Popes believe they're wrong, and no matter how strongly they believe it.

You're talking to Bob, a "denier" who doesn't believe in climate change. You say to Bob, "how can you believe what you believe, when the scientists who study this stuff totally disagree with you?"

If Bob replies, "I have this one expert who says they're wrong" ... well, in that case, you have the stronger argument: you have, maybe, twenty opposing experts to his one. By Bob's own logic -- "trust experts" -- the probabilities must be on your side. You haven't proven climate change is real, but you've convincingly destroyed Bob's argument. 

However: if Bob replies, "I think your twenty experts are wrong, and here's my logic and evidence" -- well, in that case, you have to stop arguing. He's looking at firsthand evidence, and you're not. Your experts might still be right, because maybe he's got bad data, or he's misinterpreting his evidence, or his worthless logic comes out of the pages of the Miss America Pageant. Still, your argument has been rendered worthless because he's talking evidence, which you're not willing or able to look at directly.

As I wrote in 2010,


"Disbelieving solely because of experts is NOT the result of a fallacy. The fallacy only happens when you try to use the experts as evidence. Experts are a substitute for evidence. 

"You get your choice: experts or evidence. If you choose evidence, you can't cite the experts. If you choose experts, you can't claim to be impartially evaluating the evidence, at least that part of the evidence on which you're deferring to the experts. 

"The experts are your agents -- if you look to them, it's because you are trusting them to evaluate the evidence in your stead. You're saying, "you know, your UFO arguments are extraordinary and weird. They might be absolutely correct, because you might have extraordinary evidence that refutes everyone else. But I don't have the time or inclination to bother weighing the evidence. So I'm going to just defer to the scientists who *have* looked at the evidence and decided you're wrong. Work on convincing them, and maybe I'll follow."  

In other words: it's perfectly legitimate to believe in climate change because the scientific consensus is so strong. It is also legitimate to argue with people who haven't looked at the evidence and have no firsthand arguments. But it is NOT legitimate to argue with people who ARE arguing from the evidence, when you aren't. 

That they're arguing first-hand, and you're not, doesn't necessarily mean you're wrong. It  just means that you have no argument or evidence to bring to the table. And if you have no evidence in a scientific debate, you're not doing science, so you need to just ... well, at that point, you really need to just shut up.

The climate change debate is interesting that way, because, most of the activist non-scientists who believe it's real really haven't looked at the science enough to debate it. A large number have *no* firsthand arguments, except the number of scientists who believe it. 

As a result, it's kind of fun to watch their frustration. Someone comes up with a real argument about why the data doesn't show what the scientists think it does, and ... the activists can't really respond. Like me, most have no real understanding of the evidence whatsoever. They could say, like I do to the UFO people, "prove it to the scientists and then I'll listen," but they don't. (I suspect they think that sounds like they're taking the deniers seriously.)

So, they've taken to ridiculing and name-calling and attacking the deniers' motivations. 

To a certain extent, I can't blame them. I'm in the same situation when I read about Holocaust deniers. I mean the serious ones, the "expert" deniers, the ones who post blueprints of the death camps and prepare engineering and logistics arguments about how it wasn't possible to kill that many people in that short a time. And what can I do?  I let other expert historians argue their evidence (which fortunately, they do quite vigorously), and I gnash my teeth and maybe rant to my friends.

That's just the way it has to be. You want to argue, you have to argue the evidence. You don't bring a knife to a gunfight, and you don't bring an opinion poll to a scientific debate.

Labels: , , ,

Sunday, September 14, 2014

Income inequality and the Fed report

The New York Yankees are struggling. Why don't they sign Reggie Jackson? Sure, he's 68 years old, but he'd still be a productive hitter if the Yankees signed him today.

Why do I say that? Because if you look at the data, you'll see that players' production doesn't decline over time. In 1974, the Oakland A's hit .247. In 2013, they hit .254. Their hitting was just as good -- actually, better -- even thirty-nine years later!

So how can you argue that players don't age gracefully?

------

It's obvious what's wrong with that argument: the 2013 Oakland A's aren't the same players as the 1974 Oakland A's. The team got better, but the individual players got worse -- much, much worse. Comparing the two teams doesn't tell us anything at all about aging.

The problem is ridiculously easy to see here. But it's less obvious in most articles I've seen that discuss trends in income inequality, even though it's *exactly the same flaw*.

Recently, the US Federal Reserve ("The Fed") published their regular report on the country's income distribution (.pdf). Here's a New York Times article reporting on it, which says, 


"For the most affluent 10 percent of American families, average incomes rose by 10 percent from 2010 to 2013."

Well, that's not right. The Fed didn't actually study how family income changed over time. Instead, they looked at one random sample of families in 2010, and a *different* random sample of families in 2013.  

The confusion stems from how they gave the two groups the same name. Instead of "Oakland A's," they called them "Top 10 Percent". But those are different families in the two groups.

Take the top decile both years, and call it the "Washington R's." What the Fed report says is that the 2013 R's hit for an average 10 points higher than the 2010 R's. But that does NOT mean that the average 2010 R family gained 10 points. In fact, it's theoretically possible that the 2010 R's all got poorer, just like the 1974 Oakland A's all got worse. 

In one sense, the effect is stronger in the Fed survey than in MLB. If you're a .320 hitter who drops to .260 while playing for the A's, Billy Beane might still keep you on the team. But if you're a member of the 2010 R's, but wind up earning only an middle-class wage in 2013, the Fed *must* demote you to the minor-league M's, because you're not allowed to stay on the R's unless you're still top 10 percent. 

The Fed showed that the Rs, as a team, had a higher income in 2013 than 2010. The individual Rs? They might have improved, or they might have declined. There's no way of knowing from this data alone.

-----

So that quote from the New York Times is not justified. In fact, if even one family dropped out of the top decile from 2010 to 2013, you can prove, mathematically, that the statement must be false.

That has nothing to do with any other assumptions about wealth or inequality in general. It's true regardless, as a mathematical fact. 

Could it just be bad wording on the part of the Fed and the Times, that they understand this but just said it wrong? I don't think so. It sure seems like the Times writer believes the numbers apply to individuals. For instance, he also wrote, 


"There is growing evidence that inequality may be weighing on economic growth by keeping money disproportionately in the hands of those who already have so much they are less inclined to spend it."

The phrase "already have so much" implies the author thinks they're the same people, doesn't it? Change the context a bit. "Lottery winners picked up 10 percent higher jackpots in 2013 than 2010, keeping winnings disproportionately in the hands of those who already won so much."  

That would be an absurd thing to say for someone who realizes that the jackpot winners of 2013 are not necessarily the same people as the jackpot winners of 2010.

Anyway, I shouldn't fault the Times writer too much ... he's just accepting the incorrect statements he found in the Fed paper. 

And I don't think any of the misstatements are deliberate. I suspect that the Fed writers were sometimes careless in their phrasing, and sometimes genuinely thought that "team" declines/increases implied family declines/increases. 

Still, some of the statements, in both places, are clearly not justified by the data and should not have made it into print.

------

I've read articles in the past that made a similar point, that individuals and families might be improving significantly, even though the data appears to give the impression that their group is falling behind. 

It's not hard to think of an example of how that might be possible. 

Imagine that everyone gets richer every year. During the boom, immigration grows the population by 25 percent every year, and the new arrivals all start at $10 per hour.

What happens? 

(a) the lowest bottom 20 percent of every year earn the same amount; but 
(b) everyone gets richer every year

That is: *everyone* is better off *every year*, even though the data may make it falsely appear that the poor are stagnating.

(Note: the words "rich" and "poor" are defined as "high wealth" and "low wealth," but in this post, I'm also going to [mis]use them to mean "high income" and "low income."  It should be obvious from the context which one I mean.)

-------

Now, even if you agree with everything I've said so far, you could still have other reasons to be concerned about the Fed report. For me, the me, the most important fact is the discovery that 2013's poor (bottom quintile) have 8 percent less income than 2010's poor. 

You can't conclude that any particular family dropped, but you *can* conclude that, even if they're different people, the bottom families of 2013 are worse off than the bottom families of 2010. That's real, and that's something you could certainly be concerned about. 

But, many people, like the New York Times writer, aren't just concerned about the poorer families -- they worry about how "income inequality" compares them to the richer ones. They're uncomfortable with the growing distance between top and bottom, even in good times where the "rising tide" lifts everyone's income. For them, even if every individual is made better off, it's the inequality that bothers them, not the absolute levels of income, or even now fast overall income is growing. If the "Washington R's" gain 20 percent, but the "Oakland P's" gain only 5 percent ... for them, that's something to correct.

They might say something like,


"It's nice that the overall pie is growing, and it's nice that the "P's" are getting more money than they used to. But, still, every year, it seems like the high-income "team" is getting bigger increases than the low-income "team". There must be something wrong with a system where, years ago, the top-to-bottom ratio used to be 5-to-1, but now it's 10-to-1 or 15-to-1 or higher."

"Clearly, the rich are getting richer faster than the poor are getting richer. There must be something wrong with a system that benefits the rich so much while the poor don't keep up."

Rebutting that argument is the main point of this post. Here's what I'm going to try to convince you:

Even when the rich/poor ratio increases over time, that does NOT necessarily imply that the rich are getting more benefit than the poor. 

That is: *even if inequality is a bad thing*, it could still be that the changes in the income distribution have benefited the poor more than the rich.

I can even go further: even if ALL the benefits of increased income go to the poor, it's STILL possible for the rich/poor inequality gap to grow. The government could freeze the income of every worker in the top half, and increase the income of every worker in the bottom half. And even after that, the rich/poor income gap might still be *higher*.

-------

It seems that can't be possible. If everyone's income grows at the same rate, the ratio has to stay the same, right? If rich to poor is $200K / $20K one year, and rich and poor both double equally, you get $400K / $40K, and the ratio of 10:1 doesn't change. Mathematically, R/P has to equal xR/xP.

So if benefits that are equal keep the ratio equal, benefits that favor the poor have to change the ratio in favor of the poor. No? 

No, not necessarily. For instance:

Suppose that in 2017, the ratio between rich and poor is 1.25. In 2018, the ratio between rich and poor is 1.60. Pundits say, "this is because the system only benefited the rich!"

But it could be that the pundits have it 100% backwards, and the system actually only favored the poor. 

How? Here's one way. 

There are two groups, with equal numbers of people in each group. In 2017, everyone in the bottom group made $40K, and everyone in the top group made $50K. That's how the ratio between rich group and poor group was 1.25.

The government instituted a program to help the poor, the bottom group. Within a year, the income of the poor doubled, from $40K to $80K, while the top group stagnated at $50K. 

So, in 2018, the richest half of the population earned $80K, and the poorest half earned $50K. That's how inequality increased, from 1.25 to 1.60, only from helping the poor!

------

What happened? How did our intuition go wrong? For the same reason as before: we didn't immediately realize that the groups were different people in different years. The 2017 rich aren't the same as the 2018 rich.

When the pundits argued "the system only benefited the rich," whom did they mean? The "old" 2017 rich, or the "new" 2018 rich? Without specifying, the statement is ambiguous. So ambiguous, in fact, that it almost has no meaning.

What really happened is that the system benefited the old poor, who happen to be the new rich. It failed to benefit the old rich, which happen to be new poor.

Inequality increased from 1.25 to 1.60, but it's meaningless to say the increase benefited the "rich". Which rich? Obviously, it didn't benefit the "old rich."

But, isn't it true to say that the increase benefited the new rich? 

It's true, but it doesn't tell us much -- it's true by definition! In retrospect, ANY change will have benefited the "new rich" more than the "new poor."  If you used to be relatively poor, but now you're relatively rich, you must have benefited more than average. So when you say increasing inequality favors the "new rich," you're really saying "increasing inequality favors those who benefited the most from increasing inequality."  

These examples sound absurd, but they're exact illustrations of what's happening:

-- You have a program to help disadvantaged students go to medical school. Ten years later, you follow up, and they're all earning six-figure incomes as doctors. "Damn!" you say. "It turns out that in retrospect, we only helped the rich!"

-- Or, you do a study of people who won the lottery jackpot last year, and find that most of them are rich, in the top 5%. "Damn!" you say. "Lotteries are just a subsidy for the rich!"

-- Or, you do a study of people who were treated for cancer 10 years ago, and you find most of them are healthy. "Damn!" you say. We wasted cancer treatments on healthy patients!

It makes no sense at all to regret a sequence of events on the grounds that, in retrospect, it helped the people with better outcomes more than it helped the people with worse outcomes. Because, that's EVERY sequence of events!

If you want to complain that increasing inequality is disproportionately benefiting well-off people, that can make sense only if you mean it's those who were well off *before* the increase. But the Fed data doesn't give you any way of knowing whether that's true. It might be happening; it might not be happening. But the Fed data can't prove it either way.

----

Here's an example that's a little more realistic.

Suppose that in 2010, there are five income quintiles, where people earn $20K, $40K, $60K, $80K, and $100K, respectively. I'll call them "Poor," "Lower Class," "Middle Class," "Upper Class," and "Rich", for short. We'll measure inequality by the R/P ratio, which is 5 (100 divided by 20).

Using three representative people in each group, here's what the distribution looks like:


2010 group, 2010 income
------------------------
P    L    M    U    R
------------------------
20   40   60   80   100
20   40   60   80   100
20   40   60   80   100
------------------------
R/P ratio: 5


From 2010 to 2013, people's incomes change, for the usual reasons -- school, life events, luck, shocks to the economy, whatever. In each group, it turns out that one-third of people make double what they did before, one third experience no change, and one third see their incomes drop in half. 

Overall, that means incomes have grown by 16.7%: the average of +100%, 0%, and -50%. Workers have 1/6 more income, overall. But the change gets spread unevenly, since life is unpredictable.

Here are the 2013 incomes, but still based on the 2010 grouping. The top row are the people who dropped, the middle row are the status quo, and the bottom row are the ones who doubled.


2010 group, 2010 income
------------------------
P    L    M     U     R
------------------------
10   20   30    40    50
20   40   60    80   100
40   80  120   160   200
------------------------
R/P ratio: 5


You can easily calculate that every 2010 group got, on average, the same 16.7% increase. So, since life treated the groups equally, the 2010 rich/2010 poor ratio is still 5. In chart form:


2010 group, % change 2010-2013
------------------------------
 P     L     M     U     R  
------------------------------
+17%  +17%  +17%  +17%  +17%


But the Fed doesn't have any of those numbers, because it doesn't know which 2010 group the 2013 earners fell into. It just takes the 2013 data, and mixes it into brand new groups based on 2013 income:


2013 group, 2013 income
-------------------------
P    L     M     U     R  
-------------------------
10   30    40    80   120
20   40    50    80   160
20   40    60   100   200
-------------------------
R/P ratio: 9.6


What does the Fed find? Much more inequality in 2013 than in 2010. The ratio between rich and poor is 9.6 -- almost double what it was! 

The Fed method will also see that the bottom three groups are earning less than the corresponding group earned three years previous.  Only the top two groups, the "upper class" and "rich," are higher. Here are the changes between each new group and the corresponding old group:


Perceived change 2010-2013
--------------------------
 P    L    M    U    R  
--------------------------
-17% -8%  -8%  +8%  +60%


If you don't think about what's going on, you might be alarmed. You might conclude that none of the economy's growth benefited the lowest 60 percent at all -- that all the benefits accrued to the well off! 

But, that's not right: as we saw, the benefits accrued equally. And, as we saw, the "R" group ALWAYS has to be high, by definition, since it's selectively comprised of those who benefited the most!

In effect, comparing the 2010 sample to the 2013 sample is a subtle "cheat," creating an illusion that can be used (perhaps unwittingly) to falsely exaggerate the differences. When the poor improve their lot, the method moves them to another group, and winds up ignoring that they benefited. 

For instance, when a $30K earner moves to $90K, a $90K earner moves to $120K, and a $120K earner drops to $30K, the Fed method makes it look like they all benefited equally, at zero. In reality, the "poor" gained and the "rich" declined -- the $30K earner grew 200%, the $90K earner grew 33%, and the $120K earner dropped by -75%. 

No matter how you choose the numbers, as long as there is any movement between groups, the method will invariably overestimate how much the "rich" benefited, and underestimate how much the "poor" benefited. It never works the other way.

--------

One last example.

This time, let's institute a policy that does something special for the disadvantaged groups, to try to make society more equal. For everyone in the P and L group in 2010, we institute a program that will double their eventual 2013 income. Starting with the same 20/40/60/80/100 distribution for 2010, here's what we see after the 2013 doubling:


2010 group, 2013 income
-----------------------
P     L    M    U    R  
-----------------------
20    40   30   40   50
40    80   60   80  100
80   160  120  160  200
-----------------------
R/P ratio: 2.5


Based on the 2010 classes, we've cut the rich/poor ratio in half! But, as usual, the Fed doesn't know the 2010 classes, so they sort the data this way:


2013 group, 2013 income
-----------------------
P    L    M    U     R  
-----------------------
20    40  60    80  160
30    40  80   100  160
40    50  80   120  200
-----------------------
R/P ratio: 5.8


Inequality has jumped from 5.0 to 5.8. That's even after we made a very, very serious attempt to lower it, doubling the incomes of the previous poorest 40 percent of the population!

-------

There's an easy, obvious mathematical explanation of why this happens.

When you look at income inequality, you're basically looking at the variance of the income distribution. But, changes from year-to-year are not equal, so they have their own built-in variance.

If the changes in income are independent of where you started -- that is, if the system treats rich and poor equally, in terms of unpredictability -- then

var(next year) = var(this year) + var(changes)

Which means, as long as rich and poor are equal in how their incomes change, inequality HAS TO INCREASE. 

Take 100 people, start them with perfect equality, $1 each. 

Every day, they roll a pair of dice. They multiply their money by the amount of the roll, then divide by 7. 

Obviously, on Day 2, equality disappears: some people will have $12/7, while others will have only $2/7. The third day, they'll be even more unequal. The fourth day, even more so. Eventually, some of them will be filthy, filthy rich, having more money than exists on the planet, while others will have trillionths of a dollar, or less.

That's just the arithmetic of variation. Increasing inequality is what happens naturally, not just in incomes, but in everything -- everything where things change independently of each other and independently over time. 

What if you want to fight nature, and keep inequality from growing? You have to arrange for year-to-year changes to benefit the poor more than the rich. That effect has to be large -- as we saw earlier, doubling the income of the 40 poorest percent wasn't enough. (It was a contrived example, but, still, it sure *seemed* like it should have been enough!)

-----

How much do you have to tilt the playing field in favor of the poor? Thinking out loud, scrawling equations ... I didn't double-check, so try this yourself because I may have screwed up ... but here's what I got:

Without independence, 

var(next year) = var(this year) + var(changes) + 2 cov(this year, changes)

Solving on the back of my envelope ... if I've done it right, using logarithm of income and some rough assumptions ... I get that the correlation between this year's income and the change to next year's income has to be around -0.25.

My scrawls say that if you're in the top 2.5% of income, your next-year change has to be in the bottom 30%. And if you're in the bottom 2.5%, your next-year change has to be in the top 30%. 

That seems really tough to do. In a typical year that the economy grows normally, what percentage of incomes in the Fed survey would be lower than last year's? If it's 30 percent, then ... to keep inequality constant, just ONE of the things you need to do is make sure high-income people, on average, never earn more this year than last year.

You'd almost have to repeal compound interest!

------

I don't mean to imply that increasing inequality is *completely* just the result of normal variation. There are lots of other factors. Progressive taxation creates a small effect on equality. Increased savings while the economy grows contributes to inequality. A growing population means that inequality increases where bestselling authors have a larger market. And so on. 

But the point is: because increasing inequality happens naturally, you can't conclude anything just from *the fact that there's an increase*. At the very least, you have to back out the natural effects if you want to really explain what's going on. You have to do some math, and some arguing. 

The argument, "Inequality is growing -- therefore, we must be unfairly favoring the rich" is not a valid one. It is true that inequality is growing. And it *might* be true that we are unfairly favoring the rich. But, the one doesn't necessarily follow from the other. 

It's like saying, "Philadelphia was warmer in June than April; therefore, global warming must be happening."

------

Again, I'm not trying to argue that inequality is a good thing, or that you shouldn't be concerned about it. Rather, I'm arguing that increasing inequality does NOT tell you anything reliable about who benefits from the "system" or how much (if at all) the increase favors the rich over the poor.

I am arguing that, even if you think increasing inequality is a bad thing, the following are still, objectively, true:

-- increasing inequality is a natural mathematical consequence of variation;
-- it is not necessarily the result of any deliberate government policy;
-- it does not necessarily disproportionately favor the rich or hurt the poor;
-- there is no way to know which individuals it favors just from the Fed data;
-- the natural forces that cause inequality to increase are very strong;
-- natural inequality growth may be so strong that it will persist even after successful attempts to benefit the poor generously and significantly;
-- the poor could be gaining relative to the rich even while measured inequality increases.

As for the Fed study itself,

-- the Fed statistics do not measure income changes for any family or specific group of families;
-- the Fed statistics that measure distributional income changes for percentile groups are a biased, exaggerated estimate of the income changes for the average family starting in that percentile;
-- It is impossible to tell, from the Fed's numbers, how the poor are faring relative to the rich.

Finally, and most importantly,

-- all of these statements follow necessarily from basic logic and math -- and do not require any other arguments from politics, economics, compassion, greed, fairness, or partisanship.





Labels:

Saturday, August 30, 2014

Is MLB team payroll less important than it used to be?

As of August 26, about 130 games into the 2014 MLB season, the correlation between team payroll and wins is very low. So low, in fact, that *alphabetical order* predicts the standings better than salaries!

Credit that discovery to Brian MacPherson, writing for the Providence Journal. MacPherson calculated the payroll correlation to be +0.20, and alphabetical correlation to be +0.24. 

When I tried it, I got .2277 vs. .2346 -- closer, but alphabetical still wins. (I might be using slightly different payroll numbers, I used winning percentage instead of raw win totals, and I may have done mine a day or two later.)

The alphabetical regression is cute, but it's the payroll one that raises the important questions. Why is it so low, at .20 or .23? When Berri/Schmidt/Brook did it in "The Wages of Wins," they got around .40.

It turns out that the season correlation has trended over time, and MacPherson draws a nice graph of that, for 2000-2014. (I'll steal it for this post, but link it to the original article.)  Payroll became more important in the middle of last decade, but then dropped quickly, so that 2012, 2013, and 2014 are the lowest of all 15 years in the chart:






What's going on? Why has the correlation dropped so much?

MacPherson argues it's because it's getting harder and harder to buy wins. There is an "inability of rich teams to leverage their financial resources."  The end of the steroids era  means there are fewer productive free-agent players in their 30s for teams to buy. And the pool of available signings is reduced even further, because smaller-market teams can better afford to hang on to their young stars.


"Having money to spend remains better than not having money to spend. That might not ever change. Unfortunately for the Red Sox and their brethren, however, it matters far less than it once did."


------

My thoughts:

1.  The observed 2014 correlation is artificially low, because it's taken after only about 130 games (late-August), instead of a full season. 

Between now and October, you'd expect the SD due to luck to drop by about 12 percent. So, instead of 2 parts salary to 8 parts luck (for the current correlation of .20), you'll have 2 parts salary to 7.2 parts luck. That will raise the correlation to about .22.

Well, maybe not quite. The non-salary part isn't all binomial luck; there's some other things there too, like the distribution of over- and underpriced talent. But I think .22 is still a reasonable projection.

It's a small thing, but it does explain a tenth of the discrepancy.

------

2.  The lower correlation doesn't necessarily mean that it's harder to buy wins. As MacPherson notes, It could just mean that teams are choosing not to do so. More specifically, that teams are closer in spending than they used to be, so payroll doesn't explain wins as well as it used to.

Here's an analogy I used before: in Rotisserie League Baseball, there is a $260 salary cap. If everyone spends between $255 and $260, the correlation between salary and performance will be almost zero -- the $5 isn't enough of a signal amidst the noise. But: if you let half the teams spend $520 instead, you're going to get a much higher correlation, because the high-spending half will do much, much better than the lower-spending half.

That could explain what's happening here.

In 2006, the SD of payroll was around 42% of the mean ($32MM, $78MM). In 2014, it was only 38% ($43MM, $115MM). It doesn't look that much different, but ... teams this year are 10 percent closer to each other than they were, that has to be contributing to the difference.

(This is the first time I've done something where "coefficient of variation" (the SD divided by the mean) helped me, here as a way to correct SDs for inflation.

Also, this is a rare (for me) case where the correlation (or r-squared) is actually more relevant than the coefficient of the regression equation. That's because we're debating how much salary explains what we've actually observed -- instead of the usual question of much salary leads to how many more wins.)


------

3.  While doing these calculations, I noticed something unusual. The 2014 standings are much tighter than normal. 

So far in 2014, the SD of team winning percentage is .058 (9.4 games per 162). In 2006, the SD was larger, at .075 (12.2 games per 162). That might be a bit high ... I think .068 (11 games per 162) is the recent historical average.

But even 9.4 compared to 11 is a big difference.  It's even more significant when you remember that the 2014 figure is based on only 130 games. (I'd bet the historical average for late-August would be between 12 and 13 games, not 11.)

What's going on? 

Well, it could be random luck. But, it could be real. It could be that team talent "inequality" has narrowed -- either because of the narrowing of team spending (which we noted), or because all the extra spending isn't buying much talent these days.

I think the surrounding evidence shows that it's more likely to be random luck. 

Last year, the SD of team winning percentage was at normal levels -- .074 (12.04 games per 162). It's virtually impossible for the true payroll/wins relationship to have changed so drastically in the off-season, considering the vast majority of payrolls and players stay the same from year to year.

Also, it turns out that even though the correlation between 2014 payroll and 2014 wins is low, the correlation between 2014 payroll and 2013 wins is higher. That is: this year's payroll predicts last year's wins (0.37) better than it predicts this year's wins (0.23)! 

Are there other explanations than 2014 being randomly weird? 

Maybe the low-payroll teams have young players who improved since last year, and the high-payroll teams have old players who declined. You could test that: you could check if payroll correlates better to last year's wins than this year's for all seasons, not just 2013-2014.

If that happened to be true, though, it would partially contradict MacPherson's hypothesis, wouldn't it? It would say that the money teams spend on contracts *do* buy wins as strongly as before, but those wins are front-loaded relative to payroll.

We can see how weird 2014 really is if we back out the luck variance to get an estimate of the talent variance.

After the first 130 games of 2014, the observed SD of winning percentage is .058. After 130 games, the theoretical SD of winning percentage due to luck is .044.

Since luck is generally independent of talent, we know

SD(observed)^2 - SD(luck)^2 = SD(talent)^2 

Plugging in the numbers: .058 squared minus .044 squared equals .038 squared. That gives us an estimate of SD(talent) of .038, or 6.12 games per 162.

I did the same calculation for 2013, and got 10.2.

2013: Talent SD of 10.2 games
2014: Talent SD of  6.1 games

That kind of drop in one off-season pretty much impossible, isn't it? 

If that kind huge a compression were real, it would have to be due to huge changes in the off-season -- specifically, a lot of good players retiring, or moving from good teams to bad teams.

But, the team correlation between 2013 wins and 2014 wins is +0.37. That's a bit lower than average, but not out of line (again, especially taking the short season into account). 

It would be very, very coincidental if the good teams got that much worse while the bad teams got that much better, but the *order* of the standings didn't change any more than normal.

So, I think a reasonable conclusion is that it's just random noise that compressed the standings. This year, for no reason, the the good teams have tended to be unlucky while the bad teams have tended to be lucky. And that narrowed the distance between the high-payroll teams and the low-payroll teams, which is part of the reason the payroll/wins correlation is so low. 

------

4. We can just look at the randomness directly, since the regression software gives us confidence intervals. 

Actually, it only gives an interval for the coefficient, but that's good enough. I added 2 SDs to the observed value, and then worked backwards to figure out what the correlation would be in that case. It came out to 0.60. 

That's huge!  The confidence interval actually encompasses every season on the graph, even though 2014 is the lowest of all of them.

To confirm the 0.60 number, I used this online calculator. If the true correlation for the 30 teams is 0.4, the 95% confidence interval goes up to 0.66, and down to 0.05. That's close to my calculation for the high end, and easily captures the observed value of 0.23 in its low end. 

That's not to say that I think they really ARE all the same, that the differences are just random -- I've never been a big fan of throwing away differences just because they don't meet significance thresholds. I'm just trying to show how easy it is that it *could be* random noise.

I can try to rephrase the confidence interval argument visually. Here's the actual plot for the 2014 teams:




The correlation coefficient is a rough visual measure of how closely the dots adhere to the green regression line. In this case, not that great; it's more a cloud than a line. That's why the correlation is only 0.23.

Now, take a look at the teams between $77 million and $113 million, the ones in the second rectangle from the left.

There are eighteen teams in that group bunched into that small horizontal space, a payroll range of only $46 million in spending. Even at the historically high correlations we saw last decade, and even if the entire difference was due to discretionary free-agent spending, the true talent difference in that range would be only about 3 or 4 games in the standings. That would be much smaller than the effects of random chance, which would be around 12 games between luckiest and unluckiest. 

What that means is:  no matter what happens, that second vertical block is dominated by randomness, and so the dots in that rectangle are pretty much assured of looking like a random cloud, centered around .500. (In fact, for this particular case, the correlation for that second block is almost perfectly random, at -.002.)

So those 18 teams don't help much. How much the overall curve looks like a straight line is going to depend almost completely on the remaining 12 points, the high-spending and low-spending teams. In our case, the two low-spending teams are somewhat worse than the cloud, and the ten high-spending teams are somewhat better than the cloud, so we get our positive correlation of +0.23. 

But, you can see, those two bad teams aren't *that* bad. In fact, the Marlins, despite the second-lowest payroll in MLB, are playing .496 ball.

What if we move the Marlins down to .400? If you imagine taking that one dot, and moving it close to the bottom of the graph, you'll immediately see that the dots would get a bit more linear. (The line would get steeper, too, but steepness represents the regression coefficient, not the correlation, so never mind.)  I made that one change, and the correlation went all the way up to 0.3. 

Let's now take the second-highest-payroll Yankees, and move them from their disappointing  .523 to match the highest-payroll Dodgers, at .564. Again, you can see the graph will get more linear. That brings the correlation up to 0.34 -- almost exactly the average season, after mentally adjusting it a bit higher for 162 games.

Of course, the Marlins *aren't* at .400, and the Yankees *aren't* at .564, so the lower correlation of 0.23 actually stands. But my point is not to argue that it should actually be higher -- my point is that it only takes a bit of randomness to do the trick. 

All I did was move the Marlins down by less than 2 SDs worth of luck, and the Yankees by less than 1 SD worth of luck. And that was enough to bump the correlation from historically low, to historically average.

------

5. Finally: suppose the change isn't just random luck, that there's actually something real going on. What could it be?

-- Maybe money doesn't matter as much any more because low-spending teams are getting more of their value from arbs and slaves. They could be doing that so well that the high-spending teams are forced to spend more on free agents just to catch up. It wouldn't be too hard to check that empirically, just by looking at rosters.

-- It could be that, as MacPherson believes, there are fewer productive free agents to be bought. You couuld check that easily, too: just count how many free agents there are on team rosters now, as compared to, say, 2005. If MacPherson is correct, that careers are ending after fewer years of free agency, that should show up pretty easily.

-- Maybe teams just aren't as smart as they used to be about paying for free agents. Maybe their talent evaluation isn't that great, and they're getting less value for their money. Again, you could check that, by looking at free-agent WAR, or expected WAR, and comparing it to contract value.

-- Maybe teams don't vary as much as they used to, in terms of how many free-agent wins they buy. I shouldn't say "maybe" -- as we saw, the SD of payroll, adjusted for inflation, is indeed lower in 2014 than it was in 2006, by about 10 percent. So that would almost certainly be part of the answer. 

-- More specifically: maybe the (otherwise) bad teams *more* likely to buy free agents than before, and the (otherwise) good teams are *less* likely to buy free agents than before. That actually should be expected, if teams are rational. With more teams qualifying for the post-season, there's less point making yourself into a 98-win team when a 93-win team will probably be good enough. And, even an average team has a shot at a wild card, if they get lucky, so why not spend a few bucks to raise your talent from 79 games to (say) 83 games, like maybe the Blue Jays did last year?

-----

I'll give you my gut feeling, but, first a disclaimer: I haven't really thought a whole lot about this, and some of these argument occurred to me as I wrote. So, keep in mind that I'm really just thinking out loud.

On that basis, my best guess is ... that most of the correlation drop is just random noise. 

I'd bet that money buys free agents just as reliably as always, and at the usual price. The correlation is down not because spending buys fewer wins, but because more equal spending makes it harder for the regression to notice the differences.

But I'm thinking that part of the drop might really be the changing patterns of team spending, as MacPherson described. I wonder if that knot of 18 mid-range teams, clustered in such a small payroll range, might be a permanent phenomenon, resulting from more small-market teams moving up the payroll chart after deciding their sweet spot should be a little more extravagant than in the past. 

Because, these days, it doesn't take much to almost guarantee a team a reasonable shot at a wildcard spot -- which means, meaningful games later in the season than before, which means more revenue. 

In fact, that's one area where it's not zero-sum among teams. If most of the fan fulfillment comes from being in the race and having hope, any team can enter the fray without detracting much from the others. What's more exciting for fans -- being four games out of a wildcard spot alone, or being four games out of a wildcard spot along with three other teams? It's probably about the same, right? 

Which makes me now think, the price of a free agent win could indeed change. By how much? It depends on how increased demand from the small market teams compares to decreased demand from the bigger-spending teams.

------

Anyway, bottom line: if I had to guess the reasons for the lower correlation:

-- 80% randomness
-- 20% spending patterns

But you can get better estimates with some research, by checking all those things I mentioned, and any others you might think of.





Hat Tip: Craig Calcaterra


Labels: , , ,