Luck, careers, and income inequality

According to this (anonymous) blog post from "The Economist," increased income inequality is partly due to random chance becoming more important in picking winners and losers.  

The post (and accompanying article) cites a recent paper on "herd effects," where a song or book becomes popular kind of randomly, because people tend to flock to whatever other people have already flocked to.  So, you can have two songs of equal merit, but one is a smash hit just because, for random reasons, its early adopters had better social networks.

Furthermore, there's experimental evidence.  In a recent academic study, researchers sent job applications to two different employers.  The resumes were identical except that, in one of them, the applicant said he had been unemployed for several years, while in the other, the applicant said he had a current job.  

Overall, the "out of work" applicants got around half the callbacks of the currently employed applicants.  Presumably, the employers believed that those who were out of work were less likely to be suitable candidates than those who weren't.  

The author writes,

" ... the implication is troubling: someone who ends up unemployed through bad luck, and for some idiosyncratic reason doesn't quickly land a job, finds his chances of reemployment diminish until he’s part of the long-term unemployed. ... 

"... this research opens a new and troubling dimension on  inequality. Unlike deficiencies of skill, it’s hard to tell society's  losers they should go back to school to become luckier. "

------

Certainly, there a huge, huge amount of luck involved in career success. There has to be: there's already luck in hockey, golf, and chess, and real life is infinitely more random and complicated than those.

For starters, there's when and where you were born, what kind of schools you went to, what books you happened to read, what kind of influences you had from your friends and parents, how good your teachers were in various subjects, chance remarks from strangers about what careers are likely to be lucrative, and many more.

Even in the more concrete sense, there are many, many small random things that can make a big difference.  What company you choose to join for your first job, what your first manager thinks of you, how your bosses think of your performance amidst the noise of team accomplishments, whether your co-workers give you credit or blame behind your back ... and you can probably think of hundreds more.

Basically, where you wind up in your life, in your career or any other sense, is overwhelmingly the result of so many random events that it's basically ... well, it's kind of a miracle that we are where we are now instead of somewhere else.  If we had to live our lives over again, from any given point, we'd probably be in different jobs, doing different work, with a different spouse, maybe living in a different city, earning a different income.  

In that light, I find it a bit perplexing that the authors chose these two specific examples as evidence of how luck factors into it.  They seem pretty minor, don't they?  I mean, the unemployed applicants still got half as many calls as the employed ones ... that seems like a pretty small career stopper as opposed to, say, if their first manager didn't like them and put them into a dead-end position and never promoted them, among hundreds of other possible events.

As for the "herding" argument ... wouldn't it be the case that there's LESS herding luck now, with the internet?  It used to be that you only knew songs you heard on the radio.  Now, you can find whatever you want on YouTube, on demand. There's more opportunity to find a niche.  Isn't that established wisdom?  Isn't that what "The Long Tail" is about, that, these days, there are many different markets that never existed before?

In terms of "herding," I'd bet that luck is much LESS a factor than it used to be.  

-------

Also: as much as luck is a huge, huge part of where any individual winds up, I don't think it's a big factor in the overall distribution.

In baseball, luck is, what, around 30 percent of the variation in team W-L percentage?  

Now, suppose you expanded the major leagues to include 30 little league teams.   Now, luck is a much smaller factor, isn't it?  The "real" teams always beat the kids, so, now, whether or not you're over .500 is completely a matter of talent.  Luck is still important within your group, but, overall, I'd bet the r-squared drops from 30 percent to ... I dunno, maybe 5 percent. 

Something similar happens with careers.  Two accountants, Andy and Beth, graduate from the same school with the same talent.  Andy winds up in a job where the culture matches his talents, while Beth does not.  Just by that random happening, Andy eventually winds up a CFO making $1 million a year, and Beth winds up in some middle-manager type position, making $70,000.  

Yes, Andy outearns Beth because of a lucky break.  But, the difference is contained within the accountants' income distribution, in the same way luck in our expanded MLB is contained within the teams in a single age level.

Luck matters more, obviously, when the differences in talent are small.  Within the subgroup of accountants, luck is going to matter a lot.  Within the subgroup of burger-flippers, luck is going to matter a lot.  But within the COMBINED group of accountants and burger-flippers, luck isn't that big a deal. You're not going to have many burger flippers, with average talent, getting lucky and outearning a typical accountant.

You can probably see this for yourself with a little introspection.  For my own part, I can see my career, and life, turning out radically differently based on a few key random events and decisions.  But ... I don't think I'd be poor.  I don't think I'd be making minimum wage.  I like programming, and I'd find a programming job somewhere, and even with the worst luck, I'd be making a decent living.  

Also, I don't think I'd be ultra-rich.  Even with extreme good luck -- say, 3 or 4 SDs above average -- I doubt I'd be a charismatic, celebrated CEO of a big company.  That requires skills and personality traits that I just don't have.

There's a lot of luck in individual baseball batting performance, but that doesn't mean you'll see Ozzie Smith hitting 60 home runs.  And there's a lot of luck in career outcomes too, but that doesn't mean you'll see lots of Bill Gateses working minimum wage.

At least, not because of the kind of luck the authors are talking about here.  

------

But, fundamentally ...  I wonder about the entire premise of the argument, that the prevalence of luck causes inequality to increase.  I wonder if it's the other way around.

A "system that rewards ability", as the author puts it, is an important and desirable goal.  A job, or promotion, or salary, or other reward, should go to the person who does the job best, not the one with the right resume, or the right skin color, or the right DNA.

Nor, by extension, to the one who happened to obtain the best roll of the dice.  

But, why does it follow that a system that rewards ability must also be more equal?  I think that's the wrong conclusion. It's understandable ... we see that some people get rewarded for being lucky, and we think, well, if only those people didn't get to be so rich, randomly, we'd all be more equal!  

But, I think that's backwards, at least in the hiring context. The more merit matters, the *bigger* the differences in salary.  Because, an employer is only going to pay you the expected value of your production.  

Let's use a simplified sports analogy.  

A team is trying to sign players, who are $100 on average ... but, some are worth more, perhaps much more, and some less.  

Suppose you have no scouts.  So, you have no idea of the players' relative talent (assume, also, that neither do the other teams, or the players themselves).  

That means you have to hire randomly.  What are you going to pay?  Obviously, $100 per player.  You have complete income equality among your players, even though the guys on the team got there by 100% luck.

Later, you hire some mediocre scouts.  You're able to tell the better players from the worse ones, to a certain extent, based on "whether they look like ballplayers".  So, you hire some at $130, and some at $70.   Less equality.

Finally, you hire some sabermetricians and some expert scouts, and now you can estimate each player's value within $10.  You hire the superstar at $400, and a mediocre mop-up man at $5.  Even less equality.

Isn't that a better description of how real life works?  You can't be a well-paid employee unless you can prove to an employer that you're worth it.  The more merit-based the system, the better chance you have of getting full value for what you're worth.

If you don't like that analogy ... try this.  There are a hundred brand new Honda Civics on the lot, and they all cost the same.  But, now, suppose you have more information about their merit -- somehow, it becomes possible to predict reliability.  That blue Civic over there will run trouble-free for 20 years. But that red one ... the transmission is going to blow at 80,000 miles.  

Now, the top 1% of Civics will sell for a lot more than the bottom 1%.  Right?

-------

In general: the closer you look for merit, the more differences you find.  The more differences you find, the wider the distribution of the sum of those differences.   So, the more you hire by merit, the more spread you get in salaries.

Is that a problem?  Not for me.  I'm strongly in favor of hiring by merit ... and I don't think income inequality is necessarily always a bad thing.  So, to me, there's no serious trade-off. But, if, you like both merit and equality, I think you have to choose which is the lesser evil, in this context.

As a society, I think we've made our consensus trade-off ... because, there are many, many unspoken ways in which we deliberately ignore merit.  For instance, we pay our teachers by seniority and experience, rather than by how good teachers they are.  We try to avoid situations where long-term employees get demoted or has their pay cut, even when their performance drops.  We discourage comparisons of which employees are better workers than their colleagues in the same position.

We give lip service to the idea that we should be paid and rewarded by merit, but we don't actually believe it as much as we say we do.  We claim we can't have nepotism because then we're being unfair to candidates with more talent, but then we believe we CAN favor older, worse teachers, without being unfair to younger teachers with more merit.  

Like a lot of other things in our society, we start with the outcome we want and work backwards to rationalize it.  We hate nepotism, and we hate firing mediocre teachers.  Merit is just an incidental consideration.