Racial bias and baseball card values

Recently, I found out about "Econ Journal Watch," a journal and website that debunks bad papers.  According to its website, EJW

"watches the journals for inappropriate assumptions, weak chains of argument, phony claims of relevance, and omissions of pertinent truths."

Most excellent!  And, this issue, there's a sports article, critiquing and extending a study that searched for racial bias in baseball by looking at baseball card values.

In 2005, four researchers -- John D. Hewitt, Robert Munoz Jr., William L. Oliver, and Robert M. Regoli [call them HMOR] -- published a study in "Journal of Sport and Social Issues" that looked at the rookie card values of 51 Hall of Famers.  They ran a regression to predict card price based on the player's statistics, race, and scarcity of the card.  They found no significant effect for race.

Now, David W. Findlay and John M. Santos reviewed the HMOR study, and tried to reproduce it.  They found a few problems.

-- first, there were some problems with some of the data being wrong -- probably transcription errors.  They fixed those.

-- then, they found that the original authors had taken some of their stats from one edition of Total Baseball, and some from another edition (in which the formulas had changed).  They corrected that, too.

-- third, they found that, for five of the players, HMOR had used career numbers from the 1989 edition of Total Baseball -- even though the players were still active at that time!

-- fourth, they noticed that the authors used scarcity numbers from PSA Authenticators, but card values from Beckett.  When they substituted PSA values instead, they got a much better fit.

-- fifth, they criticize the authors for omitting Hispanic players from the sample (I don’t agree that this one is a problem).

After all that, they reproduced the analysis, and still found that there was no significant effect for race.  However, to their credit, they write,

“Although our results indicate that player race has no statistically significant effect on baseball card prices, we are mindful of Ziliak and McCloskey (2004, 334) who note that "statistical significance, to put it shortly, is neither necessary nor sufficient for a finding to be economically important." The estimated coefficient on the Black dummy variable indicates that the price of a black player’s rookie card, all else fixed, is 9.3% lower than that of an otherwise identical white player."

------

Excellent stuff ... “Econ Journal Watch” is providing an extremely valuable function, giving authors a place to publish their critiques, and thus creating an incentive to do this kind of checking.  Findlay and Santos write that they submitted their article to the journal that published the original, but it was rejected as “not a good fit.”  One suspects that if EJW hadn’t existed as a backup, they wouldn’t have bothered to investigate in the first place.

So, kudos to EJW, and to Findlay and Santos.

------

While data errors are indeed a big concern -- especially the ones resulting from truncated careers -- I think there are problems with the study that are far more serious.  Those, however, are more in the line of “subject matter” issues.  Even with the data corrected by Findlay and Santos, the flaws are so large that I don’t think the study means anything. 

-----

1.  For their measure of player performance, the authors used Pete Palmer’s “Total Player Rating” and “Total Pitcher Index”.  Those are denominated in Runs Above Average.  Do we really value a player only by his career runs?  The various Hall of Fame methods, such as Bill James’, all recognize that there are other factors that influence Hall of Fame voting, such as pitcher wins, times leading the league, times hitting .300, and so on.  Wouldn’t it be expected that collector popularity be similar? 

In fairness, the authors did try to improve on the measurement by trying average runs per season, instead of career total runs.  They found little difference.  

Still, that’s not enough.  It implies that Lou Brock should be only as esteemed as any other player producing +10.5 runs over a long career.  That ignores that there are very good reasons that Brock is in the Hall, whereas most other players at +10.5 are not.

If white players tend to create runs in ways that are more valued than ways that black players create runs, that would create a false perception of racial bias in favor of whites -- and vice-versa.  Even more important: the study is very small, with only 51 players (and only 2 black pitchers!).  Even if blacks and whites are the same, it’s very possible that just by random chance, the whites in this study just happened to create runs in more popular ways.  Over several hundred players, you might be able to assume that the effects would even out.  But not with 51 players.

2.  For card scarcity, the authors used data provided by Professional Sports Authenticator (PSA), a company that grades, authenticates, and slabs cards.  

The company provides a "Population Report", listing the number of each card graded by PSA.  But PSA doesn’t grade cards randomly -- it grades them at the owner’s request and expense.  It stands to reason that owners will submit more valuable cards much more often than less valuable cards.  That will tend to understate the actual scarcity.

To take an extreme example: from the 1960 Topps set, there were 187,192 cards graded.  From the 1988 Topps set, there were only 8,043.  But, of course, there were many, many more cards printed in 1988 than 1960 -- from these estimates, by a factor of perhaps 100 (and even that seems a bit low to me).  People just don’t get 1988 Topps cards graded -- because the cards are worth a penny, and grading costs $10 or $20 or more.

Using the PSA numbers conflates two conflicting effects: scarce cards are graded less often.  But scarce cards are expensive, and expensive cards are graded *more* often.  There’s no obvious way to figure out how to break down the two effects.

And, this creates a very strong bias.  There were more white superstars than black superstars in the 50s.  But the model underestimates the scarcity of their rookie cards.  Therefore, the model predicts a lower price, which can be misread as a racial preference for whites.

3.  The only two factors the studies considered were performance and scarcity.  But there are obviously other important reasons that a player may be more popular than another.  For instance: team.  It goes without saying, doesn’t it, that a New York Yankee superstar should be more popular, than, say, a Minnesota Twins superstar with the same stats?

If the Yankees were less likely to have black superstars than other teams, that would account for some of the difference.  If the Yankees were more likely to have white superstars with low print runs but high grading numbers -- say, Mickey Mantle -- that would cause the model to doubly underestimate what the value of the card should be.

4.  There are many other factors that influence popularity, that are specific to the particular player.  Mark Fidrych and Kerry Wood, for instance, are loved for reasons other than their career totals.  We Blue Jays fans have a bigger soft spot for Ernie Whitt than for George Bell, for reasons that (I would argue) are related more to personality than race.

You’d also think that players who spent their career with one team would have different fan bases than players whose careers spanned multiple teams.  Carl Yastrzemski, for instance, had lots of seasons to make Red Sox fans love him, and the fact that he played his entire career there makes fans love him more.  On the other hand, Dave Winfield -- the most similar player to Yastrzemski -- left strong memories in at least four different places. 

What’s more important for popularity: having lots of short-term fans in different cities, or having long-term fans in one city? 

I don't know.  But the answer matters.  And it won't necessarily even out in a sample of only 51 players.

------

Anyway, I could probably go on ... the point is, that any one of these four factors could significantly affect the findings of this study.  All four, taken together ... well, I don’t think the results tell us anything at all about race affects card prices -- or even about how performance affects card prices, or how scarcity affects card prices.

Yes, the authors screwed up the data a little bit, but ... well, that’s by far the least of this study’s problems.


Hat Tip: Marginal Revolution