Is racism getting worse? Show us some data!

Seventeen months ago, in the light of several high-profile shootings, the Government of Ontario commissioned a study on young people and crime. The study is now out. Authored by two high-profile former politicians, Roy McMurtry and Alvin Curling, it's called "Roots of Youth Violence," came in at 500 pages and cost some $2,000,000.

Before I start, I should say that I haven't read the actual study; I'm going by news reports such as these two. But if those articles are correct, the report is badly flawed.

The part of the paper that got most of the publicity is its finding that racism in Ontario is getting worse – that we Ontarians are more prejudiced than ever. The money quote seems to be

"Racism is worse than it was a generation ago."

The authors concluded that youth violence is caused by poverty, racism, and untreated mental health problems. But the articles I've read mention that only in passing – their main focus is on the "racism is worse" claim.

Now, it seems to me that any quantitiative comparison between racism now and racism 30 years ago is a question of fact. That means that you're pretty much expected to

-- define racism;
-- show how you'll measure it;
-- measure it now;
-- measure it then;
-- and show how the two measurements compare.

I don't think that's just me – that's how we do it in sabermetrics, and I'm sure that's how they do that in academia. Your conclusions have to be supported by data and logic.

Of course, there will always be differences of opinion about what racism is and how best way to measure it. No matter how the report does it, there'll probably be room for reasonable people to disagree, and dispute the findings.

But, as far as I can tell, the authors didn't measure racism at all. They seem to have made absolutely no effort to quantify anything. If the news reports are accurately portraying the study, the results were pulled out of thin air.

To quote Dan Gardner, from the first article,

"... extraordinary claims require extraordinary evidence. ... So what evidence did McMurtry and Curling provide?"

Apparently just anecdote:

"... they talked to people who told, them, presumably, of their experiences and perceptions."

As Gardner points out, it's always good to hear what people have to say. But anecdote isn't evidence. More importantly, you can't compare levels of racism unless you count or measure something. And it doesn't seem like the authors did any of that. At best, they reference existing studies and random quotes.

For instance, they reference a study of university students in Peterborough (Ont.) that found

"85% of respondents reported racism in public places ... the same precentage of them experienced racism in downtown bars. Around 75% of them reported racism in university residences."

"Of all the public/private spaces studied, none was found to be racism-free," the report concluded.

Even if all this is absolutely true (although, as the newspaper article notes, the numbers are "off-the-charts astounding"), what does it really tell us? Not a whole lot. Suppose we take the findings at face value. What does it mean if 85% of people found racism in public places? Well, it depends, doesn't it, on how much interaction you have with the outside world. If everyone encounters 1900 unique other people in public, and only one-tenth of 1% of people are racist, you'll get exactly the reported number – 85% of people will have encountered at least one racist person (1 minus .999 to the power of 1900). What if they encounter only 500 people? Then it takes only about 0.04% of the population to be racist.

(Notice that even with the 85% figure, the actual number of racists can be pretty low. There are hundreds of bad things in life that have happened to 85% of us, but are nonetheless not common enough to panic about.)

So how racist is Peterborough? If you're a visible minority student responding to the survey, how much do you get out? Because it makes a huge difference – in my example, racism could be four times as high in one calculation as the other.

Needless to say, the study doesn't appear to address this issue. Nor does it talk about how the current 85% compares to whatever the number might have been back in the 70s. Small-town Ontario might be different from, say, the US South, but I'd bet you that as bad (or not bad) as racism might be today, I'd guarantee you that thirty years ago in Mississippi, the proportion of blacks reporting racism in public would be a lot higher than 85%.

Another part of the report talks about how visible minority persons in Ontario are much more likely to be lower-income than whites. Now, I knew that was the case, and I bet you did too. The question is, can you simply assume that the cause is racism? One alternative, and completely plausible explanation (as Gardner points out) is that ethnic minorities are more likely to be recent immigrants, and recent immigrants tend to be poor.

There are other possible explanations, and you've probably seen those too. There is an extensive literature, and debate, on this topic. But apparently the authors don't consider that at all! Which is absolutely ridiculous, to wade into an ongoing debate and support the most naive, simplistic view without even acknowledging the existence of any other work on the subject.

It's like ... it's like hiring someone to investigate whether smoking causes cancer, and he says, "well, my granddaddy smoked, and he lived to be 100, so smoking is fine." The problem is not that you're wrong – the problem is that the taxpayers funded you for a year and a half so you could investigate an important public-policy issue, and you didn't even bother to do any research or question your own opinions. And, indeed, the report not only doesn’t appear to have any of this kind of analysis, but it even draws quantitative conclusions in the utter absence of quantitative data!

Shouldn't a government-funded, heavily-publicized study have to undergo at least as much peer review as an obscure academic paper?