Tuesday, November 25, 2008

Is racism getting worse? Show us some data!

Seventeen months ago, in the light of several high-profile shootings, the Government of Ontario commissioned a study on young people and crime. The study is now out. Authored by two high-profile former politicians, Roy McMurtry and Alvin Curling, it's called "Roots of Youth Violence," came in at 500 pages and cost some $2,000,000.

Before I start, I should say that I haven't read the actual study; I'm going by news reports such as these two. But if those articles are correct, the report is badly flawed.

The part of the paper that got most of the publicity is its finding that racism in Ontario is getting worse – that we Ontarians are more prejudiced than ever. The money quote seems to be

"Racism is worse than it was a generation ago."

The authors concluded that youth violence is caused by poverty, racism, and untreated mental health problems. But the articles I've read mention that only in passing – their main focus is on the "racism is worse" claim.

Now, it seems to me that any quantitiative comparison between racism now and racism 30 years ago is a question of fact. That means that you're pretty much expected to

-- define racism;
-- show how you'll measure it;
-- measure it now;
-- measure it then;
-- and show how the two measurements compare.

I don't think that's just me – that's how we do it in sabermetrics, and I'm sure that's how they do that in academia. Your conclusions have to be supported by data and logic.

Of course, there will always be differences of opinion about what racism is and how best way to measure it. No matter how the report does it, there'll probably be room for reasonable people to disagree, and dispute the findings.

But, as far as I can tell, the authors didn't measure racism at all. They seem to have made absolutely no effort to quantify anything. If the news reports are accurately portraying the study, the results were pulled out of thin air.

To quote Dan Gardner, from the first article,

"... extraordinary claims require extraordinary evidence. ... So what evidence did McMurtry and Curling provide?"

Apparently just anecdote:

"... they talked to people who told, them, presumably, of their experiences and perceptions."

As Gardner points out, it's always good to hear what people have to say. But anecdote isn't evidence. More importantly, you can't compare levels of racism unless you count or measure something. And it doesn't seem like the authors did any of that. At best, they reference existing studies and random quotes.

For instance, they reference a study of university students in Peterborough (Ont.) that found

"85% of respondents reported racism in public places ... the same precentage of them experienced racism in downtown bars. Around 75% of them reported racism in university residences."

"Of all the public/private spaces studied, none was found to be racism-free," the report concluded.

Even if all this is absolutely true (although, as the newspaper article notes, the numbers are "off-the-charts astounding"), what does it really tell us? Not a whole lot. Suppose we take the findings at face value. What does it mean if 85% of people found racism in public places? Well, it depends, doesn't it, on how much interaction you have with the outside world. If everyone encounters 1900 unique other people in public, and only one-tenth of 1% of people are racist, you'll get exactly the reported number – 85% of people will have encountered at least one racist person (1 minus .999 to the power of 1900). What if they encounter only 500 people? Then it takes only about 0.04% of the population to be racist.

(Notice that even with the 85% figure, the actual number of racists can be pretty low. There are hundreds of bad things in life that have happened to 85% of us, but are nonetheless not common enough to panic about.)

So how racist is Peterborough? If you're a visible minority student responding to the survey, how much do you get out? Because it makes a huge difference – in my example, racism could be four times as high in one calculation as the other.

Needless to say, the study doesn't appear to address this issue. Nor does it talk about how the current 85% compares to whatever the number might have been back in the 70s. Small-town Ontario might be different from, say, the US South, but I'd bet you that as bad (or not bad) as racism might be today, I'd guarantee you that thirty years ago in Mississippi, the proportion of blacks reporting racism in public would be a lot higher than 85%.

Another part of the report talks about how visible minority persons in Ontario are much more likely to be lower-income than whites. Now, I knew that was the case, and I bet you did too. The question is, can you simply assume that the cause is racism? One alternative, and completely plausible explanation (as Gardner points out) is that ethnic minorities are more likely to be recent immigrants, and recent immigrants tend to be poor.

There are other possible explanations, and you've probably seen those too. There is an extensive literature, and debate, on this topic. But apparently the authors don't consider that at all! Which is absolutely ridiculous, to wade into an ongoing debate and support the most naive, simplistic view without even acknowledging the existence of any other work on the subject.

It's like ... it's like hiring someone to investigate whether smoking causes cancer, and he says, "well, my granddaddy smoked, and he lived to be 100, so smoking is fine." The problem is not that you're wrong – the problem is that the taxpayers funded you for a year and a half so you could investigate an important public-policy issue, and you didn't even bother to do any research or question your own opinions. And, indeed, the report not only doesn’t appear to have any of this kind of analysis, but it even draws quantitative conclusions in the utter absence of quantitative data!

Shouldn't a government-funded, heavily-publicized study have to undergo at least as much peer review as an obscure academic paper?


Labels:

5 Comments:

At Wednesday, November 26, 2008 9:51:00 AM, Blogger Brian Burke said...

I think part of the problem is the press. For starters, they don't hold the researchers or government accountable for this bunk. They do the opposite. The reasons undoubtedly have to do with skewed political biases, but most reporters don't have the critical thinking skills you do. They see "academic study" and "conclusions that fit with my pre-conceived notions" and go straight to print.

But the process is dynamic. Researchers know what reporters are looking for. If they want press and academic acclaim, they're going to bend things to fit the conclusions they're looking for.

 
At Monday, December 01, 2008 10:42:00 PM, Blogger doc said...

Based on Phil's post, it sounds to me as if the reporters did a pretty good job of pointing out (using a source) that the study is flawed. It also sounds to me as if the people who funded the study did not take seriously the process of finding someone to do the study, throwing the money (instead) at "two high-profile former politicians." So I have to disagree with Brian's conclusion that the press is the problem here.

And, as an academic, I'd also have to say that if you want acclaim inside the academy for doing research on something, you'd damned well better do it right. Your professional peers are going to be more than willing to tear you apart if you use sloppy techniques, crappy methodology, and anecdote-as-evidence. Based solely on what Phil reports, I'd have to say that there's no way that this study could have survived any sort of peer-review process.

And, again, it sounds as if it didn't survive the journalistic-review process either. And that's good news, frankly.

 
At Sunday, January 11, 2009 7:26:00 PM, Anonymous Anonymous said...

The day that people stop calling themselves white, black, hispanic or asian and start calling themselves American, racism will end.

 
At Saturday, March 07, 2009 5:01:00 AM, Blogger kevio said...

威創牙醫診所除了提供優質的植牙技術外還提供假牙|矯正|牙周病治療,是值得您信賴的牙醫診所植牙權威。

獅王紋身工作室提供專業的無痛刺青技術,獅王紋身在世界TATTOO大賽中,獲獎無數,獅王紋身給您最時尚的作品。

陳駿逸皮膚科診所提供了治療痘痘的服務,皮膚雷射權威,包括雷射脈衝光除斑等,讓您回復青春蘋果臉。

ck皮件處理棧提供專業洗包包|洗鞋子|各式皮件修理保養疑難雜症都有服務,清洗包包專門店讓您的包包、鞋子、永遠保持最新的況態唷。

杏儒中醫診所提供了糖尿病的治療。

seo大師e王國幫您的網站輕鬆在您的行業裡站上第一頁,e王國的關鍵字行銷是您的好幫手,包括關鍵字自然排序、都能讓您獲得完美的效果,以目前的網路行銷不外乎是關鍵字自然排序為主、而關鍵字行銷seo又是e王國的強項也包括關鍵字廣告的服務,讓e王國幫您征服網海。

Weekfun星期樂優質生活平台-各式美食,精品,住宿,夜店,批發資訊,讓您不用出門也能盡知天下事,好的行銷帶你進墳場,好的行銷讓您上戰場,Weekfun星期樂"激動"行銷團隊將提供您前所未有的網路行銷,seo,關鍵字行銷,關鍵字廣告,關鍵字,自然排序,部落格行銷,等多項網路行銷類型供您挑選適合自已店裡的好行銷,網路爆紅不是夢!!

 
At Monday, March 23, 2009 4:23:00 AM, Anonymous Anonymous said...

看房子,買房子,建商自售,自售,台北新成屋,台北豪宅,新成屋,豪宅,美髮儀器,美髮,儀器,髮型,EMBA,MBA,學位,EMBA,專業認證,認證課程,博士學位,DBA,PHD,在職進修,碩士學位,推廣教育,DBA,進修課程,碩士學位,網路廣告,關鍵字廣告,關鍵字,廣告,課程介紹,學分班,文憑,牛樟芝,段木,牛樟菇,日式料理, 台北居酒屋,燒肉,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,住宿,訂房,HOTEL,飯店,造型系列,學位,牛樟芝,腦磷脂,磷脂絲胺酸,SEO,婚宴,捷運,學區,美髮,儀器,髮型,牛樟芝,腦磷脂,磷脂絲胺酸,看房子,買房子,建商自售,自售,房子,捷運,學區,台北新成屋,台北豪宅,新成屋,豪宅,學位,碩士學位,進修,在職進修, 課程,教育,學位,證照,mba,文憑,學分班,網路廣告,關鍵字廣告,關鍵字,SEO,关键词,网络广告,关键词广告,SEO,关键词,网络广告,关键词广告,SEO,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,居酒屋,燒烤,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,小套房,小套房,進修,在職進修,留學,證照,MBA,EMBA,留學,MBA,EMBA,留學,進修,在職進修,牛樟芝,段木,牛樟菇,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,住宿,民宿,飯店,旅遊,美容,美髮,整形,造型,設計,室內設計,裝潢,房地產,進修,在職進修,MBA,EMBA,關鍵字排名,網路行銷,关键词排名,网络营销,網路行銷,關鍵字排名,关键词排名,网络营销,羅志祥,周杰倫,五月天,蔡依林,林志玲,羅志祥,周杰倫,五月天,蔡依林,林志玲,PMP,在職專班,研究所在職專班,碩士在職專班,PMP,證照,在職專班,研究所在職專班,碩士在職專班,網頁設計,網站設計,網頁設計,網站設計,网页设计,网站设计,网站设计,网页设计

 

Post a Comment

Links to this post:

Create a Link

<< Home