Wednesday, October 29, 2008

Billy Beane: use sabermetrics to improve health care

In this New York Times Op-Ed, Billy Beane teams up with Newt Gingrich and John Kerry to suggest that a sabermetric approach could help save American health care.

"Remarkably, a doctor today can get more data on the starting third baseman on his fantasy baseball team than on the effectiveness of life-and-death medical procedures. Studies have shown that most health care is not based on clinical studies of what works best and what does not — be it a test, treatment, drug or technology. Instead, most care is based on informed opinion, personal observation or tradition ...

"One success story is Cochrane Collaboration, a nonprofit group that evaluates medical research. Cochrane performs systematic, evidence-based reviews of medical literature. In 1992, a Cochrane review found that many women at risk of premature delivery were not getting corticosteroids, which improve the lung function of premature babies.

"Based on this evidence, the use of corticosteroids tripled. The result? A nearly 10 percentage point drop in the deaths of low-birth-weight babies and millions of dollars in savings by avoiding the costs of treating complications."

I have no doubt that the authors are right. Baseball's "doctors" – sportswriters and broadcasters – believe that Derek Jeter is an above-average fielder, despite mountains of evidence and tens of studies pointing the other way. If experts can be so wrong, for so long a time, *despite* the statistical evidence, doesn't it seem like doctors are going to be just as wrong when there is *no* statistical evidence?

The authors recommend:

"Working closely with doctors, the federal government and the private sector should create a new institute for evidence-based medicine. This institute would conduct new studies and systematically review the existing medical literature to help inform our nation’s over-stretched medical providers."

Absolutely this is a good idea. But I think the process can be sped up. My suggestion: open the database to everyone, and offer prizes for the most significant findings, as judged by the new institute.

Why do I think this would work?

1. People are smart. The more people who work on this, the more knowledge is going to come out of the process. Sabermetrics wouldn't have happened if anyone had waited for "formal" baseball researchers to get to it. Linux wouldn't be as advanced as it is today if not for the open-source movement, and thousands of talented volunteers contributing continuous improvement.

2. People respond to incentives. The Netflix contest, where competitors scour a database of customer preferences to try to predict that customer's other preferences, attracted hundreds of competitors. That's very similar to what's involved here. People love an interesting challenge with recognition and prizes at the end of it.

3. You don't need to be a doctor. This is an exercise, mostly, in finding relationships in data. It's almost exactly what sabermetricians do every day. Sure, you'd need to know a bit of medicine. For instance, you'd need to know that corticosteroids are thought to improve the lung function of premature babies. Once you know that, then you take a look at whether the data show it. How do you find that out? Well, I'd bet there would be no shortage of "hints" like these, just as there is no shortage of advice on what companies to invest in, or tips on what third basemen are due for breakout seasons.

4. I'd bet that, often, doctors are unaware of many of the new findings in medicine. It probably takes a while for a good idea to become standard, as it takes time for more and more doctors to become aware of it. But with a big monthly prize, and the resultant publicity, any new knowledge that comes out of this process will be hard to ignore, especially if the medical establishment gets on board.

There is a certain mindset that if something is important, you need experts and academics and government to do it. That's true when you need extremely specialized knowledge or equipment. But when it's software development, or data analysis, there's no shortage of brilliant laymen willing to take a stab at it.

I think the lay public would help, enthusiastically, to figure this stuff out. Releasing the data could save a lot of lives.

Labels: , ,


At Wednesday, October 29, 2008 2:24:00 PM, Blogger Brian Burke said...

There's an interesting chapter on this in the book Super Crunchers. It's a good read, especially for the sabermetrically-inclined.

At Wednesday, October 29, 2008 3:37:00 PM, Blogger Don Coffin said...

Except. Health information is not the same as information about someone's perforance in baseball. Data from clinical trials is not the same. Recordkeepers and researchers have to bend over backwards to prevent disclosure of any information that could identify any individual.

And that's a good thing. And it's difficult to do. So the second part of this is how to insure that open access to clinical trial (and other) medical information also protects patient privacy.

(I'm sensitive to this because my wife works in health care, and so I have become more famiiar than I ever thought I would be with the HIPPA regulations...)

At Wednesday, October 29, 2008 4:22:00 PM, Blogger Phil Birnbaum said...

doc: yes, absolutely, you'd have to make sure nobody was identifiable. I didn't mention it because it seemed to me that if you have a large amount of aggregate data, that shouldn't be a problem.

But I trust you when you say that it's difficult to do. Still, it should be worth the effort, no?

At Thursday, October 30, 2008 10:24:00 AM, Anonymous Anonymous said...

As I noted over at TheSportsEconomist, it isn't clear what planet this "Beane Group" is living on.

Evidence-based research of the type they describe has been going on forever; and will continue to be the focus of nearly all research ongoing.

Just because one invents a cute name for performance in medicine does not give them the right to ignore the entire literature and ongoing research on effective medicine.

And especially people involved with assessing player performance. The experiment isn't "Oh look, sometimes I can find an anecdote about more effective treatment" AFTER THE FACT. They all should know better since that is the form of criticism they have been fending off their entire careers.


At Thursday, October 30, 2008 7:40:00 PM, Blogger Erich said...

Re: doc/phil- check this out. I've thought about participating, though the costs currently seem to outweigh the benefits

Also, this discussion seems relevant to the viewpoints my favorite medical thinker

At Friday, October 31, 2008 10:45:00 PM, Blogger Brian Burke said...

Rodney-Maybe I'm missing something about your point. I think the point of EBM is the clinical application of the research, not the type of research or what it's called.


Post a Comment

<< Home