Clutch hitting and getting killed by a puck

There was something someone said about clutch hitting a few months ago -- I think it was Tango -- that took a while to sink in for me.

It went something like this: saying that clutch hitting ability "does not exist" is silly. Humans are different, and different people react to pressure in different ways. So, *of course* there must be differences in clutch hitting ability. The question isn't whether or not clutch hitting exists, because we know it must, but *to what extent* it exists.

It turns out that extent is small. The best we can say is that clutch hitting studies have found that the SD of individual clutch tendencies is about 3 percent of the mean. So for players who are normally .250 hitters, two out of three of them will be between .242 and .258 in the clutch, and 19 out of 20 of them will be between .235 and .265. ("The Book" study on the topic used wOBA, rather than batting average, but this is probably still roughly true.)

That's pretty minor, especially compared to other factors like platoon advantage, and so on. More importantly, there's not nearly enough data to know which are the real clutch hitters and which are the real clutch chokers. When your favorite pundit pronounces player X as "clutch," that's still completely pulled out of his butt.

So the conclusion remains that calling specific players "clutch" is silly, but the stronger statement "clutch hitting does not exist" is unjustified.

------

As I said, that took a while to sink in for me. I'm willing to agree with it. But I still have reservations. Because, you can take this "humans are different" stuff to extremes.

Suppose you found that a certain bench player hits much better than usual on the first day of the month. Some announcer notices and says the manager should always play him on those days. The sabermetricians step in and say, "that's silly: there's no reason to believe it's a real effect, and the guy's not very good on all the other days."

But, by the same logic ... all humans are different. For me personally, every day on the first of the month, I'm amazed how fast the past month flew by. It seems to me that time is going faster as I get older, and it makes me a little sad. On the other hand, other people might be happier on the first of the month. Maybe that's when they get paid, and they're feeling rich.

So, since humans are different, and circumstances are different, why couldn't their be a *real* "first of month" effect? The same logic says there *has* to be.

But, the thing is ... any such effect is probably very, very small. Too small to measure. There's no way it'll affect player performance (in my estimation) even one one-hundredth as much as clutch.

It's real, but it's too small to matter.

In cases like that, is it OK to say that "there's no such thing" as "first of month hitting talent"? Maybe it's not technically true, but I don't think that'll always stop me from saying it anyway. But, if I remember, I'll say "if it exists, it's probably infinitesimal."

For clutch, the Andy Dolphin study mentioned in "The Book" came up with an SD of clutch talent estimated at .008 in wOBA. I'm not 100% willing to accept that, mostly because, as Guy points out in a recent clutch thread on Tango's blog, there might be a natural clutch difference due to batter handedness or batting style that doesn't really reflect "clutch talent" as it's normally understood.

But, what I might choose to say is that there's weak evidence of a small clutch effect, and add that there isn't nearly enough evidence to know who's weakly clutch and who's weakly choke.

Or, I might say that there's "no real evidence of a meaningful clutch effect," which says the same thing, but with a more understandable spin.

------

Anyway, it occurred to me, while thinking about this, that we like to think about things as "yes/no" when what's really important is "how much". By that, I don't mean the moral argument about "black and white" versus "shades of grey." What I mean is something more quantitative -- "zero or not zero" versus "how much?"

Take, for example, Omega 3 fatty acids. I hear they're good for you. You read about them in the paper, and on milk cartons, and fish products.

But, *how* good for you are they? Isn't that important, to quantify it? The media doesn't seem to.

Here, for instance, is an article from the CBC. It talks about which Omega 3s are better than others, and ends with a recommendation that people eat 150 grams of fish a week. But ... how big are the benefits? The CBC article doesn't tell us. You'd expect better from the Wikipedia entry, but even that doesn't tell us.

So, how are we supposed to decide whether it's worth it?

I mean, suppose you really, really don't like fish. Won't your decision on whether or not to eat fish anyway, depend on how much the benefit is?

And, don't other foods have benefits too? If I eat fish for lunch, that means I won't be eating oat bran instead. How can I decide which is better for me without the numbers?

Or, suppose I have a choice ... go to a fish restaurant for lunch, or go for a jog around the block. Both might help my heart. Which one will help more? Or, suppose a nice piece of salmon in my favorite restaurant is $5 more than a piece of chicken. Could I do better with the $5? Maybe I could save up and use the money to switch to a more expensive gym, where I'll go more often. Or, I could put it in a travel fund, to buy better travel health insurance next trip I go abroad. Which option for my $5 is best for my long-term health?

There are lots of people who hear "fish is good for your heart," so they start buying fish. What they're really buying is a security and good feelings. Because if you ask them to quantify the benefits are, they just look at you blankly, or they quote some authority saying it's good for you. They treat it as a yes/no question -- is it good for you? -- when the important question is "how much is it good for you"?

Maybe I'm just cynical, but when someone tells me a food is good for me, without quantifying the benefits, I just assume the benefits are tiny, like clutch hitting. After all, for a study to make a journal, all you need is 5 percent significance, which means that most of the claims are probably not true. And, even if they *are* true, the study probably found that the benefits are small (though possibly statistically significant).

In his book "Innumeracy," back in the 80s, John Allen Paulos suggested there be a logarithmic scale for risk, so when the media tell you something is risky, they could also use that number to tell you how much. I'd like to see that for everything, not just risk. And, I'd like to see the scale changed. Because, if it's just a logarithm, people will still ignore it. If someone tells me I should have had the salmon, I might say, "Why? It's only a 0.3 on the Paulos scale, which is almost nothing." And they'll say, "0.3 is better than nothing. It all adds up. You should look out for your health."

But, what if you expressed the benefits in terms of something real, like exercise? Everyone intuitively understands the benefits of exercise, so the scale would make sense. And, it would make the insignificance of small numbers harder to rationalize away. So when someone insists I eat the salmon, I can say, "look, it's $5, and it's only the equivalent of 30 meters of jogging." They can still say, "30 meters is better than nothing!" And I'll say, "look, I'll have the chicken, and when we get outside the restaurant, I'll jog the 30 meters to the car, and save the $5."

They could use that scale in the supermarket, too. I buy milk with Omega 3 (not for the health benefits, but because it stays fresh longer). Wouldn't it be great if it said on the carton, "Each glass gives you the benefits of 0.3 push-ups"? That would be awesome.

I'm not sure if exercise is the absolute best reference point. Maybe "days of life" is better. ("Each floret of broccoli adds 3.2 seconds to your lifespan!") But, still, it should be possible to come up with *something* that'll work.

------

For risk, a good unit of measure would be "miles of driving". That works well because it's widely recognized that driving is dangerous, and we all know people who died in car accidents, so we have an idea of the risk. But, there's no moral stigma associated with it (unlike, say, smoking), so we can be fairly rational about it.

In a comment thread yesterday on Tango's blog, there was a discussion about putting a protective barrier down the lines of baseball stadiums, to prevent people from getting hurt by foul balls. That would be similar to what the NHL did, when they installed a mesh partition behind each net after the death of a spectator in 2002.

Suppose the hockey netting wasn't there. What would the risk be?

From 1917 to the end of the 2002 season, there were 37,480 regular season NHL games. Assuming 15,000 fans per game, that's 562 million fans. Suppose one-quarter of those fans are sitting in high-risk seats; that's 140 million fans. Finally, suppose that players shoot a lot harder now, so today's risk is double the historical average. That means it only takes 70 million of today's fans to shoulder the same risk as throughout the NHL's history. (We could add a bit for playoffs and pre-season, but never mind.)

So, that's one death out of 70 million people, or a risk of 1 / 70,000,000 of dying at any given game. Is that big or small? It's hard to say. What is it in terms of driving?

In 2010, there were 1.09 deaths per 100 million miles travelled. Let's round that down to 1.00, just to make the calculation easier. So there's 1 / 100,000,000 of a death per mile.

That means the hockey game is the equivalent of 1.43 miles.

So that's how I'd say it: putting the mesh up at hockey games makes each fan behind the net safer by 1.43 miles of driving.

Doesn't that give you a really good intuitive idea of the risk involved?

Of course, that's death only, and not injury. But I'm sure you could find injury data for car accidents, and for puck injuries, and come up with some kind of reasonable scale. I'm guessing that if you did that, you'd probably find that it was still about the same order of magnitude of a couple of miles. But I don't know for sure.

And, hey, now that I think about it, you could treat *healthy* things as "miles of driving saved*. If something saves you one minute of lifespan, that's easily converted to driving. 100 million miles, at an average of 30 miles an hour, is about 380 years of driving. (At six hours of driving (or passenging) a week, that's about 10,000 years of life per fatal accident. That would mean that about one American in 200 eventually winds up dying in a car accident. Sound about right?)

Suppose the average driver has 40 years of life left, on average. Then every 380 years of driving wipes out 40 years of life. That works out to 9.5 years of driving per year of life. Round that to 10. That means that every hour you drive -- 30 miles -- costs you six minutes of life. So five miles of driving costs you a minute of life.

(Again, that's death only, and not injury. In terms of quality of life, you'd probably want to bump up the "5 miles" figure, because bad health usually makes you miserable before you die, but car accidents often kill you instantly. But let's stick with five miles for now.)

So if eating salmon for a week saves you one minute of lifespan, eating salmon for a week is like cutting 5 miles off your commute one day. I made that "one minute" number up; if anyone knows how to figure out what the real number is, let me know.

------

Let's do cigarettes. Actually, let's just do lung cancer, to make it easier.

According to Wikipedia, 22.1% of male smokers will die of lung cancer before age 85. Let's assume that entire amount is from cigarettes.

Since this is a back-of-the-envelope calculation, let's just make some reasonable guesses. I'll assume the average male smoker starts at age 18 and smokes a pack a day. I'll also assume that when a smoker dies of lung cancer, it's at age 70 on average, and it cuts 15 years off his lifespan.

So: 52 years times 365 days times 20 cigarettes equals ... 379,600 cigarettes. 15 years of lost life equals 7,884,000 minutes. So each cigarette equals 21 minutes.

Multiply the 21 minutes by 22.1% and you get 4.6 minutes.

Google "cigarette minutes of life" and you get figures ranging from 3 to 11 minutes ... and that's of *all causes*, not just lung cancer. So, we're in the right range.

4.6 minutes equals 23 miles.

If you're a pack-a-day smoker, your risk is the same as if you drove from New York to Los Angeles every week or so.

------

If I were made evil dictator of the world, I would insist that every media report on risks and benefits tell you *how much*. Every report of a health scare, every quote from a safety group, every recommendation from a nutritionist, would need to include a number. Because, really, when someone tells you "vegetables are healthy," that's useless. Even if it's true, how true? Is it true like "the platoon advantage exists?" Is it true like "clutch hitting exists"? Or is it true like "first of month hitting exists?"

The difference matters. We need the numbers.

Everyone hates the KFC "Double Down"

Why are we humans so gullible as to believe in things that are obviously not true, like astrology, homeopathy, and stuff Joe Morgan says?

One argument is that it's a question of being numerate and understanding the scientific method. If people who believe in homeopathy truly understood that there isn't even one molecule of active ingredient of their "medicines," they might be swayed. And if only Joe Morgan learned a bit about how the Runs Created formula actually works, he'd understand why we've reached the conclusions we have and come over to our side. It's just a matter of education.

Well, I'm not so sure that's true. Case in point: the reaction to KFC's "Double Down" sandwich.

The "sandwich" consists of two breaded, boneless chicken breasts, with bacon and cheese and sauce in between. There's no bun. (If you've never seen one before, here's a picture.)

The Double Down has been available in the US for a few months now, but it just came to Canada about a month ago, and all the usual nutrition and obesity spokespeople are flipping out. Google "double down Canada," and see for yourself:

-- "KFC's Double Down hits Canada; nutritionists worried - CTV News"

-- "Double Down, Canada: We gobble artery-clogging fat fare from KFC in record numbers"

-- "KFC’s Double Down raises eyebrows among Canadian nutrition experts."

There's lots more than that. For instance, my local paper, the Ottawa Citizen, had TWO editorial cartoons about the Double Down in the last couple of weeks. (Here's one of them.)

So what's the big problem? I don't see it. The Double Down isn't really any worse than other fast food items. It has 540 calories (including 30g of fat) and 1,740 mg of sodium. The Big Mac, which nutritionists don't seem to be pooping themselves over, has the same 540 calories (28g fat) and 1,020 mg of sodium. And the double Wendy's Baconator has 980 calories (63g fat) and 1,830 mg of sodium. (Let's not even talk about the triple.)

So the Double Down doesn't seem like that big a deal. Yes, it's got a little more sodium per calorie than the other two items, but, really, not that much more. And it's not like KFC is any worse than chicken anywhere else.

A KFC breaded boneless chicken breast has 560 mg of sodium. I went down to my kitchen and pulled out a box of President's Choice frozen breaded chicken breasts. Each one has ... exactly the same, 560 mg of sodium. I checked the "blue menu" variation of those chicken breasts, the ones with less fat and fewer calories: 450 mg. And, finally, I checked non-breaded honey-lemon chicken breasts: each breast had 460 mg of sodium. Those are the healthiest, least-fatty ones, at only 150 calories each.

So, again, what's the big deal? It looks to me like seasoned chicken breasts routinely have salt added to make them taste good.

So why blame KFC? Is it because of innumeracy? Do the nutritionists and the journalists simply not understand what the numbers mean? No, it's not. It's the opposite, in fact. Every article I've seen points out that other fast food products aren't much better, and they all print the numbers for calories and sodium. But, somehow, it goes in one sentence and out the other.

Why is the Double Down singled out for so much opprobrium? Because it has no bun. I think it's a status thing. Having no bun -- or rather, having a bun made out of meat -- is weird, and weird in a lower-class non-gourmet way. It looks like gluttony -- chicken and more chicken, stacked so high it's hard to bite into. It's something that truckers might eat, but cultured nutritionists, and educated people who care about obesity of the lower classes -- well, when they want to eat chicken, they use a knife and fork ... or, at least, put it in a bun like civilized people.

Furthermore, the existence of the Double Down feels like a personal attack on nutritionists. Because it has no bun, it's kind of fun, with overtones of gluttony. People who take nutrition seriously *hate* that. Food isn't supposed to be fun. It's supposed to be serious. Nutritionists have spent years and years getting degrees and becoming experts on what's good to eat and what's not ... and now what happens? KFC tries something completely different, appealing to people's appetites in a whole new way, *without even consulting the nutritionists*!

Without the absence of the bun, none of this happens, because there's really intrinsically nothing special about the product. Think about it:

1. The Double Down is made of two breaded boneless chicken breasts. But KFC has had those same boneless chicken breasts around for some time now. They actually offered them in a meal, but flat on a plate, with a knife and fork, and two side dishes. Nobody complained, and nobody took notice.

If I were to order that, and take the tray to my table, and eat it, nobody would look twice. But if I took those two patties, stacked them up on top of each other, and spread the the honey mustard sauce between them, and then I picked up my newly-constructed double-decker greasy chicken thingy in my two hands, and ate them that way ... well, people would stare, and some of them would be grossed out. Even though it's the same meal, they'd think I was a disgusting pig, and wonder why I'm not ill and obese.

Same product, different presentation, completely different reactions. It's the shape of the Double Down, not the content. To some, stacked chicken patties with sauce in the middle is disgusting. But "that's disgusting" won't get them taken seriously, so they have to complain about the nutritional content.

2. McDonald's has a breaded chicken sandwich. They'll let you make it a double for an additional charge, and they'll be happy to add cheese and bacon. Nobody complains. Why? Because it's got a bun. Take away the bun, and what have you got? The McDonald's version of a Double Down.

3. There's a well-regarded product that are almost exactly the equivalent of the Double Down, and nobody complains. It's Chicken McNuggets. Well, it's not exactly the same, because there's no cheese or bacon. But a 10-piece McNuggets, with two sauces, otherwise has almost exactly the same profile as the Double Down: 610 calories (35g fat), and 1560 mg sodium.

But McNuggets are bite sized. You eat them individually and daintily. You don't stack them all up and ooze the sauce in between them.

4. Okay, here's an established main course that really IS almost exactly a Double Down: Chicken Cordon Bleu. It's a breaded chicken breast, and when you cut it open, there's cheese and butter and ham inside. (Yes, it's ham instead of bacon, but close enough.) If you look up nutrition information for Chicken Cordon Bleu, it's pretty close. It has to be -- the ingredients are almost the same.

Chicken cordon bleu is classy, classy enough to serve at weddings. Foodies don't mind it, and nobody campaigns against it. But if KFC sold it, and made you eat it with your hands? Yuck! It's gross, and the multinational corporate megagiant doesn't care about the health of its customers!

-----

We humans have a "blink" mentality -- we see a situation and come to a conclusion in an instant. Often, those conclusions are wrong, fed by our prejudices. Overcoming those prejudices does indeed take numeracy and intelligence.

But the most important thing it takes is a willingness to think about the issue with an open mind.

No matter how smart you are, and no matter how many math courses you've taken, you still have to be able to put aside your instinctive first impression and take a fair look at the issue and the evidence. Of all the people who'd be able to evaluate a foodstuff with a clear eye, you'd think nutritionists and health experts would be at the top of the list. But despite all their expertise -- or perhaps because of it -- they're among the worst offenders.

(In fairness, it could be selective sampling; the nutritionists who don't have a problem with it wouldn't be out for media attention. But you'd think if they were that common, some reporter would have found a few to present a countering view.)

In Ontario, the minister in charge of health promotion even started talking about *banning* the Double Down. She was quickly shot down by the Premier. But still -- the public official in charge of public health, the one who you'd think would have the most responsibility to see both sides and evaluate the situation, had the most extreme, prejudiced reaction. It's as if the head of the American Bar Association suddenly called for an impromptu lynching, with a thousand lawyers lining up behind her.

The solution is not just to teach more math. The solution is to make it socially unacceptable to accept silly, unjustified arguments. There are some places where we've been partially successful. We're able to do that, at least a little bit, in academia, where peer review at least calls out the cranks. We're able to do that in the justice system -- I'd bet that few judges would give credence to the argument that the Double Down is significantly worse than the Baconator just because you consume it more lustily.

We just have to promote that way of thinking in everyday life -- which is difficult, and perhaps impossible. But in my utopia, when nutritionist Joe Morgans say that Chicken Cordon Bleu is worse for you when you use your hands instead of a plate, we stop quoting them in approving newspaper articles, and laugh at them instead.

Properly interpreting data is more than just "numeracy"

Here's a recent post from Freakonomist Stephen Dubner on numeracy. Dubner quotes John Allen Paulos on a specific, interesting example on how numbers can mislead:

"Consider the temptation to use the five-year survival rate as the primary measure of a treatment for a particular disease. This seems quite reasonable, and yet it’s possible for the five-year survival rate for a disease in one region to be 100 percent and in a second region to be 0 percent, even if the latter region has an equally effective and cheaper approach.

"This is an extreme and hypothetical situation, but it has real-world analogues. Suppose that whenever people contract the disease, they always get it in their mid-60s and live to the age of 75. In the first region, an early screening program detects such people in their 60s. Because these people live to age 75, the five-year survival rate is 100 percent. People in the second region are not screened and thus do not receive their diagnoses until symptoms develop in their early 70s, but they, too, die at 75, so their five-year survival rate is 0 percent. The laissez-faire approach thus yields the same results as the universal screening program, yet if five-year survival were the criterion for effectiveness, universal screening would be deemed the best practice."

Dubner argues that, in order to better spot these kinds of situations, we need a public that's more numerate.

But, I think, numeracy isn't really what's needed here: it's not really mathematics, but just common logic. Really, you can rewrite Paulos's situation involving almost no numbers at all (except numbers of years, which everyone understands):

"Suppose that if you're diagnosed with a certain disease in one region, you always die within five years. But if you're diagnosed with the same disease in another region, you *never* die in five years. Does that mean that treatment is better in the second region? No, not necessarily. It could be that the disease takes ten years to kill you, and no treatment helps. In the "bad" region, it's not diagnosed until year eight. In the "good" region, they have universal screening, and the disease is always diagnosed in year one. So even if the universal screening does no good at all, it *looks* like it does."

All but the most innumerate person should be able to understand that, right? It's not really a question of mathematics: it's a question of *logic*. Sure, the more used to numbers you are, the more likely you are to think of this possibility -- at least to a certain extent. But, in this particular case, having an open mind and being able to think clearly are more important than mathematical ability.

And it needs a certain amount of creativity, cleverness, interest and skepticism. And data. Even a Ph.D. in math wouldn't necessarily be able to figure out what's going on, because you need the right details. Suppose a newspaper editorial calling for pre-screening simply said,

"In region A, which prescreens for the disease, five-year survival rates after diagnosis are 100%. In region B, which does not, the survival rate is zero. This evidence strongly suggest that we need to encourage prescreening."

If you'd never seen this example before, would you immediately realize the possibility that the pre-screening was no use at all? I probably wouldn't, and I regularly read academic studies with a skeptical eye. How would you expect a guy skimming the morning paper on his way to work to think of it?

Sure, I might think of it if I had seen other studies that had shown pre-screening to be ineffective. Then, I'd think, geez, how did this study come up with such a contradictory result? And maybe I'd then figure it out. But I'd have to care enough to want to think about it skeptically.

And even if I'd figured out that was a logical possibility, consistent with the numbers ... a possibility doesn't mean it's actually happening, right? I might correctly think, "yeah, it could be just a question of when the disease is diagnosed ... but is it really likely that that's *all* of it? It's more likely that *some* of the effect is early diagnosis, and the rest of the effect is more time to treat after diagnosis."

To know for sure, it's not enough to be numerate, or reasonable, or logical. You need more data. But the data might not exist. If you're lucky, the issue arises from a study somewhere, and you can read that study and figure out what's going on. But if it's just raw data, and all that's being recorded is the five-year survival rate, all you can do is note that both hypotheses fit the data: that prescreening always helps, and that it never helps.

Dubner recommendations:

"Footnotes, I guess, and transparency, and a generally higher level of numeracy among the populace."

All of which are good things. But "footnotes" assume that the authors of the study realize what might be going on. From what I've seen in sports studies, that's generally not the case. My experience is that authors do just fine at the "numeracy" stuff, dealing with the numbers and doing the regressions and statistical tests. But they're weaker at properly understanding what the numbers actually *mean*. I'd bet you that eight out of ten researchers who found data matching Paulos' example wouldn't think of Paulos' explanation. Peer reviewers would be even more likely to miss it.

Since Dubner's kind of "numeracy" is so difficult, isn't it more efficient to better use what's already out there, instead of making what would probably be a futile attempt to create more of it? If you really want to try to solve the problem of improper conclusions in studies like this, just get lots of people to look at the study, and give those people incentives to spot these kinds of things.

Here's how such a plan might work:

-- a researcher submits a study to the journal, as normal, and peer review proceeds as normal.

-- after deciding to provisionally accept the paper, the journal immediately posts it online, and allows for online comments and discussion.

-- after a suitable interval, a second set of senior referees review the comments and peer-review the paper again in light of those comments.

-- those peer reviewers allocate a fixed sum of money per paper -- $500, say -- among the commenters, in proportion to the usefulness of their comments, whether the paper gets published or not.

The benefits of a system like this are pretty clear: a potentially unlimited number of peer reviewers, motivated by money and reputation, are more likely to spot flaws than a couple of anonymous peer reviewers who are perhaps motivated to go easy on their peers. That means a lot fewer flawed papers, and a lot fewer press reports of incorrect conclusions -- conclusions that newspaper readers, no matter how intelligent or numerate, aren't given enough evidence to question.

There are other beneficial side effects beyond the obvious advantage of fewer fallacious papers:

-- the system provides an incentive for authors to give their conclusions more careful thought before submitting, saving the system time and money.

-- it provides a way for smart grad students to make a little extra money to support themselves.

-- it's cheap. I bet $500 a paper would be enough to clear most flaws. If you figure how long a paper takes to write, and how much professors get paid, $500 a study is a drop in the bucket compared to the cost of the original study.

-- the record of corrections makes it clear to everyone in and out of academia what kinds of flaws to avoid. (Within a few months, nobody will be neglecting to talk about regression to the mean, and everyone will know what selective sampling is.)

-- it will allow for a more accurate measure of academic accomplishment. Since academia is known to be a competitive environment but without formal scorekeeping, anything that allows a more accurate subjective rating of a researcher's proficiency has to be a good thing.

But I don't think this will ever happen, for political reasons. Nobody likes to have their errors exposed publicly. The current system, where peer review is mostly private, allows everyone to save face.

Furthermore, professors would probably be somewhat reluctant to criticize other professors, so the most effective critiquing would come from outside of academia. I am pretty certain that academic Ph.D.s, as a group, will never stand for a system that lowers their status by having laymen publicly correct their mistakes.