Does vaping induce teenagers to become smokers?

Do electronic cigarettes lead users into smoking real cigarettes? In other words, is vaping a "gateway activity" to smoking?

A recent study says that, yes, vapers are indeed more likely to become smokers than non-vapers are. In fact, they're *four times* as likely to do so. 

The study looked at a sample of young people aged 16 to 26 who said they didn't intend to become smokers. When they caught up with them a year later, only 9.6 percent of the non-vapers had smoked in the past year. But 37.5 percent of the vapers had!

Seems like pretty strong evidence, right? The difference was certainly statistically significant.

Except ... here's an article from FiveThirtyEight that suggests that, no, this is NOT strong evidence that vaping leads to smoking. Why not? Because the sample size was very small. The vaping group comprised only 16 participants, compared to 678 for the control group.

Vaping:      6/ 16  (37.5%)

Non-Vaping: 65/678   (9.6%)

FiveThirtyEight says,

"Voila, six out of 16 makes 37.5 percent — it’s a big number that comes from a small number, which makes it a dubious one. 

So because six people started smoking, news reports alleged that e-cigs were a gateway to analog cigs."

Well, I have some sympathy for that argument, but ... just a little. Statistical significance does adjust for sample size, so, in effect, the data does actually say that the sample size issue isn't that big a deal. To argue that 16 people isn't enough, you need something other than a "gut feel" argument. For instance, you could hypothesize that 16 vapers out of 694 people is a lower incidence of vaping than in the general population, and, therefore, you're getting only "out of the closet vapers" self-identifying, which makes the 16 vapers unrepresentative. 

But, the article doesn't make any arguments like that.

------

The FiveThirtyEight story tries to make the case that the study, and the press release describing it, are biased, because they're too overconfident about a sample that's too small to draw any conclusions. 

I don't agree with that, but I DO agree that there's bias. A much, much worse bias, one that's obvious when you think about it, but one that has nothing to do with the actual statistical techniques. 

What's the actual problem? It's that the whole premise is mistaken. Comparing vapers to non-vapers is NOT evidence for whether vaping entices young people into smoking. Not at all. Even with a huge sample size. Even if you actually counted everyone in the world, and it turned out that vapers were five times as likely to become smokers as non-vapers, that would NOT imply that vaping leads to smoking, and it would NOT imply that banning vaping would "protect our youth" from the dangers of smoking real cigarettes.

It could even be that, depsite vapers being five times as likely to take up smoking, vaping actually *reduces* the incidence of smoking.

How? Well, suppose that vapers and smokers are the same "types" of people, those who want to send a signal that they're risk-takers and nonconformists. Before, they all took up smoking. Now, some take up smoking and some vaping. Sure, some of the vapers become smokers later. But, overall, you could easily have fewer smokers than before you started. 

As Tom Tango put it:

"What do I think? A vaper is in denial. It’s not the vaping itself that causes you to become a smoker, but simply that a smoker is a closet-vaper. 

"This is likely true of most vices. It won’t act as a gateway, but simply that you will try it because you were going to try to harder stuff anyway. Even if you didn’t want to admit it. 

" ... There’s a dozen ways to get from Chinatown to Times Square. Manhattan then adds a direct bus line that goes up Broadway. Does that bus “cause” people to go from Chinatown to Times Square? Or, does it simply become a stepping stone that they would have otherwise bypassed? 

"Basically, do the same number of people end up going Chinatown to Times Square? 

"Do the same number of people end up smoking the real stuff anyway? All vaping is doing is redirecting the flow of people?"

------

If that sounds too abstract in words, it'll become crystal clear if we just change the context, but leave the numbers and arguments the same.

"Ignore The Headlines: We Don’t Know If Suicide Hotlines Lead Kids to Kill Themselves.

"After a year, 37.5 percent of those who had called a Suicide Hotline had gone on to end their own lives. That's a big percentage when you consider that the suicide rate was only 9.6 percent among respondents who hadn’t called the hotline.  

"Our study identified a longitudinal association between suicide hotline use and progression to actual suicide, among adolescents and young adults. Especially considering the rapid increase and promotion of distress lines, these findings support regulations to limit suicide hotlines and decrease their appeal."

It's exactly the same thing! Really. I edited a bit, but most of the words come exactly from the original articles on vaping.

Now, you could argue: well, it's not REALLY the same thing. We know that suicide hotlines decrease suicide, but, come on, can you really believe that vaping reduces smoking?

To which I answer: absolutely. I *do* believe that vaping reduces smoking. If you believe differently, then, study the issue! This particular study doesn't provide evidence either way.

And, more importantly: "can you really believe?" is not science, no matter how incredulously you say it.

------

Logically and statistically, the relevant number is NOT what percentage of vapers (hotline callers) go on to smoke (commit suicide). The relevant number is, actually, how many people would go on to smoke (commit suicide) if vaping (suicide hotlines) did not exist. 

Why is this not as obvious in the vaping case as in the hotline case? Because of bias against vaping. No other reason. The researchers and doctors start out with the prejudice that vaping is a bad thing, and, because of confirmation bias, interpret the result as, obviously, supporting their view. It seems so obvious that they don't even consider any other possibility.

I bet it's not just vaping and suicide hotlines. I suspect that we'd be eager to accept the "A leads to more bad things than non-A" if we're against A, but we see it's obviously a ridiculous argument if we approve of A. Here are a few I thought of:

"37% of teenagers who play hockey went on to commit assault, as compared to only 9% who didn't play hockey. Therefore, hockey is a gateway to violence, and we need to limit access to hockey and make it less appealing to adolescents." 

"37% of teenagers who use meth go on to commit crimes, as opposed to only 9% who didn't use meth. Therefore, meth is a gateway to criminal behavior, and we need to limit access to meth and make it less appealing to adolescents." 

"37% of patients who get chemotherapy go on to die of cancer, as opposed to only 9% of patients who don't get chemo. Therefore, chemotherapy leads to cancer, and we need to limit access to chemo and make it less appealing to oncologists." 

"37% of men who harass women at work go on to commit at least one sexual assault in the next ten years. This shows that harassment is a precursor to violence, and we need to take steps to reduce it in society."

If you're like me, in the cases of "bad" precursors -- meth and harassment and vaping -- the arguments seem to make sense. But, in the cases of "good" precursors -- hockey and chemotherapy and suicide prevention lines -- the conclusions seem obviously, laughably, wrong.

It's all just confirmation bias at work.

-------

The FiveThirtyEight piece references one of their other posts, titled: "Science Isn’t Broken.  It’s just a hell of a lot harder than we give it credit for."

In that piece, they give several reasons for why so many scientific findings turn out to be false. They mention poor peer review, "p-hacking" results, and failure to self-correct.

Those may all be happening, but, in my opinion, it's much less complicated than that. 

It's just bad logic. It's not as obvious as the bad logic in this case, but, a lot of the time, it's just errors in statistical reasoning that have nothing to do with confidence intervals or methodology or formal statistics. It's a misunderstanding of what a number really means, or a reversal of cause and effect, or an "evidence of absence" fallacy, or ... well, lots of other simple logical errors, like this one.

Regular readers of this blog should not be too surprised by my diagnosis here: most of the papers I've critiqued here suffer from that kind of error, the kind that's obvious only after you catch it. 

FiveThirtyEight writes:

"Science is hard — really f*cking hard."

But, no. It's *thinking straight* that's hard. It's being unbiased that's hard. It must be. There were hundreds of people involved in that vaping study -- scientists, FiveThirtyEight writers, doctors, statisticians, public policy analysts, editors, peer reviewers, anti-smoking groups -- and NONE of them, as far as I know, noticed the real problem: that the argument just doesn't make any sense.

Hat Tip: Tom Tango, who figured it out.