A research study is just a peer-reviewed argument (part II)
I've always said that a regression doesn't speak for itself. A regression is just manipulated data. To support a hypothesis, you need more that just data: you need an argument about why that data matters.
I wrote about that here, when I said that a research paper is just a peer-reviewed argument. Some commenters disagreed. They argued that science is, and has to be, objective -- whereas, arguments are always subjective.
Having thought about it further, I don't understand how it isn't more obvious that there's always a subjective argument involved. At the very least, if you find a significant association between X and Y, you have to at least suggest whether X causes Y, whether Y causes X, or whether something else causes both.
So, I don't get it. For those of you who don't believe that studies need to argue subjectively, what is it you're thinking?
---
Here's an example to let you be specific. It's an imaginary regression, where A, B, and C are used to predict X. I'm assuming .05 is the threshold for significance, but if you prefer a different level, feel free to change the p-values accordingly.
Here are the dependent variables, the coefficients, and the significance levels. An asterisk means the value is significantly different from zero.
A +0.15 p=0.05 *
B +0.13 p=0.08
C +0.16 p=0.04 *
What can you conclude?
Sure, you can say, "a unit increase in A was associated with a 0.15 increase in the dependent variable X, and that was statistically significantly different from zero." But that's not really a conclusion, that's just reading the results right off the regression. Papers wouldn't have a "conclusions" section if that was all they contained.
So, now, let me ask you: what would you write in your conclusions that's not subjective?
Labels: academics, regression, scientific method
posted by Phil Birnbaum @ 2/08/2012 01:04:00 PM
5 comments
5 Comments:
The biggest subjective part to any research is the *assumptions* part of it. Often, the biggest point of subjective contention are unstated assumptions. They are things like "the data sample is representative of the population" and "the underlying systemic relationships in the future are the same as for the past."
Researchers don't state assumptions for many reasons. One is that they will get exposed as untrue, or questionable at best, bringing down the house of cards that the study really is. Another reason is that because they are such tightly held beliefs, no one honestly expects anyone to disagree.
Un-articulated assumptions are where most debates really lie, and they're why so many arguments just go around in circles. Adversaries simply don't share each other's assumptions and never discover the true root of disagreement.
Most political arguments come down to assumptions. The left and right have diametrically opposed notions of human nature--un-articulated beliefs that form the basis from which all reason and logic flow. The left and right share the same facts (e.g. unemployment is high), they share the same logic (if a>b and b>c, then a>c), but they come to opposite policy conclusions. One side thinks the other is stupid, and one thinks the other is naive. They both think the other is corrupt. Without exploring our assumptions, those are the only remaining explanations for disagreement.
Research papers aren't much different. They are just facts processed by logic, based on models built from assumptions about the way the world works. And the conclusions in any paper are heavily colored by the author's assumptions. 'Peer-reviewed argument' is a good way of putting it.
-1 on me for leaving a comment longer than the original post!
No problem at all ... I do not share your assumption that your comment has to be shorter than the post!
Will think about what you wrote and reply when my ideas solidify better.
While it's probably true that most research papers are driven by subjective arguments, that doesn't necessarily mean that the researcher DOING the research has any subjective "stake" in the results. I work with a large database of medical records from a very specific and relatively rare disease, and so when anyone suggests some causal association between risk factors and outcomes I am able to set up an analysis to test it without making any of my own assumptions.
When I write a manuscript, I typically say "Our results suggest...". I never use the word "prove". Show me a paper (in science, anyway) that uses the word "prove", and I'll show you a bad paper. When discussing findings that don't have an obvious or easy explanation, I tend to use the word "speculate" frequently. "We speculate that this is due to..." You learn fairly quickly in peer review how to hedge your wording so as 1) not to piss people off and 2) to avoid future embarrassment if what you "proved" turns out not to be proved, after all.
Post a Comment
<< Home