Sunday, March 20, 2011

Psychology should be your last resort

Bill James, from a 2006 article:

"... in order to show that something is a psychological effect, you need to show that it is a psychological effect -- not merely that it isn't something else. Which people still don't get. They look at things as logically as they can, and, not seeing any other difference between A and B conclude that the difference between them is psychology."


Bill wrote something similar in one of the old Abstracts, too. At the time, I thought it referred to things like clutch hitting, and clubhouse chemistry, where people would just say "psychology" as (in James' words) a substitute for "witchcraft." It was kind of a shortcut for "I don't know what's going on."

Today, it's a little more sophisticated. They don't say "psychology" just like that as if that one word answers the question. Now, they do a little justification. Here's a recent sports study, described recently in the New York Times by David Brooks:

"Joachim Huffmeier and Guido Hertel ... studied relay swim teams in the 2008 Summer Olympics. They found that swimmers on the first legs of a relay did about as well as they did when swimming in individual events. Swimmers on the later legs outperformed their individual event times."


Interesting! Why do you think this happens? The authors, of course, say it's psychology. But they have an explanation:

"in the heat of a competition, it seems, later swimmers feel indispensible to their team’s success and are more motivated than when swimming just for themselves."


OK ... but what's the evidence?

"A large body of research suggests it’s best to motivate groups, not individuals. [Other researchers] compared compensation schemes in different manufacturing settings and found that group incentive pay and hourly pay motivate workers more effectively than individual incentive pay."


Well, that paragraph actually makes sense, and I have no objection with the finding that group pressure is a good motivator. Still, that doesn't constitute evidence that that's what's going on in the swimming case. Yes, it shows that the results are *consistent* with the hypothesis, but that's all it shows.

You can easily come up with a similar argument in which the same logic would be obviously ridiculous. Try this:

I've done some research, and I've found that a lot of runs were scored in Colorado's Coors Field in the 1990s -- more than in any other National League ballpark. Why? It's because the Rockies led the league in attendance.

How do I know that? Because there's a large body of research that shows that people are less likely to slack off when lots of other people are watching them. Since Coors Field had so many observers, batters on both teams were motivated to concentrate harder, and so more runs were scored.


See?

The point is that, as Bill James points out, it's very, very hard to prove psychology is the cause, when there are so many other possible causes that you haven't looked at. When the Brooks article came out, someone e-mailed me saying, couldn't it be that later swimmers do better "because they can see their teammate approach the wall, and time their dive, as opposed to reacting to starter's gun"? Well, yes, that would explain it perfectly, and it's very plausible. Indeed, it's a lot better than the psychology theory. Because, why would later swimmers feel more indispensable to their team's success than the first swimmer? Does the second guy really get that much more credit than the first guy?


In fairness, I haven't read the original paper, so I don't know if the authors took any of these arguments into account. They might have. But even so, couldn't there be other factors? Just off the top of my head: maybe in later legs, the swimmers are more likely to be spread farther apart from each other, which creates a difference in the current, which makes everyone faster. Or maybe in later legs, each swimmer has a worse idea of his individual time, because he can't gauge himself by comparing himself to the others. Maybe he's more likely to push his limits when he doesn't know where he stands.

I have no idea if those are plausible, or even if the authors of the paper considered them. But the point is: you can always come up with more. Sports are complicated endeavors, full of picky rules and confounding factors. If you're going to attribute a certain finding to psychology, you need to work very, very hard to understand the sport you're analyzing, and spend a lot of time searching for factors that might otherwise explain your finding.

Your paper should go something along the lines of, "here are all the things I thought of that might make the second through fourth swimmers faster. Here's my research into why I don't think they could be right. Can you think of any more? If not, than maybe, just maybe, it's psychology."


If you don't to do that, you're not really practicing science. You're just practicing wishful thinking.


Labels: ,

13 Comments:

At Sunday, March 20, 2011 6:14:00 PM, Anonymous Anonymous said...

As I see it, the most elemental definition of a game or sport is any activity in which skill is needed to persevere over psychological biases. After that, it's all scorekeeping.

 
At Sunday, March 20, 2011 6:27:00 PM, Anonymous kds said...

Phil, I'm not sure of this since I am not an expert on rules in swimming competitions. At the start of the race I am quite sure that swimmers are prohibitted from moving before the gun. They will be charged with a false start if they anticipate the gun. The later swimmers in a relay are prohibitted from leaving before the one swimming the previous leg touches the wall. I do not think that they are prohibitted from anticipating that touch. So they can leave several hundredths of a second earlier, maybe more than a tenth of a second.

 
At Sunday, March 20, 2011 9:34:00 PM, Anonymous Damon Rutherford said...

I talked to my wife, who swam for a Big Ten school, and my summary of her response is here:

The 2nd, 3rd, and 4th swimmers in the relay are free to move around on the block and get a "running start" as they dive into the water.

Also, as kds already stated, the swimmers can better anticipate when the previous swimmer's fingers will touch the wall.

And, the swimmers can indeed leave early when diving into the water, as only their toes need to still be in contact with the blocks when the previous swimmer's fingers touch the wall.

It was a NO-BRAINER to her that the latter swimmers in a relay will post better splits than they would have either swimming first or swimming in an individual event. And never did she and fellow swimmers think "psychology" was a factor in their better relay splits.

Her last sentence: "Idiots."

 
At Monday, March 21, 2011 5:37:00 PM, Anonymous Resolution said...

Isn't it a bit foolish to:

1. Assume that a proper scientific article that had to pass through peer review to get published wouldn't have addressed alternative hypotheses?

2. To fully doubt the findings of the article because a synopsis of the article in a newspaper doesn't get into the details?

3. To start this off by mentioning how people will often just attribute things to 'psychology' when they don't really know what they're talking about, yet then go into discussing an article never having read the actual psychology article? I mean, if you know anyone at all who is a college student, they could get you the article in about 5 minutes...

It could very well be a crappy article where they're dealing with an archival data set and aren't familiar enough with the nuances of actual competitive swimming. Or maybe they could have addressed the competing hypotheses, explained why those are valid and why this hypothesis may explain additional variance, and possibly even controlled for the running head start of latter swimmers...

 
At Monday, March 21, 2011 10:29:00 PM, Anonymous Guy said...

Resolution: So why don't you spend those 5 minutes to obtain the article, and tell us if the authors somehow controlled for the enormous advantage enjoyed by the 2nd, 3rd, and 4th swimmers? It's possible that they did, and that Brooks did a terrible job of summarizing their evidence. On the other hand, how likely is it that this team effect just happens to benefit only the same 3 swimmers who enjoy a big edge off the starting block? It's not even clear how one could control for that and detect some separate advantage shared by the same 3 swimmers. Seems pretty unlikely.....

 
At Monday, March 21, 2011 10:50:00 PM, Blogger Jarrod said...

This comment has been removed by the author.

 
At Monday, March 21, 2011 10:51:00 PM, Anonymous Anonymous said...

psychology is the study of human behavior. so, if you're trying to explain something in sports, which consists of a human (or team of humans) doing a several goal-oriented behaviors (e.g., dribble, avoid defender, pass, pump fake, shoot, rebound, put-back, score), it's all psychology.

 
At Tuesday, March 22, 2011 10:12:00 AM, Anonymous Guy said...

Excellent point, anon. If I could just find the right behavior modification program, I'm sure I could take Michael Phelps in the 100 meter butterfly. It's all in the mind!

 
At Tuesday, March 22, 2011 7:22:00 PM, Blogger David Barry said...

The authors of the swimming relay study subtract off the reaction times.

 
At Tuesday, March 22, 2011 8:32:00 PM, Blogger Phil Birnbaum said...

Thanks, David. Is the study available online?

 
At Tuesday, March 22, 2011 9:34:00 PM, Blogger David Barry said...

Not that I know of - I had to get a uni student friend of mine to access it for me. I can send it on to you if you'd like (is it [your surname] at sympatico dot ca?).

 
At Tuesday, March 22, 2011 10:07:00 PM, Blogger Phil Birnbaum said...

Yes, that's right. Thanks!!

 
At Wednesday, March 23, 2011 1:06:00 AM, Anonymous Andrew said...

Although I see your point, I think you could have done a better job of making it. For instance, the example you use about the Rockies could just have easily stated that pitchers pitch better in front of big crowds and therefore reduce runs being scored. In addition, a simple correlation between attendance and runs scored could be performed and we would likely see no correlation. The swimming study, while it certainly has flaws, is infinitely more plausible than the straw-man parallel you provided.

 

Post a Comment

<< Home