The swimming psychology paper

In the previous post, I wrote about a paper (gated) that showed an anomaly in team swimming relays. It turned out that the first swimmer's times were roughly equivalent to his times in individual events -- but the second through fourth swimmers had relay times are significantly faster than their individual times.

The paper concluded that this happens because people put more effort into group tasks than individual tasks. They do this because other people are depending on their contributions. However, the leadoff swimmer's time is seen to be less important to the team's finish than the other three swimmers' times, and that explains why swimmers 2-4 are more motivated to do better in the team context.

My point was not really to criticize that individual paper, but to make a broader point -- that just because the results are *consistent* with your hypothesis, doesn't necessarily mean that's what's causing them. In this case, I agreed with an anonymous e-mailer, who speculated that it might have to do with reaction times. The first swimmer starts by a gun, while the other swimmers start by watching the preceding swimmer touch the wall. I said that I didn't know whether the authors of the paper considered this or other possibilities.

Commenter David Barry kindly sent me a copy of the study, and it turns out the authors *did* consider that:

"We corrected both performance times for the swimmer's respective reaction time by subtracting the time the athlete spent on the starting block after the starting signal (also retrieved from [swimrankings.net]). ... Please note, however, that previous research did not find any differences between the individual and relay competition after a swimming distance of 10m. Thus, faster swimming times for relay swimmers are unlikely to be merely due to differences in the starting procedure."

Fair enough. But ... well, the effect is so strong that I'm still skeptical. Could it really be that swimmers, who have trained their entire lives for this one Olympic individual moment, are still sufficiently unmotivated that they can give so much more to their relay efforts?

Here are the results for the four relay positions. (Times are an average of 100m and 200m):

#1: individual 78.19, team 78.38. Diff: -0.19
#2: individual 87.30, team 86.92. Diff: +0.38
#3: individual 87.73, team 87.39. Diff: +0.34
#4: individual 87.40, team 86.66. Diff: +0.74

It seems to me that the 2-4 differences are *huge*. Are the #4 individual swimmers so blase about the Olympics that they swim almost three-quarters of a second slower than they could if they were just more motivated? My gut says: no way.

One thing I wonder, following Damon Rutherford's comment in the previous post: could it be that correcting for the swimmer's reaction time to the starting gun isn't enough? Mr. Rutherford implies that the first swimmer's reaction time is for him to *start moving*. But he implies that subsequent swimmers are already well into their diving motions when the previous swimmers touch the wall. That could explain the large discrepancies, if the reaction time correction only compensates for part of the difference.

Is there anyone who knows swimming and is able to comment?

Oh, and there's one more issue with the differences, and that's a selective sampling issue. The authors write,

"We focused our analysis on the data from the semi-finals to obtain a reasonable sample size. If a swimmer did not advance to the semi-finals in the individual competition, we included his/her performance time from the first heats."

That means the individual times are going to be skewed slow: if the swimmer did poorly in the heats, his unsuccessful time is included in the sample. But if the swimmer did *well* in the heats, his successful result is thrown away in favor of his semi-final time.

That would certainly account for some of the differences observed.

Psychology should be your last resort

Bill James, from a 2006 article:

"... in order to show that something is a psychological effect, you need to show that it is a psychological effect -- not merely that it isn't something else. Which people still don't get. They look at things as logically as they can, and, not seeing any other difference between A and B conclude that the difference between them is psychology."

Bill wrote something similar in one of the old Abstracts, too. At the time, I thought it referred to things like clutch hitting, and clubhouse chemistry, where people would just say "psychology" as (in James' words) a substitute for "witchcraft." It was kind of a shortcut for "I don't know what's going on."

Today, it's a little more sophisticated. They don't say "psychology" just like that as if that one word answers the question. Now, they do a little justification. Here's a recent sports study, described recently in the New York Times by David Brooks:

"Joachim Huffmeier and Guido Hertel ... studied relay swim teams in the 2008 Summer Olympics. They found that swimmers on the first legs of a relay did about as well as they did when swimming in individual events. Swimmers on the later legs outperformed their individual event times."

Interesting! Why do you think this happens? The authors, of course, say it's psychology. But they have an explanation:

"in the heat of a competition, it seems, later swimmers feel indispensible to their team’s success and are more motivated than when swimming just for themselves."

OK ... but what's the evidence?

"A large body of research suggests it’s best to motivate groups, not individuals. [Other researchers] compared compensation schemes in different manufacturing settings and found that group incentive pay and hourly pay motivate workers more effectively than individual incentive pay."

Well, that paragraph actually makes sense, and I have no objection with the finding that group pressure is a good motivator. Still, that doesn't constitute evidence that that's what's going on in the swimming case. Yes, it shows that the results are *consistent* with the hypothesis, but that's all it shows.

You can easily come up with a similar argument in which the same logic would be obviously ridiculous. Try this:

I've done some research, and I've found that a lot of runs were scored in Colorado's Coors Field in the 1990s -- more than in any other National League ballpark. Why? It's because the Rockies led the league in attendance.

How do I know that? Because there's a large body of research that shows that people are less likely to slack off when lots of other people are watching them. Since Coors Field had so many observers, batters on both teams were motivated to concentrate harder, and so more runs were scored.

See?

The point is that, as Bill James points out, it's very, very hard to prove psychology is the cause, when there are so many other possible causes that you haven't looked at. When the Brooks article came out, someone e-mailed me saying, couldn't it be that later swimmers do better "because they can see their teammate approach the wall, and time their dive, as opposed to reacting to starter's gun"? Well, yes, that would explain it perfectly, and it's very plausible. Indeed, it's a lot better than the psychology theory. Because, why would later swimmers feel more indispensable to their team's success than the first swimmer? Does the second guy really get that much more credit than the first guy?

In fairness, I haven't read the original paper, so I don't know if the authors took any of these arguments into account. They might have. But even so, couldn't there be other factors? Just off the top of my head: maybe in later legs, the swimmers are more likely to be spread farther apart from each other, which creates a difference in the current, which makes everyone faster. Or maybe in later legs, each swimmer has a worse idea of his individual time, because he can't gauge himself by comparing himself to the others. Maybe he's more likely to push his limits when he doesn't know where he stands.

I have no idea if those are plausible, or even if the authors of the paper considered them. But the point is: you can always come up with more. Sports are complicated endeavors, full of picky rules and confounding factors. If you're going to attribute a certain finding to psychology, you need to work very, very hard to understand the sport you're analyzing, and spend a lot of time searching for factors that might otherwise explain your finding.

Your paper should go something along the lines of, "here are all the things I thought of that might make the second through fourth swimmers faster. Here's my research into why I don't think they could be right. Can you think of any more? If not, than maybe, just maybe, it's psychology."

If you don't to do that, you're not really practicing science. You're just practicing wishful thinking.