Wednesday, June 16, 2010

Do younger brothers steal more bases than older brothers? Part II

A couple of weeks ago, I wrote about a study that purported to show huge differences in steal attempts between brothers in major league baseball. According to the New York Times article describing the study,

"For more than 90 percent of sibling pairs who had played in the major leagues throughout baseball’s long recorded history, including Joe and Dom DiMaggio and Cal and Billy Ripken, the younger brother (regardless of overall talent) tried to steal more often than his older brother."


If that's true, that would be huge. It would, I think, be one of the most surprising findings ever, that nine out of ten times the younger brother steals more than the older brother. But I checked, and found nothing close to that. I was left wondering how the authors, Frank J. Sulloway and Richard L. Zweigenhaft, got the results they did.

Since then, I managed to get hold of the actual study. And I still don't know where the 90% figure comes from.

Our raw numbers, it appears, are almost the same. The authors' results are different from mine, but only by a bit; they might have better data, or used different criteria for eliminating pairs from the study.

(Update: strikeouts below are where I misinterpreted some of the numbers in the authors' tables. The authors didn't actually give the numbers I thought they did.)

Let me start with steals. My numbers showed 56% of younger brothers outstealing their older brothers (adjusted for times on base). Their numbers actually showed only 48%. Pretty close. However, that's not the right comparison, because the authors are talking about steal *attempts* -- that is, SB+CS, not just SB. (I missed that the first time.)

Rerunning the numbers for attempts instead of steals, I get that 57 out of 98 younger brothers out-attempted their older brother, or 58%. What did the authors find? 97 out of 185, or 52%.

Putting that in table form:

58% me
52% them

So we're on the same page, right? I think so. It sure seems like their comparisons line up with mine.


So, then, why does the article say 90%? Because the authors, after showing 52% in their chart, change their estimate to 90% later.

Why? I can't figure it out for sure; I don't completely understand where they're coming from. The logic is flawed somewhere, but the authors don't explain themselves fully, so I can't actually spot where the flaw happens.

Let me show you what the authors give as the broad reason for bumping the 52% up to 90%: they say the sample is biased. Why?

"Call-up sequence and its relationship to athletic talent introduces a potential bias in athletic performance by birth order, as older brothers were more likely to be called up first owing to the difference in age. The extent of this effect turns out to be pronounced, with older brothers being 6.6 times more likely than their younger brothers to receive a call first to the major leagues. We are therefore comparing somewhat more talented athletes who were called up first (and who, by virtue of their relative age, tend to be older brothers) with somewhat less talented athletes who were called up second (and who tend to be older brothers). To correct for this bias, we have controlled performance data for call-up sequence in all analyses involving birth order."

(UPDATE: I had wondered why the authors assumed older brothers were better. I now see they ran a regression and found the older brothers had longer careers.)

But why is this bias?

Suppose you wanted to know if younger brothers grew up to be taller than older brothers. You measure a couple of hundred sets of siblings, and you find they grew to exactly the same height, on average. But, wait! At the time the older brother started high school, he's almost always taller than the younger brother! See? We have bias!

Well, of course, we *don't* have bias. Height (and talent) is related to age of the person, not calendar year. You'd expect brothers to be of equal height at equal ages, and of equal major-league experience at equal ages, NOT at equal calendar years. The "entering high school" and "getting called up" are just red herrings.

Regardless, the authors proceed to divide the sample of players into three groups:

-- younger brother called up first
-- younger and older brother called up the same year
-- older player called up first

If you do that, you'll find that the younger brother has a huge advantage in the first two groups. Why? It's obvious: if the a player gets called up before his brother *even though he's younger*, he's probably a much better player. In addition, speed is a talent more suited to younger players. So when it comes to attempted steals, you'd expect younger brothers called up early to decimate their older brothers in the steals department.

I ran my numbers for all three groups, and got:

-- 87% of younger brothers called up first attempted more steals (7/8)

-- 71% of younger brothers called up the same year attempted more steals (5/7)


-- 54% of younger brothers called up last attemped more steals (45/83)

The overall average, of course, is still 58% (57/98). But by splitting the data this way, it makes the effect seem larger. But that's not really telling you anything -- you're selectively sampling based on the results, because the younger players who come up first are precisely those players who you'd expect to steal only for reasons of being called up young.

Another analogy: home teams win 54% of games. But home teams that only use up one out in the ninth inning win 100% of games! That doesn't mean the 54% is biased, nor does it mean that you learn any more about home field advantage by breaking out walk-off wins. It's just another way of looking at the same result.

Anyway, having claimed to correct for a bias that I don't think really exists, they go on to compute odds ratios for the three cases separately, and use something called the "Mantel-Haenszel" technique to combine them. The statistical techniques look OK, and it seems to me that should have given them the same 52% that they found for the one group. But they somehow come up with 90%.

So, again: what's going on?

My only guess is that they somehow decided that the "younger brother called up first group" is the important one, and they just quoted that number. I got 87% for that, and their numbers are a little different, so maybe they got 90%. Another possibility: it might have something to do with the regression they used to correct for a bunch of things, including which "called up first" group the batter belonged to. They don't give the regression equation or the details, but depending on the way they chose dummy variables, I can see the regression looking something like:

Percentage of younger brothers outstealing older brothers equals:
-- 90%
-- minus 20% if the brothers were called up at the same time
-- minus 20% if the older brother was called up first.

So you can see "90%" coming up as a estimate in a regression equation. But still, the authors would certainly have noticed that you can't just quote the 90% figure without adjusting for the other dummies.

----

Here's another way they might have got it wrong. As you see in the quote above, the authors note that the older players turned out to play more games, and they were also likely to get called up first. So maybe they adjusted the numbers in the service of "controlling athletic performance" (the actual words they use).

Suppose that in 1950, older brother A stole 10 bases and younger brother B didn't steal any because he wasn't in the majors yet. If you "adjust" for that by including it in the regression, effectively subtracting 10 for every line of A's career, I can see how the "adjustment" might now show B having a 90% chance to steal more "adjusted" bases than A.

Again, I'm speculating, which I shouldn't. I don't think the authors did this, but I wouldn't be surprised if it's an adjustment something along those lines, just more subtle.

----

Another related stat the study comes up with is that younger brothers are 10.58 times more likely to steal a base than their older brother. I don't know where that one comes from either. Again, the authors' own raw data comes up with something much more realistic. As a ratio of times on base, the authors' SB+CS rates were

5.6% for older brothers
9.3% for younger brothers

That's only 1.66 times more likely.

So, obviously, the authors feel there's some huge bias here that, when you correct for it, brings the 1.66 up to 10.58. I can't imagine what that might be, and I can't really duplicate the authors' regression because they don't even give the equation.

Here are my calculated batting lines for the two groups. Is there really a factor of ten difference here?

------ AB -R --H 2B 3B HR RBI BB SB CS -avg RC/G
Young 553 70 146 26 04 11 062 50 12 05 .265 4.39
-Old- 546 75 148 26 03 17 074 54 08 04 .271 4.98


You'd think that they'd give a simple English explanation of how, when you look at the batting lines of the two groups, you see young players attempting steals at maybe 50% more than the older ones, but how when *they* look at the two batting lines, they see one line stealing bases at almost 1000% more than the older ones. But I couldn't find any such explanation in the paper.

So when they say,

"... younger brothers were 10.6 times more likely than their older brothers to attempt steals"

... well, I don't see any way that can possibly be true.

-----

UPDATE: batting lines updated, they were slightly off ... also updated to reflect the authors' explanation of something I had missed.

-----

ADDENDUM: as you can see, my data show the younger players indeed attempting more steals than the older players. In my sample, it's 46% more; in the authors' sample, it's 66% more.

My 46% seems like a lot ... I wondered there's perhaps a real effect there. So I ran a simulation to check for statistical significance.

For each of my 98 pairs of brothers, I *randomly* decided which one to consider as older, instead of looking at their real ages. Then I computed the relative attempt rate between the two random groups. I repeated that 60,000 times.

The results? 4,126 of the 60,000 random sets had one group attempting at least 46% more steals than the other group. That's a p-value of 0.069 ... not significant at 5%, but close.

I had expected more significance than that, but the players are "lumpy," in that some players steal a lot, and some don't. A few faster players landing in the same group can make a big difference.

-----

ADDENDUM 2: In the comments, I think Guy came up with the answer. Younger brothers have shorter careers. Shorter careers tend to be centered on lower ages (there are 40 year old players, but no 10 year old players). Young players are faster.

That's why younger brothers have more attempted steals -- they play more years during peak stealing age.

I bet if you did it season-by-season and controlled for age, most of the effect would disappear.

Good catch by Guy.





Labels: , ,

14 Comments:

At Thursday, June 17, 2010 12:13:00 AM, Blogger Phil Birnbaum said...

I should add that it's logically possible for there to be a strong effect that doesn't show up in the aggregate totals ... see "Simpson's Paradox".

But the authors don't imply or even mention the possibility, and I can't think of any plausible way such a thing could be happening in this case.

 
At Thursday, June 17, 2010 1:34:00 AM, Blogger Phil Birnbaum said...

Also, just noticed this, from another site, from "JHB":

"Also, it seems that the more likely the siblings were to steal bases, the more likely that the younger sibling dominates. If one sorts the player pairs in descending order of stolen base rates for the pair combined, the rate at which the younger sibling dominates is somewhat higher, as much as a five-out-of-eight chance of being higher, four-fifths of the way down the list. Where neither brother is much of a base stealer, it becomes roughly an even shot for either to be better. Where the numbers of steals are greater, the dominance of the younger sibling increases."

I haven't done what JHB suggests, but the five-out-of-eight (62.5%) he finds isn't much higher than the 56% I got in the previous post.

 
At Thursday, June 17, 2010 9:20:00 AM, Blogger Greg Finley said...

Very interesting stuff. I agree with you: their results seem highly suspect.

 
At Thursday, June 17, 2010 10:48:00 AM, Anonymous Guy said...

The slightly higher rate of attempts for younger brothers is interesting. This could partially be a playing era effect, as SBAs increased from the 1950s thru the 1990s, meaning younger brothers played in periods of higher SBA rates. But rates also dropped for periods early in the 20th century, so depends on distribution of the pairs.

There may also be a batting order effect. Since the older brothers are much better hitters, they likely hit higher in the order on average. That could mean they were more likely to have a runner on 2B in front of them, and/or be followed by better hitters in their lineups (making managers less likely to give the green light).

 
At Thursday, June 17, 2010 10:55:00 AM, Blogger Phil Birnbaum said...

I agree, the era effect is probably pretty small, if any. The difference should be only between the years at the beginning of the study (which would feature only older players) and the end (younger players). All the years in the middle would contain both equally.

Another way to look at it is that the better older players and faster younger players are opposite sides of the same coin. Just as pitchers are less likely to hit, so are fast baserunners, since they can make the team for other reasons.

And most of the effect is in the CS column, not the SB column. Is it possible that worse players have lower steal percentages in general, because they have to attempt more steals to show value to their team? Just a thought.

 
At Thursday, June 17, 2010 11:01:00 AM, Anonymous Guy said...

I forgot the most obvious explanation: the younger brothers have shorter careers (because they are worse players). SBAs peak very early, early- to mid-20s, then decline steadily. So the longer a player's career, the lower his career SBA rate will tend to be. That probably accounts for 90% of this.

Phil: I don't think the main difference is CS. You have younger bros. at 12 for 17, compared to 8 for 12 among older bros.

 
At Thursday, June 17, 2010 11:03:00 AM, Blogger Phil Birnbaum said...

Oh, geez, you're right! It's probably all about career length!

Nicely done.

 
At Thursday, June 17, 2010 12:05:00 PM, Anonymous Guy said...

Just as a reality check, is it easy for you to calculate mean ages (weighted by PA) for the younger and older brothers? I'd guess the younger brothers are about a year younger during their careers on average.

 
At Thursday, June 17, 2010 2:20:00 PM, Blogger Phil Birnbaum said...

Yeah, I'll do that tonight. I can also do young/old comparisons by age group.

 
At Thursday, June 17, 2010 7:37:00 PM, Blogger bradluen said...

The NYT article says:

"For more than 90 percent of sibling pairs... the younger brother (regardless of overall talent) tried to steal more often than his older brother."

Does the study actually say this anywhere?

The study does say:

"Using within-family contrasts... younger brothers were 10.6 times more likely than their older brothers to attempt steals..."

If you convert the odds ratio 10.6 to a probability, you get 91%. But I don't think it is a probability. It means, I think, that the proportion "attempts/opportunities" is 10.6 times higher for younger brothers than older brothers (after controlling for things that you may or may not actually want to control for).

Is the "more than 90 percent" statement a NYT gaffe, or have I missed something?

 
At Thursday, June 17, 2010 7:49:00 PM, Blogger Phil Birnbaum said...

bradluen: good point. I don't have time to reread extremely thoroughly now, but the only place I can find that they say that is

"Using our summary indicator of base-stealing proclivities [the sum of Z-scores for SB, SB+CS, CS, success %], younger brothers were 10.0 times more likely to achieve a higher score." [page 8]

But now that you mention it, that's probably not what they really mean. Because, for (say) SB+CS, they say,

"younger brothers were 10.6 times more likely than their older brothers to attempt steals."

But the calculation is the same and in the same chart, so how can they mean two different things?

But I think you're right that there isn't any actual data about the likelihood of which brother would have more SB+CS, except for that one quote. I will think a bit more and reread.

To answer your question: I don't think it's an NYT gaffe, I think it's a gaffe by the authors in the first quote above. Maybe.

 
At Friday, June 18, 2010 12:52:00 AM, Blogger Phil Birnbaum said...

OK, I've read the paper again, and I'm pretty sure the 10.6 figure is an odds ratio for the odds of the younger player attempting more steals over his career than his older brother.

The last paragraph on page 6 explains the regression.

However, I made a mistake in saying that the authors gave the overall unadjusted proportion of younger players who "beat" their brothers. I don't know where I got those numbers. I'll correct the post for that.

 
At Friday, June 18, 2010 2:44:00 AM, Blogger bradluen said...

If the goal is to find the odds of the younger player attempting more steals over his career than his older brother, I don't know how to use Mantel-Haenszel to calculate it without running into all kinds of dependence problems. There may well be (there probably is!) an appropriate method I haven't heard of, but it's not in the references I looked up.

***

For those that don't have the study, I think this sentence is the key to understanding the Simpson's Paradox (or perhaps Reverse Simpson's Paradox) going on here:

"This control is equivalent to directly comparing older and younger brothers who were called up during the same year (n 304), and then separately comparing older and younger brothers who were called up during the same year (n 74), and finally comparing all older and younger brothers who were called up after another sibling (n 304)."

 
At Friday, June 18, 2010 12:47:00 PM, Blogger Phil Birnbaum said...

brad,

Thanks ... I don't know anything about Mantel-Haentzel (although I once had a TA named Mantel -- not the same guy).

 

Post a Comment

Links to this post:

Create a Link

<< Home