How can we separate referee bias from other sources of home-field advantage? Part II
You have two competing theories about home field advantage (HFA). Theory one is that HFA is almost all the result of referee bias. Theory two is that referee bias is part of the cause, but only a small part, and that there are other factors involved.
What kind of evidence would distinguish between the two? You need something that's independent of referee or umpire influence. That's harder than it looks. For instance, in the NHL, you'd expect most of the referee bias to come from penalty calls. However, even if you leave out power plays, you can see home teams doing significantly better than road teams. With both teams at full strength, the home side still outscores the visiting side.
Does that prove that something other than officiating is at work? No, it doesn't. Because it's possible that, knowing that the referees are likely to call more infractions against them, road teams are forced to play a less aggressive style of hockey, one that costs them goals even at full strength. So that part of HFA could still be referee bias at work, even if you don't see it in the penalty calls.
In order to more legitimately show that there are other factors at work, you need to find some statistic where there's no plausible story about how the refereeing could be the cause.
For basketball, we have foul shooting. The evidence shows that HFA in foul shooting exists, and is about what you'd expected it to be. Can we do the same for baseball? A couple of weeks ago, I asked for suggestions. Someone e-mailed me to look at fastball speed. That's a great idea, but I don't know where to find data (can anyone help)? So, I tried something else: wild pitches.
It's hard to see how a wild pitch (or passed ball) could be the result of umpire bias. It's a straightforward call based on what happens on the field. I suppose that it's possible that if the catcher tries to throw out a baserunner, and the umpire is more likely to call that runner out, that would be a bias for the home team (since a WP is not charged when a runner is thrown out at the next base). But that happens so infrequently that it's not an issue.
Also, it has been found (by John Walsh, in the 2011 Hardball Times Annual), that home plate umpires favor home teams in their pitch calling (by 0.8 pitches per game). And so, you could argue that maybe visiting pitchers have to pitch a little differently to overcome that disadvantage, and that's what causes any increase in wild pitches.
That's possible. However, you'd expect any effect to go the other way, to *fewer* wild pitches, wouldn't you? If the strike zone is smaller for the visiting pitcher, he's more likely to compensate by generally being closer to the strike zone. It's hard to see how that would lead to more pitches in the dirt.
So, all things considered, I think wild pitches are a pretty good candidate. For all MLB games from 2000 to 2009, I figured how many wild pitches were thrown by home teams and road teams. I omitted wild pitches on third strikes, for reasons I'll explain later.
The raw numbers:
Road: 6943
Home: 6829
So far, it looks like a definite HFA exists. But, since a wild pitch can only occur with runners on base, maybe it's just that home teams aren't in that situation as much as visiting teams (since home teams generally pitch better). Also, maybe road teams throw more pitches per batter than home teams, for whatever reason.
So I adjusted for the number of pitches thrown with runners on base (leaving out intentional balls). The results, per 100,000 pitches:
Road: 460 WP per 100,000 pitches
Home: 448 WP per 100,000 pitches
That's a 2.7% difference, which is reasonably large. For passed balls, the difference was even wider:
Road: 97 PB per 100,000 pitches
Home: 91 PB per 100,000 pitches
The difference is 6.6% this time.
In case you're interested, there were 1,506,500 pitches for the visiting team, and 1,522,647 pitches for the home team. The difference is mostly the home team always having to pitch the ninth inning, partially offset by the fact that the home team faces proportionally fewer situations with men on base.
-------
One other possible objection is that the probability of a wild pitch could vary by count. Maybe "wasting a pitch" on 0-2 leads to more wild pitches than at other counts.
So, I rechecked, but included only pitches at 0-0 counts, which seems like a reasonable control. The results:
Road: 428 WP per 100,000
Home: 401 WP per 100,000
Road: 116 PB per 100,000
Home: 110 PB per 100,000
Very similar. I'll check the statistical significance before I present this at SABR next week, but I'm pretty sure that the chance of this happening by luck is pretty low.
-------
What does this mean for HFA?
For the 1988 American League (which is all I have handy right now), the linear weight for a wild pitch was about 0.27 runs. For the entire sample, the total difference was 181 pitches (combining WP and PB). So that's 49 runs, which is about 5 wins.
5 wins divided by 30 teams, divided by 10 years, divided by 81 home games, equals 0.0002 wins per game. Since the total HFA per game is about .040, that means that wild pitches (+PB) make up one-half of one percent of home field advantage.
That seems reasonable to me.
So, I think we have some good evidence that there's HFA in aspects of the game not influenced by the umpire -- namely, wild pitches.
Labels: baseball, referee bias, wild pitches
posted by Phil Birnbaum @ 6/26/2011 11:10:00 PM
18 comments
18 Comments:
Phil, I have data on fastball speeds, and I can get it to you. High-level summary: pitchers throw slightly harder on the road (not sure if the difference is significant).
Also, I've mentioned this here before, but I don't believe John Walsh's study is the best one for establishing home field advantage for strike calls. Dan Turkenkopf and J-Doug Mathewson have done some good work on this topic. I can dig up those links again if you want.
Hi, Mike,
Sorry, I don't remember you mentioning those links before, or commenting on John Walsh's study. My bad. If you send those links again, I promise to take a look at 'em, I swear.
Really ingenious idea, Phil.
I've been working with a comprehensive model of strike-calling (at least trying as hard as possible to be comprehensive, and my main interest is not HFA). While I'm not ready to publish the results, the bias for home field in my model looks to be about 0.8% to 1.3% for the home team (that's fixed umpire effects, controlling for location and a slew of other factors).
That's between 1 and 2 pitches per game. If I'm reading the other analyses right, it makes for about one run every 4 or 5 games, rather than 8. I think that would make up about 25% of HFA assuming the estimations are in the ballpark of being correct. Given that there are 3 models saying otherwise (that it's closer to 15%), I'm still checking the validity of the model.
I really like this idea though, as it seems to give some idea of how comfortable a pitcher is out there on the mound. Whether it's the mound, the crowd, or the jet lag, who knows.
Have fielding guys looked extensively at Home/Away fielding splits for players? That may be another area to look at which would be irrespective of the umpires (and obviously, not using errors, as those are subject to much home field bias).
Millsy,
The problem with fielding is that it depends on the quality of the batted balls ... and home batters hit better balls than visiting batters. So there's some confounding there.
If you had field f/x data, maybe you could compare identical trajectories, but the HFA you're looking for would be so small that you'd need the data to be very, very accurate in order to be able to trust the results.
Good point. I imagine it would be possible with high quality batted ball data (i.e. Trackman or Hit F/X type stuff like you say).
I'll be very interested to see the results and details of your study, Millsy, when you are ready to publish it.
J-Doug's study was reported here:
http://www.beyondtheboxscore.com/2011/1/29/1961942/strike-zone-a-marginal-component-of-home-field-advantage
J-Doug links back to the Scorecasting post at this blog which mentions Dan's study, which was here:
http://www.beyondtheboxscore.com/2008/5/12/506919/a-nibble-here-a-nibble-the
I wonder, too, if what we are counting as umpire bias here doesn't also include catcher and pitcher skill in targeting and framing the pitch. We have good reason to believe that umpires call the strike zone relative to the catcher's target. If catchers do better at taking advantage of this at home, and pitchers do better at hitting that target just off the corner when they are at home, we could see a home/road change in the strike zone even if umpires were entirely consistent in how they called the zone. Not consistent in terms of a fixed-size spatial boundary, but consistent in their methodology of judging the player performance, where the change in player performance is what drives the changing size of the zone because umpires do not (and probably cannot) call the zone to some abstract invisible set of planes in space.
Mike,
Outside of player ability (pitcher and batter), I'm not sure how I would control for that catcher spotting ability. Though, I agree with you on that point. I do include current season (end of season, as a proxy for overall skill) pitcher and batter WAR, so perhaps that can pick up some of the pitchers' ability to spot the ball.
I should correct myself a little bit here. The effect I report above is *at most*. The effect I report is for pitches at about 50-50 strike-ball percentage in the pooled sample. The reason I think mine is higher is because I am reporting this value, rather than the average marginal value across all pitches. Was not trying to be misleading, but realized this point after I posted.
So I may have incorrectly applied the difference to all pitches, when it should only be for 'borderline' pitches called in the game. Depending on the number of pitches in the area where the difference in home-away strike-calling applies, it may be less than 1 pitch per game. Depending on the homogeneity of HFA across all pitches, the overall effect on the game could change dramatically.
After a little more thought, I suspect that what I'm reporting is in fact almost exactly what those other studies find. After all, a pitch 10 feet high is a pitch 10 feet high...and for the most part a pitch down the middle is a pitch down the middle. I would be nearly 100% certain that the HFA difference is not homogenous across the entire sample of pitches.
Regarding fastball speeds, visiting starting pitchers average fastball speed from 2008 through mid-May 2011 was 90.64 mph, compared to home starting pitchers at 90.62 mph.
The home pitchers throw harder in the first inning, 90.91 to 90.78 mph and marginally slower than the visiting pitchers in every inning after the first (collectively 90.61 mph for the visiting starters and 90.55 for the home starters).
Of the 153 starting pitchers who threw at least 1000 fastballs during this period, 70 of them threw harder on average at home, and 83 of them threw harder on average on the road.
Visiting relievers also throw harder in aggregate than home relievers in any given inning (except the 6th, where the difference is 0.01 mph in favor of the home pitcher), though with relievers it is a lot harder to disentangle differences in role, quality, score, etc., than it is with starters. The average difference is about 0.14 mph in favor of the visiting relievers when comparing on an inning-by-inning basis. Even that comparison is probably not free of reliever-quality bias, however.
Mike,
My unscientific opinion is that framing the pitch is something that wouldn't vary much between home and road, since it's more a decision than a physical attempt. That is, you wouldn't be expecting a catcher to try to be framing a certain way, but failing to do so because he just isn't good enough today.
I mean, what you suggest is possible, but I just don't see it. I might be wrong; feel free to convince me.
Interesting that fastball speeds go the "wrong" way! Maybe visiting pitchers get into trouble more, and thus have to throw harder to get out of innings? It's established that pitchers exude more effort in crucial situations than otherwise, right?
nice work
For basketball, we have foul shooting. The evidence shows that HFA in foul shooting exists, and is about what you'd expected it to be.
Yeah, I don't know about this. I calculcated the difference between all players home and away FT% from 1987 to 2010 - the mean difference was .002%. (Here's a plot.) I also ran a mixed effects regression on the same dataset, allowing the intercept and slope to vary by player-season -- the mean of the coefficients for HCA was even tinier.
Anonymous: Yes, .2% is what we got too. It's statistically significant and about the right size. See the post I linked to, and the comments.
Phil, is there evidence that the HFA is mainly influenced by physical factors rather than mental ones or decision-making? I wasn't aware that that had been established, and it's at least not obvious to me.
It's widely believed that pitchers try to throw harder when they are in crucial situations. I've not been able to find evidence of that. They don't particularly throw harder from the stretch than from the windup.
>"is there evidence that the HFA is mainly influenced by physical factors rather than mental ones or decision-making?"
No, it's probably both. But HFA probably applies to things that are very difficult to do with precision ... HFA probably increases the SD of accuracy (mentally or physically) by a small amount.
It doesn't seem like glove positioning is one of those things. It's like, if you gave a speed multiplication table quiz home and road, the home would probably be better than the road. But if you gave an UNTIMED multiplication table quiz home and road, the results would probably be 100% both ways.
Glove positioning before the pitch seems more like an untimed task than a pressure task.
I will be the first to say I don't have a good feel for differences between home and road. My limited amateur athletic experience is probably not very applicable to major professional sports.
However, on the topic of strike zone, maybe I can add something. I see three basic components to be executed by the players: (1) catcher sets the target, (2) pitcher delivers the pitch at some location relative to the target, and (3) catcher receives the pitch into his glove.
Here's how I speculate that HFA could affect each of those three pieces:
1. Catcher is probably relatively unaffected by HFA in his ability to place the glove where he wants. But he might very well be affected in his decision-making about what the right place is to put the target in the first place. Whether it's being more rested by sleeping in his own bed or feeling more confident because the fans are cheering his pitcher or he's more likely to be working with a lead, he may make more correct decisions about how to gain a few extra strikes for his pitcher. I'm not sure how one would measure that.
2. Pitchers may have better mechanics at home such that they have better control over where the pitches go. This is probably measureable.
3. The catcher's mechanics for receiving the pitch may be sharper at home. He may be more confident about how the ball will bounce off the dirt if he has to block the pitch or other factors that help him to be more relaxed as he receives the pitch. Anecdotally, umpires react poorly on giving strikes when the catcher is reacting to his pitcher rather than seeming to be in control. It's possible to observe from video the catcher's mechanics in receiving pitches, but doing that on a large scale is probably impossible. Short of that, I don't how to measure this.
Mike,
All fair enough. My strong intuition is that the most important home/road differences are in the skills where a small change in ability makes a big change in results. That would be placing the pitch, reacting to the pitch, and swinging the bat. In all three of those, 1/100 of an inch difference can be HUGE in terms of outcomes.
But, in terms of placing the glove and receiving the pitch, a 1/100 of an inch difference doesn't seem to me to be very material.
That is: I'm willing to believe the H/R difference in glove position is similar to the H/R difference in bat position. But it just makes so much more difference in bat position that it would dwarf what the catcher is doing.
If that makes sense.
Mike, could I get that pitch speed data you offered? My e-mail is available at my website, www.philbirnbaum.com .
Post a Comment
<< Home