## Tuesday, June 26, 2007

### How important is a hot goalie in the playoffs?

How important is a hot goalie in the NHL playoffs? Conventional wisdom says: very important. A study by Alan Ryder says: not as important as you might think.

Ryder collected shot quality data for the first 46 games of this year's playoffs. He looked at not just the raw shot numbers, but the type of shot and distance from the goal – for instance, a wrist shot from 50 feet out. From that data, Ryder calculated the expected number of goals each team "should have" scored from shots of that quality.

In the 46 games studied, Ryder found that the team that was expected to win actually finished with a record of 38-8. In those eight losses, goaltending likely made the difference. Examining those games more closely, Ryder concludes that four of those games were goalies blowing a win, so that leaves only four goalies getting hot and stealing the win. He writes,

" ... we have seen exactly four games stolen by goaltenders in 46 playoff games to date. Strong goaltending is critical to playoff success. And there has been some tremendous play in the blue paint this spring. But the conclusion here is obvious. With only four steals, goaltending is not running the results board."

While Ryder's finding is important, that the best team (in terms of shot quality) usually wins, I'm not so sure that the influence of goalies is as minor as Ryder implies. First, aren't four games blown just as important as the four games stolen? A "hot" goalie is judged not just by having lots of great games, but also by having few poor ones.

Also, Ryder tells us that 8 games in 46 is about one game per series (although I'm not sure which series he used to add up to 46 games). One game per series is a lot, isn't it?

How much is one game worth, assuming two teams of equal ability?

If a team starts out 1-0, it has a 42/64 chance to go on to win the series. So if it starts out 0-1, its chance is 22/64.

Turning a loss into a win, then, is worth 20/64 of a series, or .3125. That's a lot, almost a third of a series!

And it's twice the value of an overtime goal (which only turns a tie into a win). Even if a team scores two overtime goals in a series, it's probably different players scoring them (and goals are a team effort anyway). So it does seem that a goalie having a hot game does have has a disproportional influence on the series.

Having said that, it's probably not fair to credit (or blame) the goalie any time the "wrong" team wins in terms of shot quality. Not every 30-foot snap shot is the same, and it's very possible that the team with the lower number of expected goals actually had better chances. So perhaps the number of games where the goalie made a difference is overestimated by this method.

And, of course, a large part of goaltending (and batting, and pitching, and quarterbacking, and foul shooting) is random chance. If Domenik Hasek steals a game with some spectacular saves, it's probably mostly luck, rather than a real increase in his usual (high level of) skill. For instance, suppose the Sharks have four point-blank chances, each with a 50% chance of being a goal. By luck, Hasek stops three of them, and thus saves one expected goal. The Wings win 3-2 in overtime. The goalie is the hero, and it looks like Hasek's clutchness was critical, but it wasn't – he was just luckier than usual.

Unless, of course, you believe in "clutch" performance among goalies. I'm naturally skeptical, given the evidence against clutch performance in baseball.

Finally, here's one more argument.

In baseball analysis, the WPA approach (as calculated by
Fangraphs and others) calculates the change in probability of winning after each event. For instance, if player X hits a three-run home run in the top of the ninth in a tie game, he might increase his team's chances of winning from (making these numbers up) 60% to 95%. You might be tempted to say that X made the difference in the game. But if the other team's player Y hits a grand slam in the ninth, he might have increased his team's chances from 40% to 100%.

In an up-and-down game, the sum of the probability changes might be very large – easily well over 100%. And so looking at just one play might give you an overestimate of how important that play was.

Same thing with the goalie. A goalie "steal" might be worth 31% of a series. But if you add up 15% for each overtime goal, and, in fact, all the probability changes for every goal, you might find that the sum of all the changes is 400% of a series for one team, and 300% of a series for the other team. In that light, 31% of a series for the goalie might be huge, but not *that* huge.

So there are several things to consider, and, overall, I don't know what to conclude about all this. My gut says that goaltending is important, but overrated because, with the goalie on the ice the entire game, his luck is exceptionally visible. All the other luck – offensive and defensive – is spread among the other players on the team, and is a combination of their efforts.

I guess I agree with Alan Ryder. Hot goaltending is important, but not as important as a first impression might suggest.

Labels: , , ,

At Wednesday, June 27, 2007 11:49:00 AM,  Unknown said...

I think you are overestimating the impact of winning a game on the odds of winning the series. A team that is 3-0 up that loses a game will have a much lower swing in win probability than if it was, say, 2-2, or 0-0 (as in your example.

Perhaps when you weight it all up the answer is similar to what you had.

At Wednesday, June 27, 2007 3:22:00 PM,  Phil Birnbaum said...

>Perhaps when you weight it all up the answer is similar to what you had.

I think that's what happens.

Here's another way to think about it. It takes about three wins to turn a series loss into a win. Since half the series are won anyway, it takes about 1.5 wins *in a series you lost* to retroactively turn it into a win.

If you lost 4-3, turning 1.0 losses into a win makes it 4-3 for you and gives you the win.

If you lost 4-2, one win makes it 3-3, and now you need another half win, so it's 1.5 wins to turn the series around.

If it was 4-1 or 4-0, it takes even more wins: two wins to turn around 4-1, and 2.5 wins to turn around 4-0.

So: 1 win for 4-3, 1.5 for 4-2, 2 wins for 4-1, and 2.5 wins for 4-0. The average is 1.75 wins. But, taking into account that 4-3 and 4-2 happen more often than 4-1 and 4-0, that drops the average to 1.5.

At Monday, July 02, 2007 1:52:00 PM,  Anonymous said...

Back on the old version of my Hot Hand website (and thus not shown on my new version), I did an analysis related to the idea of hot goalies.

To summarize briefly, I was interested in two issues. One was goalies' improvement in Goals Against Average from the regular season to the play-offs, and the other was year-to-year correlation in play-off GAA. The latter would get at any stable ability on the part of goalies to raise their level of play in the post-season.

For each elgible goalie in each year I studied (2001, 2002, and 2003), I computed a difference score (play-off GAA minus regular-season GAA), where a negative score would indicate improvement.

I then computed year-to-year correlations. There was substantial stability from 2002 to 2003 (correlation of .662), but less from 2001 to 2003 (.327) and 2001 to 2002 (-.269).

The sample sizes were very small and so these results are not statistically reliable, but there is some suggestion that the same goalies tend to get hot for the play-offs, year to year.

One exception was Carolina Hurricanes goalie Arturs Irbe, who recorded a 3.33 play-off GAA in 2001, but then went down to 1.67 in the 2002 play-offs in leading the Hurricanes to the Stanley Cup finals.

There's also an article entitled, "It Takes a Hot Goalie to Raise the Stanley Cup," by D.G. Morrison and D.C. Schmittlein, published in Chance magazine in 1998. This article takes yet another approach.

At Monday, July 02, 2007 8:35:00 PM,  Phil Birnbaum said...

Very interesting! I wouldn't have expected a high correlation, especially because of the small sample sizes. Is your study available on the web?