An adjusted NHL plus-minus stat

There were a whole bunch of new research papers presented at last week's MIT Sloan Sports Analytics Conference. I actually didn't see any of the research presentations -- I concentrated more on the celebrity panels, as did most of the attendees -- but that doesn't matter much, because every attendee got an electronic copy of all the papers presented. Also, there were poster summaries of most of the presentations, with the authors there to answer questions.

Anyway, I'm slowly going through those papers, and my plan is to summarize a few of them here.

I'll start with a hockey paper. This one (.PDF) is called "An Improved Adjusted Plus-Minus Statistic for NHL Players." It's by Brian Macdonald, a civilian math professor at West Point.

In hockey, the "plus-minus" statistic is the difference between the number of goals (excluding power-play goals) a team scores when a player is on the ice, and the number the opposition scores when the player is on the ice. The idea is great. The problem, though, is that a player's plus-minus depends heavily on his teammates and the quality of the opposition. Even the best player on a bad team would struggle to score a plus, if his linemates are giving up the puck all the time and missing the net.

So what this paper does is try to adjust for that. The author took the past three seasons' worth of hockey data, and ran a huge regression, which tries to predict goals scored based on which players are on the ice. In that regression, every row represents a "shift" -- a period of time in which the same players (for both teams) are on the ice. The regression helps to estimate out the value of a player, by simultaneously teasing out the values of his linemates and opponents, and adjusting for those.

Another improvement that Macdonald's stat holds over traditional plus-minus is that he was able to include power play and shorthanded situations as well. He did that by running separate regressions for those situations, and combining them. Also, in addition to the identities of the players on the ice, he included one additional variable -- which zone the originating faceoff was in (if, indeed, the shift started with a faceoff).

Here are his results. The numbers are "per season", by which Macdonald means the number of minutes the guy actually played on average over the three years. The number in brackets at the end is the standard error of the estimate.

+52.2 Pavel Datsyuk (20.9)
+45.8 Ryan Getzlaf (19.6)
+45.3 Jeff Carter (15.7)
+43.0 Mike Richards (17.2)
+42.6 Joe Thornton (17.6)
+42.1 Marc Savard (15.5)
+40.2 Alex Burrows (13.3)
+40.0 Jonathan Toews (15.5)
+39.8 Nicklas Lidstrom (25.9)
+38.3 Nicklas Backstrom (18.4)

As you can see, the standard errors are pretty big. I'd say you have pretty good assurance that these players are good, but not very much hope that the method gets the order right. You look at the list and see Pavel Datsyuk looks like the best player, but with such wide error bars, it's much more likely that one of the other players is actually better.

The standard errors are large because there's not a whole lot of data available, compared to all the players you're trying to estimate. But why do the standard errors vary so much from player to player? Because they depend on how many different sets of teammates and opponents a player was combined with. The highest overall standard error was Henrik Sedin (+33.8, SE 27.0), because he and his twin brother Daniel "spend almost all of their time on the ice playing together, and the model has difficulty separating the contributions of the two players."

(If I'm not mistaken, this problem is why a similar Adjusted Plus-Minus technique doesn't work well in the NBA. With so few players on a basketball team, and most of the superstars spending a lot of time playing together, there aren't enough "control" shifts to allow the contributions of the various players to be separated.)

However, the imprecision doesn't mean the statistic isn't useful. It's still a lot better than traditional plus-minus. That may not be obvious, because traditional plus-minus doesn't come with estimates of the standard error, like this study does. But if it did, those SEs would be significantly higher -- and the estimates would be biased, too. As far as I know, Macdonald's statistic is the best plus-minus available for hockey, and the fact that it explicitly acknowledges and estimates its shortcomings is a positive, not a negative.

---

Oh, a couple more things.

Macdonald actually ran separate regressions for offense and defense (the numbers above are the sum of the two). It turns out that the way Datsyuk wound up leading the league, was, in large part, by virtue of his defense. His +52.2 is comprised of +37.8 offense and +14.5 defense. But Datsyuk's is not the league's highest defensive score: the superstar of defense turns out to be the Canucks' Alex Burrows, at least among the players Macdonald lists in the paper. Burrows looks like most of his value came on D: +21.3, versus +18.9 on offense.

And: where's Sidney Crosby, who's supposedly the best player in the NHL? He's down the list at +33.6: +36.4 on offense, and -2.8 on defense. But the standard error is 16.5, so if you tack on two SEs to his score, he pretty much doubles to 66.6. So you can't really say where Crosby really ranks -- it's still very possible that he's the best.

Also, the numbers make it look like Crosby is below-average on defense, which I suppose he might be ... but the relevant statistic is the sum of the two components, not how they're broken up. The idea is to score more than your opponents, whether it's 2-1 or 5-4.

Alexander Ovechkin is similar to Crosby: +38.7 offense, -1.5 defense, total +37.2. Nicklas Backstrom has the best power play results; Alex Burrows is, by far, the highest-ranking penalty killer. Download the paper for lots more.