Corsi, shot quality, and the Toronto Maple Leafs

The Toronto Maple Leafs had a decent season in 2012-13, finishing fifth in the conference and making the playoffs for the first time since 2004.  But, perhaps, we fans of God's Team shouldn't get too optimistic.  For months now, hockey sabermetricians have been arguing that the Leafs were still a bad team -- a bad team that just happened to get exceptionally lucky.

But ... I've been fiddling a bit with the numbers, and I'm not sure I agree.

Before I get to my own case, though, let me tell you why the consensus says what it says.  First, Sean McIndoe has an excellent Grantland article that summarizes the issue.  Second, when you're done that, here's my own summary, which is a bit more statistical.  

-------

One of the new sabermetric statistics in hockey is the "Corsi" statistic.  Corsi is much like the NHL's official "plus-minus", but, instead of goals, it counts shots.  (Not just official shots on goal, but all shots directed at the net.)  

A player's Corsi is the difference between team shots for and shots against while he's on the ice in 5-on-5 situations.   Applied to teams, Corsi is just the difference between shots taken and shots allowed.  

The idea is, that there's a lot of luck in terms of whether shots actually go in the net.  So, instead of goals, you can better measure a team's talent by looking at shots.  It's the baseball equivalent of using Runs Created instead of runs scored.  In the baseball case, you eliminate "cluster luck" (as Joe Peta calls it), to get closer to true talent.  In the hockey case, you eliminate "bounce in off a player's butt luck" (among other randomness) to also get closer to true talent.

Corsi is a very good predictor of team success.  In one study, Corsi correlated with team standings points at r=0.62, which is pretty high.

So, the consensus is that if a team's Corsi doesn't really match their won-lost record, the difference is probably luck, and the team shouldn't be expected to repeat. 

Last year, Toronto did not look good in the Corsi standings.  In 5-on-5 situations, they took only 44.1 percent of the shots (meaning their opposition took the other 55.9 percent).  That was worst in the NHL.

So how did the Leafs win so many games, finishing in the top half of the standings?  Even though they took few shots, the shots they did take went in at an exceptionally high rate.  The Leafs had a 10.56% shooting percentage (goals divided by shots on goal), the highest in the league.  No other team was over 10.  The league average was roughly 8, with a standard deviation of roughly 1, so the Leafs were well over 2 SDs above the mean.

Now, you might be thinking: "Sure, the Leafs took fewer shots, but maybe it's just that they took BETTER shots, and that's why they did so well.  Corsi counts all shots equally, whether they're weak shots from the point, or point-blank shots with the goalie out of position.  How can you call the Leafs a bad team without also checking their shot quality?"

The sabermetric community responds that, if you look at the evidence, shot quality seems to be luck, rather than a skill that varies among teams to such a large extent.  If shot quality were actually non-random, a team with a high shooting percentage this year would tend to also have a high shooting percentage next year.  But that doesn't seem to happen.  One study, by Cam Charron, divided teams into five groups based on their shooting percentage this year.  The following year, all five groups were almost identical!  If you look at Charron's chart, there actually is a small effect that remains, but it's only about 10 percent of the original.  In other words, you have to discount 90 percent of the differences between teams.

Another study computed shooting percentages for individual players.  There were substantial differences, but: (a) only two players had shooting percentages higher than the entire Leaf team last year; (b) there's still luck in the individual numbers, so even those players probably don't have that kind of talent; and (c) those players may be taking the team's higher-quality shots because of their role, rather than because they create those shots.  So, it doesn't seem like the Leafs' 10.56% could be actual talent.

So, the argument in a nutshell: 

-- the Leafs took very few shots
-- teams that take very few shots are usually bad
-- the Leafs weren't very bad only because of their exceptionally high shooting percentage
-- an exceptionally high shooting percentage is usually luck

Therefore, the Leafs were probably just a bad team that got lucky.

------

I'm going to argue that that's not necessarily right.  There's another explanation that works just as well.

I'll give you that explanation now, in case you don't feel like reading the numbers to follow.  Actually, instead of the explanation, I'm going to give you an analogy, which might convey it in a more meaningful way.

A company pays its commissioned salesmen in cash, normally in 20 Euro bills.  The value of a Euro fluctuates, so the workers have learned that you can figure out who made the most money just by counting the bills.  If Joe has 35 bills, but Mary has 38, then Mary made more money than Joe.

One month, the paymaster happens to have some extra British currency he wants to get rid of, so he substitutes pounds for Euros in Mary's pay envelope.  A pound is worth more than a Euro, so instead of 42 banknotes of 20 euros, Mary receives 35 banknotes of 20 pounds.  Also, Mary wins the "salesperson of the month" award.

The sabermetricians seize on this.  

"We have found that the statistic called 'Borsi,' which is the number of banknotes received, is one of the most realiable indicators of sales performance," they say.  "But, this month, Mary's 'Borsi' was only 35."

"Sure, Mary won the award because her 35 Borsis were more valuable than normal banknotes.  But, our research shows that receiving British pounds is not a repeatable skill -- salespeople who receive pounds this month tend to revert back to Euros next month.  Therefore, you can't credit that to Mary's talent.  Therefore, she was lucky to make as much in commission as she did."

"In summary:

-- Mary had a low Borsi
-- salesmen with low Borsis are usually not productive
-- Mary did well only because her banknotes were worth more than normal;
-- receiving high-value banknotes is usually just random chance.

Therefore, Mary was lucky."

See the flaw?  Each of the four points above is actually true.  But what the analysis doesn't consider is that, even though receiving high-value banknotes is luck, there is a real, non-random relationship between that luck, and the number of banknotes received.  So, the analysis correctly adjusts for "high-value banknotes luck", but not the "too few banknotes luck" that corresponds to it exactly.

I suspect the same is true for Corsi.  It was luck that the Leafs scored on more than 10 percent of their shots, but that luck is actually tied to the fact that they took fewer shots.  

OK, here we go.

--------

If shooting percentage is almost all luck, the implication is that it's not something a team controls, or can even *choose* to control.  It's like clutch hitting -- just randomness that looks like there's something real behind it.

In that case, you'd expect every team to be around the league average of 8%, in all situations.  Shot quality must be about the same for all teams.  Intuitively, you might think some teams are good enough to have more breakaways and blind passes, while other teams take a lot of harmless shots from the point.  But, the data show otherwise.

Except that ... there ARE situations in which shooting percentage varies meaningfully from 8%.  For instance, a team's shooting percentage depends heavily on the score.  

Here are the situational averages for the six years from 2007-08 to 2012-13, with every team's six year total weighted equally.  

7.60% ... down 2+ goals
7.75% ... down 1 goal
7.52% ... tied
8.40% ... up 1 goal
9.19% ... up 2+ goals

(By the way, all the numbers in this post come from David Johnson's data pages at hockeyanalysis.com.  Thanks, Mr. Johnson ... never could have figured all this stuff out otherwise.)

Why such big differences?  My guess is ... when a team is behind in the game, it changes its style of play.  It probably presses a little more in the other team's zone, trying for better opportunities -- which is why its percentage rises a little bit.  On the other hand, when it presses more, that increases the chance of being caught behind on defense.  That gives the other team more chances at odd-man rushes and breakaways.  Which is why the opposition -- the team that's up 1 or 2+ goals -- sees its shooting percentage rise significantly, all the way to 9.19%.

I just made that explanation up, off the top of my head ... some of you guys know hockey a lot better than I do, so there's probably a better description of what's going on that would be more plausible to a real hockey strategist.

But, regardless of the details of the explanation: how you play does indeed seem to influence shot quality.  It's not all just random.

So: why isn't it possible that the Leafs' numbers are the result of style of play?  Couldn't they be deliberately playing the "up 2+ goals" style of play all the time?  If they're the only ones doing that, one team out of 30, it would be too small to show up in the statistical studies, and it would still look like shot quality is 90% luck.

I'm not saying they *are* doing that, just that it's *possible*.  

------

Now, when you look at the above numbers, you might think: it must be the team that's UP that changes the style of play.  Because, that's the team that looks like it gets the much bigger advantage in shot quality!  The team that's behind probably doesn't like it, but has no choice.

But then you'd ask the obvious question: if playing that style is so beneficial, why don't teams do it ALL THE TIME, instead of just when they're in the lead?

The answer is: it's not that beneficial.  The higher shooting percentage is offset by the fact that the teams in the lead take fewer shots -- that is, they have a lower Corsi.  Here are the percentages of (Corsi) shots taken based on score:

57.0% ... down 2+ goals
54.1% ... down 1 goal
50.0% ... tied
46.0% ... up 1 goal
45.1% ... down 2+ goals

This makes sense too.  If you're behind in the game, you have to concentrate on offense more than on defense.  The higher offense means you'll be taking more shots.  

But, as we saw, the lower defense means you'll be giving the other team better quality shots.  

The quantity and quality factors go opposite ways, and they roughly cancel each other out.  How do we know that?  First, it just looks like it to the eye; if you put the numbers together, one goes up roughly at the same rate that the other goes down:

57.0% ... 7.60% ... down 2+ goals
54.1% ... 7.75% ... down 1 goal
50.0% ... 7.52% ... tied
46.0% ... 8.40% ... up 1 goal
45.1% ... 9.19% ... up 2+ goals

But, more empirically, we know from goal scoring, which remains roughly even between the teams regardless of the situation.  Here are the rates of goals scored (team and opposition) per 60 minutes of even strength play:

2.42 - 2.32 ... down 2+ goals
2.39 - 2.26 ... down 1 goal
2.21 - 2.21 ... tied
2.26 - 2.39 ... up 1 goal
2.31 - 2.42 ... up 2+ goals

It's not perfectly even ... being down actually gives you a small advantage in future goals.  That might be random, but it might be real.  There's a plausible reason it might happen.  The team that's ahead has an interest in limiting scoring by wasting time.  It might be worth playing a style that gives the opponent a slight advantage in expected number of future goals, if that's offset by a lower probability of getting a goal in the first place.  

-------

So, I can say again: isn't it just possible that the Leafs are deliberately playing the "up 2+ goals" style during tie games?

Here's a comparison of the two sets of numbers.  The first is the "leading by 2+" from the above chart, and the second is last year's Leaf team with the score tied:

2+ games: 43.1% Corsi.  Leafs: 43.8% Corsi.
2+ games: 9.19% shooting.  Leafs: 10.82% shooting.
2+ games: 7.60% opposition shooting.  Leafs: 8.64% opposition shooting.
2+ games: 2.31 goals to 2.42.  Leafs: 2.96 goals to 2.80.

Three of the four numbers are roughly the right magnitude and direction: lower Corsi, much higher shooting percentage, slightly higher opposition shooting percentage.  The goals thing goes the wrong way, though. 

Still, reasonably consistent with the Leafs choosing to play a "concentrate more on defense and jump on opposition mistakes" style of game.  

--------

The situational numbers suggest that one way to lower your Corsi and raise your shooting percentage is to consciously choose defense over offense.  But it can also happen randomly, because opportunities are random.  Every player, every time he has the puck, has to decide whether to shoot or not.  Some days, you might have cases where the best option is to shoot.  Other days, you might have options where the best option is to pass, in hopes of a better shot.

Imagine a player has the puck.  If he shoots, he has a 5% chance of scoring.  If he gets the puck to his teammate on the other wing, the chance goes up to 10%.  Should he pass?  If he does, there's the risk that the defense will intercept the pass and take over.  Given these numbers, he should only choose the pass if there's a better than 50/50 chance it'll get through the defense.  

Now, options like that present themselves all the time, with different probabilities.  Sometimes, you have only a 15% chance of completing a pass (across the slot through a bunch of legs, say), but, if it works, there's an 80% chance of the shot going in.  Sometimes, it turns out nobody is open, and the 4% wrist shot from a bad angle is your best option.

All that, to a certain extent, is random.  Perhaps one day, by chance, everything is a shot.  You may take 60 Corsi shots, at 5% each.  Your shooting percentage will average 5% -- 3 goals in 60 shots.

The next day, randomly, everything is a 50/50 pass to a 10% shot.  You'll make 60 passes.  30 of them will be intercepted, and 30 of them will turn into Corsi shots.  Your shooting percentage will average 10% -- 3 goals in 30 shots.

Corsi will think your first day was a much better day: you had twice the shots!  But ... it wasn't.  One day you had more low-probability shots, and one day you had fewer high-probability shots.  One day you had 60 five-dollar bills, and the next day you had 30 ten-dollar bills.  

My example is too extreme ... your swings won't be that wide, from 60 passes to 60 shots because of random changes in offense and defense patterns.  But there will be SOME random variation in opportunities, which means there will be SOME random variation in Corsi that will move shooting percentage the other way.  

In this scenario, it is absolutely true that shooting percentage is not a repeatable skill.  But that doesn't mean that you didn't earn the extra goals.  You earned the extra goals because the "lucky" shooting percentage came from "unlucky" reductions in shots taken.

--------

Here's more evidence that Corsi and shot quality are inversely related.

I again looked at the last six years of the NHL, 180 team-seasons.  The correlation between shooting percentage and Corsi was -0.22.  When I took only the 36 most extreme shooting percentages, the correlation was -0.37.  

In the regression, each point of shooting percentage decreased Corsi by 0.785.  That means the Leafs' shooting accounted for around 2 points of Corsi, enough to move them from 44.1 (last) to 46.1 (third-last).

I think that's not enough of an adjustment, that the Leafs are more extreme than the regression suggests. That would be possible, I think, if the Leafs are different from other teams in some way other than random variation in shots.

But that's just my gut.  And I may be improperly biased by knowing, in advance, that they had a positive goal differential.  

--------

After looking at all these results, my overall view of Corsi is that it's decent enough to tell us something useful, but way too biased by situation and shot quality to be taken seriously on its own.  

As for the 2012-13 Leafs, I suspect shot quality is the biggest explanation for their low Corsi and high shooting percentage.  Not that I'm dismissing luck -- any time you have an extreme result, without a full explanation, luck is probably involved somewhere.  So, yes, I think the Leafs were a bit lucky in their shooting.  But, the rest of it, I think, was something real.  Specifically, I think it was some combination of:

1.  The Leafs playing a more defensive style, allowing them to capitalize better on opposition mistakes;

2.  Taking fewer low-percentage shots, and passing instead; and

3.  Random variation that made passes a better option more often.

And I say that because, from all the results I looked at, it seems that Corsi and shot quality are indeed related to each other.  If you accept Corsi, you can't dismiss shot quality.  To a significant extent, they're opposite sides of the same coin.  


(There are seven parts.  This is Part I.  Part II is next.)