Did Team Canada overestimate its players?

Joel Colomby, the fantasy columnist for the Sun chain of newspapers, has an interesting finding today showing how, in 2010, Team Canada's NHL players seemed to get worse after the Olympics.

Of nineteen of Canada's "top scorers," only five had a higher NHL points-per-game rate after the Olympics than before.  The other 14 dropped.

Colomby included only players still in the NHL in 2014, and who are "likely owned in a fantasy pool" this year.  That creates a selective-sampling issue, but if you take the results at face value, a 5-14 record is about 2 SDs from 50-50.

Looking at the other countries, there didn't seem to be any real effect.  It did vary from team to team, but, if you put all the other teams together, you get 16 of 34.  So:

 5-14 Canada

16-18 Rest of World

Going by actual performance differences, the results are similar.  The average player's points-per-game (PPG) dropoffs were:

-0.086 PPG Canada

-0.011 PPG World

By my rough estimate, Canada came in at roughly 1.8 SD from zero.  

------

These averages are from calculations I did by hand; Colomby gives only the individual player numbers.  He hints that we might see repeats for certain players.  For instance, "Leafs fans, and Randy Carlyle, have to hope Phil Kessel rebounds better than he did four years ago [-0.17]."

Of course, I don't think the individual numbers mean anything at all -- small sample size, as Colomby mentions.  But ... this got me wondering about something else.

The Canadian players dropped off in the latter part of the season.  Players who dropped off are more likely to have been lucky earlier in the season, just before Team Canada would have been selected.  

So: isn't it possible that the selection process was perhaps too influenced by randomness?   Could it be that GM Steve Yzerman and his staff put a little too much weight on players' recent "hot" performances, and wound up with a team that perhaps wasn't as good as it could have been?

That theory also explains why Canada's players dropped off more than the rest of the world's.  All the players on Team Canada came from the NHL, so Canada's roster is based on whom management thought were the best Canadian NHL players.  For other teams, the list of NHL players to draw from is smaller, so those decisions are a lot easier.  In fact, several of the teams would obviously need to include *all* of their NHL players, lucky or not.

Also, in this case, the Sun's selective sampling doesn't hurt this hypothesis, and may actually help it.  The players omitted from Colomby's list are the ones who are no longer regulars in the NHL.  Those guys, you'd think, would have been *more* likely to have declined after the Olympics, not less.

Are there other explanations?  Probably.  I bet some of the dropoff has to do with injuries.  The team is selectively sampled for prior good health (injured players don't go to the Olympics), so you'd expect a certain amount of dropoff regardless when those players later get their normal share of injuries.  (I wonder if that might be the source of the slight decrease for the other countries' players.)

And it goes without saying that it could just be random.

------

My gut says ... I bet the "luck" explanation is at least part of what happened, that Team Canada management wound up slightly overestimating players who were hot the first half of 2009-10.  

I could be wrong.  Those of you who know hockey better than I do, check out the list of players, and see if there are any questionable selections that seem to have been based on the player's uncharacteristically good recent play. I'll list the players for you (and also repeat the link).

Player                  Pre   Post  Diff

----------------------------------------

Sidney Crosby PIT      1.28 / 1.55 / +27

Patrice Bergeron BOS    .68 /  .79 / +11

Brenden Morrow DAL      .59 /  .65 /  +6

Jonathan Toews CHI      .89 /  .90 /  +1

Scott Niedermayer ANA   .60 /  .61 /  +1

Mike Richards PHIL      .77 /  .73 /  -4

Drew Doughty LA         .74 /  .67 /  -7

Eric Staal CAR         1.02 /  .95 /  -7

Patrick Marleau SJ     1.03 /  .95 /  -8

Corey Perry ANA         .95 /  .85 / -10

Duncan Keith CHI        .87 /  .76 / -11

Chris Pronger PHIL      .70 /  .59 / -11

Rick Nash CLB           .90 /  .77 / -13

Dan Boyle SJ            .80 /  .65 / -15

Shea Weber NAS          .59 /  .42 / -17

Joe Thornton SJ        1.21 /  .82 / -19

Dany Heatley SJ        1.06 /  .80 / -26

Ryan Getzlaf ANA       1.09 /  .80 / -29

Jarome Iginla CGY       .92 /  .60 / -32

-----

Of course, we will eventually be able to check whether the same thing happens this year.  Maybe, for the current season, we might also see an effect for Team USA.  According to quanthockey.com, there were 136 American players with at least 30 games played at the Olympic break.  With fewer than half the (309) equivalent Canadian NHLers to choose from, the USA might have faced fewer tough decisions, which means less reliance on luck.  But, it's still worth checking.

If the result repeats for 2014, we'd have the cleanest evidence I've seen that sports GMs fail to fully consider luck when predicting future performance.  We already have a strong intuition that happens, but it's been hard to tell for sure.  

All decent players get contracts, even unlucky ones.  So, if John Doe has a career year, we need to know not *whether* he was signed, but for *how much*.  And even then, it's orders of magnitude more difficult to compare performance to salary than it is to compare first-half performance to second-half performance.  

It's a small sample, but this time we have an unambiguous "yes/no" of whether Team Canada thought this player was among the best.  And it turned out that almost three-quarters of the players chosen had, at the time, been playing over their (later-selves') heads.  

Was Team Canada fooled by randomness?