Pete Palmer

Whenever someone mentions David Romer and fourth downs, I think of Pete Palmer, and how he might be the most under-appreciated sabermetrician ever.

While Romer gets all the mentions, Pete was actually first to figure out that NFL coaches are too conservative.  I have a copy of the 1998 edition of "The Hidden Game of Football" (which Pete wrote with Bob Carroll and John Thorn).  Chapter 10, "Kicking Up a Storm," goes through the logic of when you should go for it on fourth down, as opposed to punting or trying a field goal.  Like Romer, Pete finds that teams should try for the first down more often.  One of Pete's many conclusions, just as an example:

" ... you should NOT kick a field goal unless you have six or more yards to go on fourth down.  And if you're inside your opponent's 10-yard line, you shouldn't kick no matter what the distance."  

Romer cites the Palmer chapter in his paper.   He reports that the book's method yields "implausible results," but isn't specific about which results. I think some of the differences come from assuming different values for field position: Romer's data comes from some fancy math with quadratic spline curves, while Palmer's comes from 1997 play-by-play data.  I discussed some of the differences in my blog post on the subject.

But, I've digressed ... my point is not to analyze who's right, just to point out that Palmer had done roughly the same thing, but is barely remembered for it.  Part of the reason, as far as the mainstream press is concerned, might be that Pete is just some guy who wrote a book, whereas Romer is instantly credible as a Ph.D. economist.  Still, my impression is that Palmer gets doesn't get as much recognition even within the football sabermetric community.

In fact, I can't believe "The Hidden Game of Football" gets so little mention at all.  It was the first sabermetric analysis of football I'd ever seen, when the first edition came out in 1988.

------

This "Pete Palmer wrote a book and nobody notices" thing happened again a couple of years ago.  Pete and Dave Heeren combined on "Basic Ball," a book that combined baseball, football, and basketball (Heeren wrote the basketball part, Pete the baseball and football).  I reviewed the book for "By the Numbers" (.pdf).  After my review appeared, Tom Tango wrote, 

"I’m as big a fan of Pete Palmer as there is (which is why we asked him to write the foreword to The Book).  And I had no idea he had a book out since last September.  And I know I’ve corresponded with Pete a few times since, and he never said anything to me."

Commenters at Tom's post note that Pete is very humble and doesn't do much self-promotion.  That's not really the point of this post, Pete's character, but ... if you ask around, almost everyone who's encountered Pete has stories about what a nice guy he is.  For my part, Pete has been exceptionally kind to me, and has gone out of his way for me more than once.  And I don't even know him that well.

------

A few years ago, Tango wrote about the method of finding true talent levels for teams in various sports.  Basically, you look at the overall variance in performance, you subtract the theoretical (binomial) variance that would happen if all teams were the same, and that leaves you the talent variance.

It's simple, but I'd never thought of it, and I started calling it "Tango's method". 

Well, again, Pete was there first.  In a guest chapter of "Baseball Hacks," which came out a few months before Tango's post, Pete describes the method and some applications, and does a little study (see "Hack #68").  And the thing is -- I had actually read that book, and missed Pete's contribution completely.  

If you're a programmer, you'll love seeing how Pete is an engineering geek, and from a different generation than most of the rest of us ... in the book, Pete gives us the computer program he used for his study.  It's written in Fortran.  It uses single-letter variables.  It doesn't indent for structure.  And, it's got GOTOs all over it.  

And this in a book that uses the "R" language for everything else!

For those of you who aren't programmers ... it's like walking into an Apple Store, and one of the techs at the Genius Bar pulls out a 1985 cell phone, the size and shape of a brick with the 12-inch antenna.  And he's not using it as a joke -- hey, he's been using it for 25 years, and he's used to it, and it does the job!

And I'm not making fun of Pete, here, by any means ... I do a lot of my simulations using a version of Microsoft QBASIC from the late 80s ... it comes in one .EXE file, and every time I install a new version of Windows, I just copy it over.

------

And, finally, one more story.  A couple of years ago, I posted an illustration I thought of on why, in baseball, 10 runs equals 1 win.  Tango hadn't seen that particular method before, and e-mailed Pete about it.  

Is it rude to quote a private e-mail?  Well, paraphrased, Pete wrote back something like, "yeah, I actually figured it out that same way years ago ... I guess maybe I should have mentioned it!"

-------

David Romer writes about being more aggressive on fourth down; Pete had already said the same thing.  Tango writes about variances and team talent; Pete had already said the same thing.  I write an explanation of 10 runs = 1 win; Pete had already figured out the same thing.

Pete, you need a publicist!

Corsi, shot quality, and the Toronto Maple Leafs, part V

In the past four posts, I speculated that NHL teams may vary in shooting percentage partly because they take different quality shots.  I also speculated that, maybe, their shots vary in quality just randomly.  

But I hadn't checked whether the numbers would work out consistent with plausible limits on how teams might vary.  So, to check for reasonableness, I created a simulation, and played around with the specifics until I got something that looked reasonably like the 2012-13 season.

The simulation worked like this.  Teams vary in "possession" talent, because some teams are simply better than others.  Their talent is the number of times they get the puck into the opponent's zone for a decent chance to shoot.  

Once they get into the zone, they all behave identically.  Sometimes, the shoot right away.  Most of the time, though, they move the puck around trying to get a higher-quality shot.

When they enter the zone, they have a 3.2 percent shooting percentage if they shoot right away.  But, they first decide if they want to shoot or pass.  Randomly, 17 percent of the time, they shoot.  The other 83 percent, they pass.  If they pass, it's successful 70 percent of the time; the other 30 percent, they lose the possession.  If the pass does work, the shot quality improves by the inverse of 70 percent; that is, it goes from 3.2 percent to 4.57 percent (since 4.57 percent is 3.2 divided by .7).

Actually, I set it up so that you keep deciding whether to shoot or pass -- shooting 17 percent of the time, and passing 83 percent.  You could wind up passing 2, 3, 4 or more times before shooting.  I limited it to 7 passes, then you always shoot.  (After 7 successful passes, your shooting percentage is 38.9%.)

No matter how many times you try to pass before shooting, your expectation of scoring is the same: 3.2 percent.  If you shoot every time, you score 32 goals per 1000 possessions.  If you pass and then shoot, you wind up with only 700 shots in those same 1000 possessions, but you still score 32 goals on those shots (since your shooting percentage has improved to 4.57%).   And so on.

It turns out that, under this model, teams will shoot around 419 times out of 1000.  The rest of the times, they'll lose possession while moving the puck around.

-------

Now, clearly, there WILL be a negative correlation, here, between shots taken and shooting percentage.  Because, no matter how many or how few shots you take, your expectation is still 32 goals.  

The inverse relationship is true even though the model chose *randomly* whether to pass or shoot.  If the random numbers come up so that you shoot early, you'll have more shots of lower quality.  If the random numbers come up so that you shoot later, you'll have fewer shots of higher quality.  

-------

OK, now, the first result.  I created 2,000 teams, and ran the simulation.  I expected to see a negative correlation between shots and shooting percentage.  But I didn't.  The correlations were always close to zero, and I didn't see any real effect at all.  I think that's because:

1.  Over 1,000 possessions, shot quality evens out enough that there's not much difference between teams.  400 shots with a quality of 8% isn't really that much different than 375 shots with a quality of 8.53%, which is two teams about one standard devaition apart.

2.  My simulation included quality differences between teams.  On average, they had 1000 possessions, but with a standard deviation of 47, which means the better teams will have significantly more shots than the worse teams.  The "more shots means a better team" effect is much larger than the "more shots means worse shots" effect.

3.  SH% depends on whether a goal goes in, which is just random.  If you have, say, 400 shots with an average quality of 8%, the standard deviation of goals is 5.4, or a shooting percentage of 1.36 percent.  That's pretty big, compared to the random differences in shot quality that we're looking for.

So, under this model, I have to admit that real-life differences in shooting percentage aren't due to just random differences in shot quality between teams that would otherwise be the same.  

That's not to say that there isn't another model in which this would work -- one involving more breakaways, say.  But, I doubt even a more realistic model would show enough random variation to explain the 2012-13 Toronto Maple Leafs.

--------

Which means: I'm forced to stick with the idea that there are differences between teams.  So, here's what I did to create those differences.  

As I said, the model assumed a 17% chance of shooting, versus an 83% chance of attempting a pass.  I made those percentages random.  The average was still 17%, but with a standard deviation of about 1.8 percentage points.  So, around 1 team in 6 would shoot 18.8% of the time (or more), and 1 team in 6 would shoot 15.2% of the time (or less).  That is: some teams like to shoot more, and some teams like to shoot less.  

Those differences seem small, but they made the effect really come through.  With that one change, I got a negative correlation of about -.21.  

That's smaller than the actual real-life Corsi vs. Sh% correlation of -.24.  But, in the simulation, I'm using shots instead of Corsi.   Last year, the shot/SH% correlation was -.17, and the year before, it was -0.21.  So, close enough.

(One thing to keep in mind, though, is that the real-life numbers were based on shots for AND shots against.  I'm only using shots for.  So, when we talk about the effect, you should mentally split it between offense and defense.  For instance, instead of thinking about a team that shoots 19 percent of the time instead of 17 percent -- a 2 percentage point difference -- maybe think of it as one point on offense, and one point on defense.)

-------

So, anyway, here are the results of the "shots vs. SH%" regression, as compared to the actual 2012-13 NHL season (5-on-5 tied):

Average Team Sh%:    simulation 7.7%,  real life 7.7%
SD Team Sh%:         simulation 1.42%, real life 1.60%
Average Team shots:  simulation 420,   real life 405
SD Team shots:       simulation 62,    real life 58 (SF-SA)

Correlation:         simulation -0.21, real life -0.17
Coefficient:         simulation 9.0,   real-life 9.4

(Note: some of the "real life" numbers are approximate, but close enough for these purposes.)

It's actually not quite as good a fit as it looks ... in real life, the SD of SH% is significantly higher than in the simulation.  If I had matched them, the correlation would have been much more extreme than -0.21.  I'm not sure how to explain that; it could be random, or it could be something I'm not seeing.

-------

The coefficient of around 9 means that if you take a team like the Leafs that's 3 percentage points "too high" in shooting percentage, you'd expect it to be 27 "too low" in shots.  If you reduce the Leafs' shot difference by 27, it moves them from 45.7% to 47.6% of shots taken.  (That is, 47.6% of all shots taken by the Leafs, and 52.4% by their opponents.)

If Corsis are roughly twice as frequent as shots ... that bumps the difference to 54.  Reducing the Leafs' Corsi difference by 54 moves them from 43.8% to 45.7%.  

Either way -- shots or Corsi -- that's a jump of 4 or 5 teams, which seems reasonable.

But is the *model* reasonable?  Is it plausible that teams vary in their willingness to take shots, and/or their ability to affect their opponents' propensity to shoot?  Because, that's what this is about.  If you accept that teams can differ that much in how they shoot, then you have an explanation for at least part of the Leafs' bad Corsi.

I know the model isn't very realistic, but I'm trying to get a feel quantitatively, rather than qualitatively.  If you built your own model, and included breakaways and counterattacks and defensive zone alignment and all kinds of other things, would differences in team strategy be roughly equivalent to what I've done here?

It seems not too unreasonable to me, but, to be honest, I have little expertise in hockey strategy.  As usual, I'll wait to see what you readers say.



(There are seven parts. Part IV was previous. This is Part V.  Part VI is next.)

Corsi, shot quality, and the Toronto Maple Leafs, part IV

My argument, the last three posts, is that shooting percentage and number of shots taken are negatively related, so when the 2012-13 Leafs have a low Corsi, but a high shooting percentage, that might be an indication of evidence that they took higher quality shots.

Many hockey analysts think that's not true.  One of the reasons they give is that shooting percentage seems to be completely random.  This year's shooting percentage is no indication of what the team's shooting percentage will be next year.  If shooting percentage isn't a skill, how can it mean anything?

I tried to explain, in my first post, how SH% can be important even if it's random.  That was the analogy with the foreign currency.  Let me try a different story that might make things clearer.  

(To be clear, this story is NOT new evidence ... it's just another way of explaining my argument.  It still requires the assumption that SH% and Corsi are negatively correlated. The evidence for that, from the data, was in the previous post.)

-------

Suppose, in a certain country, everyone pays a different rate of income tax.  And the percentage is random.  Every year, each employee goes into a government office, spins a big wheel, and where it lands, that's his tax rate. 

Some people claim that certain employees are so good at spinning that they're able to regularly get themselves a low tax rate.  Researchers test this theory.  They compile census data, and they run regressions, and they conclude that it's not true -- your tax rate this year is not predictive of your tax rate next year.  "Spinning a low percentage doesn't appear to be a skill," they conclude.  "It's just random."

But after-tax income ... that's a different story.  The researchers find that when you take home a lot of money this year, you take home a lot of money next year.  The correlation is quite high.  It's not perfect, of course, because you might get promoted, or lose your job.  And, more importantly, there's randomness.  You might spin a low tax rate this year, and a high tax rate next year, just by luck.  

One year, Carlton Leaf takes home only $43,800 after taxes. That's way below the average for employees in his league.  In fact, 28 out of 29 comparable co-workers took home more money than he did.

The researchers say, "Look, Leaf didn't bring home a lot of money this year ... and the correlation predicts that he won't take home a lot of money next year, either.  Obviously, Leaf isn't very good at what he does."

Leaf's wife doesn't like that. "Wait a second!" she says.  "If you notice, my beloved Carlton's tax rate was especially high this year, much higher than normal, and that's why he brought home less money.  So he's a better employee than you'd otherwise think.  You need to take the tax rate into account!  I think he's going to be employee of the year soon, but ... at least, you have to admit that even if he's not a superstar, he's at least better than the $43,800 makes him look."

The statisticians reply, "The problem is, that tax rate is random.  It doesn't predict anything.  It might be hard to accept because you're a Leaf fan, but, unfortunately, that's how it is.  The low $43,800 is meaningful.  The high 10.8 percent tax rate is not.  You can't use tax rate to measure Leaf's true ability to bring home income, because, after all, you can't give credit for something that's just luck."

-------

Um, that may sound a bit too sarcastic, which is not what I intended.  I'm caricaturing the argument, in order to make the point clearer.  I'm trying to explain better why, even if SH% is random, you might have to still take it into account when you evaluate Corsi.  

The argument boils down to: you have to take SH% into account when evaluating Corsi for the same reason you have to take tax rate into account when evaluating take-home income.

-------

OK, here's an even better analogy that I thought of later. Instead of SH%, consider injury days lost.  It's likely injuries are pretty much random -- if you have lots of injuries this year, you'll probably revert back to the normal amount next year.  

But: if a team has a low Corsi, but high injuries, the high injury rate DOES affect your estimate for next year.  "Injuries are random so we shouldn't consider them in evaluating the team's talent" doesn't fly.

-------

As I said, I'm not trying to be snarky.  The hockey sabermetricians who disagree with me don't dispute my main argument.  They just disagree that there's a real negative correlation between SH% and Corsi.  They know much more about hockey than I do, so there's a good chance that they may be right.

If so, my argument falls apart.





(There are seven parts. Part III was previous. This is Part IV.  Part V is next.)