Pete Palmer on luck vs. skill

Pete Palmer has a new article on skill and luck in baseball, in which he crams a whole lot of results into five pages. 

It's called "Calculating Skill and Luck in Major League Baseball," and appears in the new issue of SABR's "Baseball Research Journal."  It's downloadable only by SABR members at the moment, but will be made publicly available when the next issue comes out this fall.

For most of the results, Pete uses what I used to call the "Tango method," which I should call the "Palmer method," because I think Pete was actually the first to use it in the context of sabermetrics, in the 2005 book "Baseball Hacks."  (The mathematical method is very old; Wikipedia says it's the "Bienaymé formula," discovered in 1853. But its use in sabermetrics is recent, as far as I can tell.)

Anyway, to go through the method yet one more time ... 

Pete found that the standard deviation (SD) of MLB season team wins, from 1981 to 1990, was 9.98. Mathematically, you can calculate that the expected SD of luck is 6.25 wins. Since a team's wins is the total of (a) its expected wins due to talent, and (b) deviation due to luck, the 1853 formula says

SD(actual)^2 = SD(talent)^2 + SD(luck)^2

Subbing in the numbers, we get

9.98 ^ 2 = SD(talent)^2 + 6.25^2 

Which means SD(talent) = 7.78.

In terms of the variation in team wins for single seasons from 1981 to 1990, we can estimate that differences in skill were only slightly more important than differences in luck -- 7.8 games to 6.3 games.

------

That 7.8 is actually the narrowest range of team talent for any decade. Team skill has been narrowing since the beginning of baseball, but seems to have widened a bit since 1990. Here's part of Pete's table:

decade 
ending   SD(talent)
-------------------
 1880     9.93
 1890    14.44
 1900    14.72
 1910    15.33
 1920    13.06
 1930    12.51
 1940    13.66
 1950    12.99
 1960    11.95
 1970    11.17
 1980     9.75
 1990     7.78
 2000     8.46
 2010     9.87
 2016     8.91

Anyway, we've seen that many times, in various forms (although perhaps not by decade). But that's just the beginning of what Pete provides. I don't want to give away his entire article, but here some of the findings I hadn't seen before, at least not in this form:

1. For players who had at least 300 PA in a season, the spread in their batting average is roughly evenly caused by luck and skill.

2. Switching from BA to NOPS (normalized on-base plus slugging), skill now surpasses luck, by an SD of 20 points to 15.

3. For pitchers with 150 IP or more, luck and skill are again roughly even.

In the article, these are broken down by decade. There's other stuff too, including comparisons with the NBA and NFL (OK, that's not new, but still). Check it out if you can.

-------

OK, one thing that surprised me. Pete used simulations to estimate the true talent of teams, based on their W-L record. For instance, teams who win 95-97 games are, on average, 5.6 games lucky -- they're probably 90 or 91-win talents rather than 96.

That makes sense, and is consistent with other studies that tried to figure out the same thing. But Pete went one step further: he found actual teams that won 95-97 games, and checked how they did next year.

For the year in question, you'd expect them to have been 91 win teams. For the following year, you'd expect them to be *worse* than 91 wins, though. Because, team talent tends to revert to .500 over the medium term, unless you're a Yankee dynasty or something.

But ... for those teams, the difference was only six-tenths of a win. Instead of being 91 wins (90.8), they finished with an average of 90.2.

I would have thought the difference would have been more than 0.6 wins. And it's not just this group. For teams who finished between 58 and 103 wins, no group regressed more than 1.8 wins beyond their luck estimate. 

I guess that makes sense, when you think about it. A 90-win team is really an 87-win talent. If they regress to 81-81 over the next five seasons, that's only about one win per year. It's my intuition that was off, and it took Pete's chart to make me see that.

Pete Palmer

Whenever someone mentions David Romer and fourth downs, I think of Pete Palmer, and how he might be the most under-appreciated sabermetrician ever.

While Romer gets all the mentions, Pete was actually first to figure out that NFL coaches are too conservative.  I have a copy of the 1998 edition of "The Hidden Game of Football" (which Pete wrote with Bob Carroll and John Thorn).  Chapter 10, "Kicking Up a Storm," goes through the logic of when you should go for it on fourth down, as opposed to punting or trying a field goal.  Like Romer, Pete finds that teams should try for the first down more often.  One of Pete's many conclusions, just as an example:

" ... you should NOT kick a field goal unless you have six or more yards to go on fourth down.  And if you're inside your opponent's 10-yard line, you shouldn't kick no matter what the distance."  

Romer cites the Palmer chapter in his paper.   He reports that the book's method yields "implausible results," but isn't specific about which results. I think some of the differences come from assuming different values for field position: Romer's data comes from some fancy math with quadratic spline curves, while Palmer's comes from 1997 play-by-play data.  I discussed some of the differences in my blog post on the subject.

But, I've digressed ... my point is not to analyze who's right, just to point out that Palmer had done roughly the same thing, but is barely remembered for it.  Part of the reason, as far as the mainstream press is concerned, might be that Pete is just some guy who wrote a book, whereas Romer is instantly credible as a Ph.D. economist.  Still, my impression is that Palmer gets doesn't get as much recognition even within the football sabermetric community.

In fact, I can't believe "The Hidden Game of Football" gets so little mention at all.  It was the first sabermetric analysis of football I'd ever seen, when the first edition came out in 1988.

------

This "Pete Palmer wrote a book and nobody notices" thing happened again a couple of years ago.  Pete and Dave Heeren combined on "Basic Ball," a book that combined baseball, football, and basketball (Heeren wrote the basketball part, Pete the baseball and football).  I reviewed the book for "By the Numbers" (.pdf).  After my review appeared, Tom Tango wrote, 

"I’m as big a fan of Pete Palmer as there is (which is why we asked him to write the foreword to The Book).  And I had no idea he had a book out since last September.  And I know I’ve corresponded with Pete a few times since, and he never said anything to me."

Commenters at Tom's post note that Pete is very humble and doesn't do much self-promotion.  That's not really the point of this post, Pete's character, but ... if you ask around, almost everyone who's encountered Pete has stories about what a nice guy he is.  For my part, Pete has been exceptionally kind to me, and has gone out of his way for me more than once.  And I don't even know him that well.

------

A few years ago, Tango wrote about the method of finding true talent levels for teams in various sports.  Basically, you look at the overall variance in performance, you subtract the theoretical (binomial) variance that would happen if all teams were the same, and that leaves you the talent variance.

It's simple, but I'd never thought of it, and I started calling it "Tango's method". 

Well, again, Pete was there first.  In a guest chapter of "Baseball Hacks," which came out a few months before Tango's post, Pete describes the method and some applications, and does a little study (see "Hack #68").  And the thing is -- I had actually read that book, and missed Pete's contribution completely.  

If you're a programmer, you'll love seeing how Pete is an engineering geek, and from a different generation than most of the rest of us ... in the book, Pete gives us the computer program he used for his study.  It's written in Fortran.  It uses single-letter variables.  It doesn't indent for structure.  And, it's got GOTOs all over it.  

And this in a book that uses the "R" language for everything else!

For those of you who aren't programmers ... it's like walking into an Apple Store, and one of the techs at the Genius Bar pulls out a 1985 cell phone, the size and shape of a brick with the 12-inch antenna.  And he's not using it as a joke -- hey, he's been using it for 25 years, and he's used to it, and it does the job!

And I'm not making fun of Pete, here, by any means ... I do a lot of my simulations using a version of Microsoft QBASIC from the late 80s ... it comes in one .EXE file, and every time I install a new version of Windows, I just copy it over.

------

And, finally, one more story.  A couple of years ago, I posted an illustration I thought of on why, in baseball, 10 runs equals 1 win.  Tango hadn't seen that particular method before, and e-mailed Pete about it.  

Is it rude to quote a private e-mail?  Well, paraphrased, Pete wrote back something like, "yeah, I actually figured it out that same way years ago ... I guess maybe I should have mentioned it!"

-------

David Romer writes about being more aggressive on fourth down; Pete had already said the same thing.  Tango writes about variances and team talent; Pete had already said the same thing.  I write an explanation of 10 runs = 1 win; Pete had already figured out the same thing.

Pete, you need a publicist!