Wednesday, January 02, 2008

A non-analysis analysis of NFL playoff trends

You know those systems that try to figure out what kinds of teams do well in the baseball playoffs? Bill James had a system that did that a couple of decades ago (details here), and I've seen a couple of others since. Well, now, Mike Sando, of espn.com, tries to do something similar for football. It would be a reasonable effort, except that he doesn't give us the comparisons we need to figure out if he's right.

Sando looks at eight bits of "conventional wisdom." For instance: does defense win championships? Sando looked at 2000-2006, and found the 30 NFL teams that gave up the fewest (regular-season) points per game, to see how they did in the playoffs. They won 38 playoff games.

So? Is that good? Does that mean that defensive teams do better in the playoffs that non-defensive teams? We can't tell. He doesn't tell us how many *losses* went with those 38 wins, and he doesn’t tell us how many games the *other* teams won!

I am boggled. How can you just look at the number of games won and reach any kind of conclusion?

I guess you can probably figure it out, to some extent ... if winning were random, then, of 30 teams, 15 would win their first playoff game. 7.5 of those teams would win a second game, and 3.75 of those would win a third game. Not all those 3.75 teams would have a fourth game, so lets make it just one extra win for the "fourth round" (which is the Super Bowl). The total: 15 plus 7.5 plus 3.75 plus 1 equals about 27.

Yup, 38 wins is better than the expected 27 wins.

But does that mean that defense wins championships? Maybe if you look at the teams with good *offenses*, they also win about 38 games. In that case, just being good wins championships. But Sando didn't think of checking that.

Here's another test, of whether "home-field advantage means everything." Well, of course it doesn’t mean *everything*, but is it more important in the playoffs? The way to answer that question, of course, would be to show us the HFA for the playoffs, and compare it to the HFA for the regular season. Of course, we don’t get that. What we *do* get is these two facts:


Road teams won three-fourths of wild-card playoff games following the 2004 and 2005 seasons;

Home teams won 20 of 28 divisional-round games since 2000.


So: why are we picking and choosing 2004-2005 in one case, but 2000-2006 for another? Why are they separated? And what's the home winning percentage in the regular season, so that we can compare?

Anyway, I'll stop here; there are actually eight questions, not just these two, but the other six are similarly exasperating. The article sounds like analysis, and it wants to be analysis, but what we get are anecdotes and (deliberate) selective sampling. It must have taken a fair bit of research to compile the data, but, alas, we learn almost nothing from it.

(Thanks to Alan Reifman for the tip.)



Labels: ,

1 Comments:

At Friday, January 04, 2008 7:28:00 PM, Blogger Brian Burke said...

Regarding the wildcard games, road team upsets are happening more often since the 2002 restructuring. With eight division winners instead of six, there is a greater possibility for weak teams in weak divisions to earn a home wildcard game. The wildcard teams themselves, the visitors, are often stronger teams from stronger divisions. 2004 and 2005 were, by chance probably, the years with the highest % of road winners.

 

Post a Comment

Links to this post:

Create a Link

<< Home