Consumer Reports on extended warranties

When you buy a new car, the dealer will try to sell you an extended warranty.  It would add on to your regular warranty, extending coverage for a certain number of extra miles or months.  It's like extra "repair insurance" for your car.

Is it worth buying?  I've always wondered, so when I saw that Consumer Reports (CR) tackles the subject in this month's magazine, I thought it might help with my decision next car.  

It turns out that CR believes an extended warranty to be a bad purchase.  Alas, their logic is so flawed that I don't think their conclusion is supported at all.

-----

Several months ago, CR surveyed 12,000 subscribers who had purchased an extended warranty for a 2006 to 2010 car, and asked them how satisfied they were.

But their answers don't tell us much, because of hindsight bias.  For instance, many customers were unsatisfied with their warranty *because their cars never broke down.*  

"100,000 miles came and went, and the car never needed any repairs other than regular maintenance.  What a waste!  I will never buy another extended warranty for a car," said Honda Civic owner Liz Garibaldi.

At the risk of stating the obvious: the fact that you didn't have a claim doesn't mean you shouldn't have bought the insurance.  In fact, you almost *always* hope not to use your coverage.  Your life insurance policy wasn't a waste of money just because you didn't die last year.  

You'd think CR would recognize the effects of hindsight bias.  They do, but concentrate only on the side of that supports their position:  

"[Making more claims] probably helps customers feel more justified about having spent money for the coverage -- a bittersweet way to rationalize the purchase."

But doesn't it go both ways?  What about customers who had the opposite experience, like Liz Garibaldi?  Why doesn't CR say,

"[Not having had a claim] probably helps customers feel unjustified about having spent money for the coverage -- a convenient way to rationalize not making the purchase in future."

"Rationalize" is a biased word, one that presupposes faulty reasoning.  CR doesn't use the second quote because they don't believe you "rationalize" a correct decision.  If the article were about fire insurance, instead, you can be sure it would be the second quote that would have made it into print.

-----

Buyer satisfaction isn't very useful in figuring out whether or not to buy an extended warranty, but what *would* be useful is to know what the coverage is actually worth.  On average, how much does an extended warranty save you, out of pocket?

In CR's survey, the median price of a warranty was $1,214.  The expected value of covered repairs would certainly be less than that, to allow for a sufficient margin of profit.  But how much less?  If it's only a bit less -- say, $100 -- it might be a small price to pay for peace of mind.  But if it's $1,000 less, then it's probably not worth it, even to the most risk-averse buyer.

So that's the key number, isn't it?  But CR doesn't give it to us.  They don't even try to estimate it, or acknowledge that it matters.  What they *do* tell us is:

-- 55 percent of owners didn't use their extended warranty at all;

-- of the other 45 percent, the median out-of-pocket savings was $837.

That helps a little, but not enough.  

First, why do they tell us the median but not the mean?  The mean is almost certainly higher than the median: a blown engine runs several thousand dollars.

Second, the survey includes warranties that are still in force, which means the final number will wind up higher once all the claims are in.  Wouldn't it have been better to restrict the sample to warranties that had already expired? 

From what we're given, all we can tell is that the value of the warranty is at least $376 (45 percent of $837).  But it's probably significantly more.  

It's very strange, the way CR laid it all out.  I don't know what was going on in their heads, but ... the most obvious possibility is that once they concluded that extended warranties were a bad buy, they chose to give only the numbers, arguments, and shadings that support their conclusion.   

-----

In a box, on the first page of the article, in a huge font, is the number "26%".  That's the "percentage of consumers who would definitely buy the same extended warranty again."

Note the word "definitely."  I've seen that word in surveys before ... usually, it's when they give you several choices: "definitely," "probably," "not sure," "probably not," and "definitely not."

But CR doesn't tell us the options, or even how many there were.  So how can we intepret what the "26 percent" means?  If the other 74 percent said "probably," that's very different from if they said "definitely not."  

The "26%" on its own doesn't mean anything, but if you don't realize that -- or you miss the word "definitely" -- it sure looks damning.

-----

On the second page of the article, there's a box with a picture of an unhappy looking customer named Helene Heller, who says,

"It was a horrific experience.  I feel like the dealer ripped me off."

Ms. Heller makes no other appearance in the article.  We never find out why the experience was horrific and why Ms. Heller feels cheated.  It sounds like what someone would say when the dealer refuses to honor a claim.  Perhaps there was a dispute over whether a particular problem was covered?  

Probably not, since the article doesn't actually raise any issues about claims being dishonored -- or, indeed, any issues other than cost and usage.  So, we're in the dark.

Expecting us to judge by anecdote is bad enough, but CR refuses to even give us the anecdote!

-----

Another photo features Brent Lammers, who says,

"I feel like I probably paid too much for peace of mind."

That's a fair enough comment, but ... why choose that one?  Since 26 percent of buyers would "definitely" repeat their purchase, CR could have easily find a quote from someone who feels otherwise.  And was there nobody who blew a transmission and regretted *not* buying the warranty?

In any case, what's the relevance?  We don't want to know what Mr. Lammers feels.  We do want to know if he did, in fact, pay too much.  We never find out.

-----

And then there's the title: "Extended warranties: An expensive gamble."

That's completely backwards!  The warranty may be expensive, but it isn't a gamble at all.  The gamble is in NOT buying the warranty!

A gamble is something that increases your risk.  That is: it increases the variance of possible outcomes.  You can have a sure $10 in your wallet (variance of zero), or you can bet it on a hand of blackjack (outcome from $0 to $25).  

If you buy the warranty, you're buying certainty: $1,214, or whatever, is what your repairs will cost.  If you don't, anything can happen: you could be out anywhere from zero to tens of thousands of dollars.  That's why the warranty gives some people "peace of mind" -- it eliminates the worry of the outlier, an extremely expensive repair bill.

Now, it's fine to accept the gamble if the warranty is too expensive.  There's only so much a reasonable person should be willing to pay to eliminate the risk.  But a justifiable gamble is still a gamble.

What CR was probably thinking, is something like this: "buying the warranty is a bad decision, and you could easily regret it if the car doesn't break.  That sounds like a gamble to us."  

But: a gamble is a gamble regardless of whether or not it's a good decision. If the manufacturers had a 60% off sale on an extended warranty, I bet CR would quickly stop calling it a gamble.  But a gamble doesn't stop being a gamble just because the odds shift in you favor.

Moreover, *any* decision can lead to regret over the alternative.  If a tipster suggests I put a month's pay on a certain longshot horse, I'll probably pass.  I will certainly regret that if the horse winds up winning at 100:1.  But my regret doesn't mean my refusal itself was a "gamble."  

There's some "status quo bias" here too.  Suppose that, historically, the extended warranty was built into the price of the car, but you could get $1,214 back in the mail by agreeing to get rid of it.  CR's headline would now have to read: "Keeping your six-year warranty instead of forfeiting half of it: an expensive gamble."

Less likely, I'd say.

-----

There's one thing that CR didn't talk about that I wish it had.  When I got my car, I wondered if the warranty might pay for itself when I eventually sold the vehicle. 

Suppose that my $1,500 warranty has an actuarial value of only $1,000.  Wouldn't I recoup the extra $500 when I sell the car?  If I'm still covered, buyers will be able to expect that there are no significant problems with my car (because I'd have had them fixed for free).  And, since most warranties are transferable, they can even claim any repairs that I missed.

I know I'd pay a lot more than a $500 premium for a car still under warranty, and I bet others would too.

CR didn't consider that.  Still, if they had at least told us the actual value of the warranty, I might have been able to estimate it for myself.

-----

OK, here's the final kicker.  After ragging on warranties with every argument they can think of for why they're not worth the money and you'll regret having bought one ... CR actually recommends you buy one!

"Consider an extended warranty for the long haul.  All cars tend to become less reliable over time, so an extended warranty might be worth considering if you're planning to keep your vehicle long after the factory warranty runs out."

What they seem to be saying is: we just spent hundreds and hundreds of words telling you, over and over, why extended warranties aren't worth it.  But we meant that only for short warranties.  For long warranties, you should buy one, because our reasoning doesn't apply any more!

Why the change of mind?  

It's true, of course, that cars break down more often when they're older.  But the prices of the warranties rise to reflect that.  Don't they?  Does CR believe that that short warranties are overpriced, but longer ones are underpriced?

Obviously, the total value of repairs over a longer warranty is going to be higher than for a shorter warranty.  That means the median value of covered repairs -- which CR earlier told us is zero -- might move into positive territory.  Is that the difference, the higher median?

In their survey, CR likely found fewer buyers regretting their warranty purchase when they had more time to use it.  Did that change their thinking, the higher satisfaction scores?

Maybe it's an editing issue, and they just said it wrong.  The quote actually appears in a separate section with separate advice for people who *do* decide to buy a warranty.  Maybe they're just saying, "if you *must* buy one, at least buy a long one."  (But it sure doesn't read that way.)

Perhaps they didn't change their mind at all.  Maybe the extra section and the article were written by two different people, with two different opinions.  Or they just cut and pasted their "buy" advice from a previous article, one written before their recent survey data came in.

Or, they're just going with their gut.  Before writing the article, the CR editorial staff thinks, "Newer cars don't break much, so extended warranties are dumb.  But older cars have lots of problems, so it makes sense then."  And then it's all confirmation bias from there.

I really have no idea.

------

Sometimes, it seems like Consumer Reports is two different magazines.  You've got the product ratings, which are written by respected sabermetricians.  Then you've got the advice and investigative pieces, which are written by outraged sportswriters who are sure that Jack Morris needs to be in the Hall of Fame, and have the hand-selected numbers to prove it, and logic and reason be damned.

Can I buy just half a subscription?




(Some of my previous Consumer Reports posts are here.)

Do NFL underdogs consistently beat the spread?

I learned two things from investment blogger Eddy Elfenbein this week.  

First, I learned that if you were invested in the S&P 500 from 1932 to 2009, you'd have made a total return of 63,000% 14,000% (not including dividends).  But if you were invested only the middle 2/3 of each month, you'd have LOST money.  Wow.

Second, I learned that, in the NFL, heavy favorites consistently fail to beat the spread.

------

Since 1978, teams favored by 12 points or more were 220-275-9 against the spread.  Ignoring the nine pushes, that's a winning percentage of only .444.  The effect was easily large enough to turn a profit, even after the bookie's vigorish.

I wondered if, maybe, this was an anomaly that existed in earlier years, but that bookies eventually caught on to and erased.  But, since 2005, those heavy favorites are 64-94-2.

Does the effect disappear for "less heavy" favorites?  Again going back to 1978, and looking at teams favored by 6 to 11.5 points ... they were 1361-1475-57 (.480 excluding pushes).  Teams favored by 0.5 to 5.5 points were 2197-2330-156 (.485).  So, yes, it seems the effect is more pronounced for the heavy favorites.

I broke it down a bit further, into "one point" buckets from 8.5 up.  Teams favored by 8.5 or 9 points were 163-187 (.466).  Teams favored by 9.5 or 10 were 186-193-12 (.491).  And so on.

For every one of those groups, except one, the favorites had a losing record.  (The exception was teams favored by 15.5 or 16, who went 19-15 against the spread.)  No favorite above 20 points has ever covered (0-7). 

------

Is this a known anomaly?  Maybe it's just my ignorance, but I've never heard that this happens.  Well, actually, I should have known ... it was obvious in the numbers for the "home underdog" effect.  

But, actually, it seems to applies equally to home/road.  Home favorites (12+) were 194-236-8, while road favorites were 26-39-1.  

I'm very, very surprised.  

Add this to the list of arguments against NCAA basketball point shaving.  If favorites failing to cover is evidence of point shaving in the NCAA, then it must also be evidence of point shaving in the NFL too, right?

But hardly anyone argues that.  I still think it's just a case of bookies shading their lines towards the underdog favorite.  



(P.S.  Good discussion of bookies' lines in some of MGL's comments here.)

"Trading Bases" beats Vegas odds on baseball

"Trading Bases" is a new book on the theme of sabermetrics and money.  The publisher was kind enough to send me a copy.

Author Joe Peta was a Wall St. trader.  A couple of years back, a traffic accident seriously injured his leg, and he lost his job shortly thereafter.  Unemployed and with limited mobility, he spent his time figuring a method of betting on baseball.  This book is primarily about that system, and its results, although Peta also touches on finance, risk, and gambling in general.

You might expect that Peta's betting system worked -- otherwise, the book wouldn't exist.  You'd be right.  Over the 2011 season, where most of the story takes place, Peta made a profit of 41 percent on his original capital.  In an addendum, Peta shows his results for 2012, where he made 14 percent.

------

What was the system?  It was pretty simple, actually.  Peta started with teams' 2010 stats.  He adjusted for Pythagorean luck, and for Runs Created luck.  He used PECOTA projections to adjust for player performance luck ... except for relievers, where he assumed each team would perform at league-average rates.

On game day, he would figure the winning percentage for each team, using his projections for the starting pitchers and projected lineups.  He then used log5 to calculate the odds of each team winning.  If they were substantially different from the Vegas odds, he'd place a bet.

Over the 2010 season, Peta bet on 2,095 games, winding up with a 1087-1008 record.  That was enough for a gain of 28.81 percent on his original bankroll.  Combining that with a winning record in the postseason, and over/under bets on season wins, gives a final figure of a 41 percent overall return, the number you read about in the blurbs and on the cover.

------

Peta describes the sabermetrics quite well.  He explains Pythagoras, and DIPS, and why those are better estimators than actual outcomes.  For what I call "Runs Created" luck, Peta uses the term "cluster luck", since it's caused by clustering of offensive events.  He doesn't actually mention Runs Created, or any of the other estimators (perhaps because they seem to not work as well these days).  Instead, he explains the logic behind them, how you can look at teams with similar batting lines and see how they have different numbers of runs scored.  He also gives a good intuitive explanation of log5, again without mentioning the term.

If there is fault, it's that the book sometimes seems a little too breathless in hyping the sabermetrics as revolutionary.  For instance,

"Some trial-and-error fiddling with the [Pythagorean] formula reveals that for every ten runs ... [you] can expect to win one more game. ... This really is an amazing revelation, not just for statistically minded fans but for the front offices of major league teams as well."

Of course, that "revelation" is thirty years old, and I'd be shocked if there were any front offices for whom this would be news.

------

One thing that Peta didn't do in his projections, which surprised me, was regress performance to the mean.  In one respect, he didn't have to, because, presumably, PECOTA takes care of that for him.  But, then, he explicitly rejects the idea when talking about his postseason predictions:

"I wasn't going to adjust for players who had played above or below expectations during the regular season.  I had to assume that the best indicator of a player's production over the next week was the body of work he had put forth over the previous 162 games."

And, he didn't do any regressing in his over/unders, either.  When adjusting for RC luck and Pythagorean luck, he assumed that the resulting estimate was the team's true talent.  The 2010 Astros, for instance, came out as a 66-96 team.  Adjusting for talent luck, I might have bumped them up a bit; Peta did not.  (In any case, he easily won his "under" bet, as the Astros collapsed to 56 wins.)

-------

A big part of Peta's system was his money management -- deciding how much to bet on each game.  The bigger his advantage, the more he'd bet.  As a percentage of his capital, his bets went:

Bet 2.0% of capital with advantage 15 percent plus
Bet 1.5% of capital with advantage 13 to 15 percent
Bet 1.0% of capital with advantage 11 to 13 percent
Bet 0.5% of capital with advantage  9 to 11 percent
Bet 0.4% of capital with advantage  6 to  9 percent
Bet 0.2% of capital with advantage  3 to  6 percent
Bet 0.1% of capital with advantage  0 to  3 percent

The advantages are percentage points.  For instance, one game Peta projected the Reds at a 57.6% chance of beating the Brewers.  Vegas odds were only 51.2%, so Peta "deemed the Reds had a 6.4 percent better chance of winning than the implied odds".

Most of his wagers came with a smaller advantage, rather than a larger one.  Here's his table of results (from page 296):

2.0% bets:  26- 13, cash return of +14.8 percent
1.5% bets:  20- 15, cash return of  +4.0 percent
1.0% bets:  38- 34, cash return of  +2.5 percent
0.5% bets:  59- 56, cash return of  -4.1 percent
0.4% bets: 189-164, cash return of +10.2 percent
0.2% bets: 307-339, cash return of -12.1 percent
0.1% bets: 448-387, cash return of  +5.6 percent
------------------------------------------------
Total    1087-1008, cash return of +28.8 percent

The great majority of bets were in the bottom three groups.

------

I was surprised that Peta was able to beat the bookies using a system that's really not that complicated.  Are the oddsmakers so ill-informed that they be so easily exploited?  Does Vegas really leave that many 15 percent overlays, when thousands of people have read Bill James and follow sabermetrics?

So I did some calculating and simulating ... and, no, the oddsmakers aren't quite that bad.  

Take a look at Peta's table.  He placed 72 bets in the "1%" group.  On those bets, Peta went 38-34.  On "pick-em" bets of a dollar each, he'd earn $38 for the winners, and lose $35.70 for the losers (losses cost $1.05, which is the source of the casino's profit).  That's a total win of $2.70.  Since each bet is 1 percent, the total bankroll must be $100.  So that's a return of 2.7 percent.

That comes close to Peta's report of 2.5 percent.  Why the difference?  Because mine is a simplified estimate.  First, Peta bet a combination of favorites and underdogs, which would change the numbers slightly ... and, second, because the payroll varied throughout the season, so not all bets were for the same amount.  

Now, what "should" have happened?  Well, on those bets, Peta thought he had an edge of 11 to 13 percentage points. Let's call it 12, which means that he should have won 62 percent of his bets instead of 50 percent.  That is: he "should have" gone roughly 44-27.  That would have been a return of over 15 percent, not 2.5 percent. 

The difference might just be luck; after all, we're only talking about 72 games.  But, if you repeat the calculations for the other categories, you find that, if Peta's assumptions were correct, he should have made WAY more money.  Here are the "should haves", based on Peta's percentages (I used an 18 percent edge for the top group, and the midpoint of the range for all the others):

2.0% bets:  26- 13, cash return of +28 percent
1.5% bets:  22- 13, cash return of +15 percent
1.0% bets:  45- 27, cash return of +18 percent
0.5% bets:  69- 46, cash return of +12 percent
0.4% bets: 203-150, cash return of +21 percent
0.2% bets: 352-294, cash return of +12 percent
0.1% bets: 430-405, cash return of + 3 percent
-----------------------------------------------
Total     1147-948, cash return of +108 percent         

The system should have made 108 percent, not 28!  And, taking compounding into account, because the capital would be growing over the course of the season ... he probably should have wound up with almost triple his original bankroll.  (Why triple?  Because 100 percent interest, compounded "instantly", yields "e" times your original capital, or 271.8 percent.  Here, we have 108 percent compounded daily, which, coincidentally, is very close.)

28 percent is about one-quarter of 108 percent, which suggests that Peta overestimated his edge by about three times!  

I ran a simulation to confirm, and, yes, it seems to work out.  If I move every edge three-quarters of the way back to even -- that is, regress 75 percent to the mean -- I get almost the same bottom line Peta did:

Peta actual: 1087-1008, cash return of 29 percent
Simulation:  1072-1023, cash return of 30 percent

Of course, there's still plenty of luck involved.  In my simulation, the SD of the final return was around 27 percent.  Peta was around 1 SD better than breaking even.  But: breaking even shouldn't be the reference point.  On Peto's bets, the casino had an advantage of around 1.67 percent.  That eats away at your capital.  If you were no better than an even guesser, my simulation says you'd wind up with around a negative 10 percent return, with an SD of 20 percent or so.  In that light, Peta's +29 percent is about 2 SD above random.

And the simulation confirms that almost exactly ... when I run 1,000 unskilled seasons, 25 of them came out above +28.8%.

I'm actually surprised that a random bettor can beat the house edge by this much, even 25 times out of 1000.  I expected it to be much harder.

------

So Joe Peta's edge was only about a quarter as large as he thought ... but it was still enough to give him a hefty profit, and finish 2 SD above random.  

But ... was Peta just lucky?  It's possible.  But ... my gut thinks that the system was truly good enough to be profitable.  Part of the reason is that Peta turned another profit the year after (albeit a smaller one, only 14 percent).  The other part is ... well, the system seems reasonable.  If bettors have biases in favor of certain teams or pitchers, or they overbet momentum when they shouldn't, or they have other irrationalities, there should be money to be made using a decent, disciplined system.

Still, I suspect there was *some* luck in Peta's results.  If Peta does this again for 2013, my over/under for his profit would be somewhere in the single digits.  

I could be wrong. 


-----

Update, 3/10: good comments at Tango's site ... in particular, check out MGL's #2 and #3.  

Bettors don't regress to the mean enough, investment firm claims

An investment firm has a Super Bowl prediction method that they think can beat the spread, according to this Bloomberg story.

The firm, Analytic Investors LLC, does this: for each of the NFL's 32 teams, they figure out how much money you would have made betting that team during the regular season (betting them outright, it appears, not betting them against the spread).

In 2011, betting the 49ers would have generated an investment return of 52.9 percent this year (I don't know how that's calculated, but that's all they say), making them the team with the highest "alpha". The Colts were the worst, at minus 57.6 percent. The Giants were +32.3 percent, and the Patriots +16.1 percent.

For the Super Bowl, the method says, you should bet on the team that returned the least during the regular season. That's because you should avoid the team with the higher return, because bettors who made more money off their team are "overreacting to information."

"[The Giants] have been the hotter team. They are like the cocktail party stock that everyone’s talking about, that some people have made a lot of money on.”

OK, fair enough. But ... is there evidence that this works? The article doesn't give any, except to say that it's beat the spread for the last eight consecutive Super Bowls. That doesn't mean much, of course, since nobody's claiming the system is accurate enough for eight in a row to be expected.

(Suppose the method predicts with a 60% success rate, which seems way optimistic. Then the chance of 8 in a row is around 1 in 60. At 50%, the chance is 1 in 256.)

This is probably just a publicity stunt to get some exposure for the firm. But it seems like an interesting hypothesis to check out. If a team outperforms expectations during the regular season, it probably did so by luck. And, it seems reasonable to suggest that maybe bettors misinterpret that luck as skill, and overweight the team's future chances.

You'd need a bigger study, of course. Suppose every year you looked at the last three games of the season, for the top 6 and bottom 6 teams in terms of "alpha" in the earlier weeks. That would give you maybe 25 games a year, 500 games over twenty years, 250 games for each group. Worth a shot. I don't have NFL data or betting line data, but I'm sure someone out there does.



Hat tip: Freakonomics

How good are sports pundits' predictions?

According to this Freakonomics post, the experts who make NFL predictions aren't very good at it. Freakonomist Hayes Davenport checked the past three years' worth of predictions from USA Today, Sports Illustrated, and ESPN. He found that the prognosticators correctly picked only 36% of the NFL division winners.

That doesn't seem that great. Picking randomly would get you 25%. And, as the post points out,

"if the pickers were allowed to rule out one team from every division and then choose at random, they’d pick winners 33% of the time. So if you consider that most NFL divisions include at least one team with no hope of finishing first (this year’s Bengals, Chiefs, Dolphins, Panthers, Broncos, Vikings, and Manning-less Colts, for example), the pickers only need a minimum of NFL knowledge before essentially guessing in the dark."

Well, it sounds right, but you have to look deeper.

Winning depends on two things: talent, and luck. Since luck is, by definition, random, when you predict a winner, the only thing you can do is pick the team with the most talent. And, despite the 36% figure, there's no evidence that the pundits misjudged the talent. Because, just by luck, sometimes the best team won't win, and that's unpredictable.

Two days ago, the Buffalo Bills upset the New England Patriots, despite being 7:2 underdogs (I'm estimating 7:2 based on the 9-point spread). What percentage of experts would you expect to have got that right? Your answer should be zero percent. Nobody with any knowledge of football should have thought the Bills had a better than 50 percent chance of winning. On the off-chance that you DO find someone who picked the Bills to win, he's probably a crappy predictor -- maybe he just flips a coin all the time.

In the case where the underdogs wind up winning a game, or finishing first in their division, the truth is the opposite of what Freakonomics implies. In that case, the higher percentage of correct predictions, the WORSE the pundits.

So what does that 36% figure actually tell you? By itself, absolutely nothing. You have no idea, looking at that bare number, how good the pundits are. It depends. If it was *all* bad teams that won, a number as high as 36% means the experts are wrong a lot, but 36% of their bad picks happened to turn out OK. If it was all good teams that won, a number as low as 36% means the experts are wrong a lot -- they must have picked 64% bad teams. And, if it was exactly 36% of the best teams that won, but those aren't the cases where the experts were right, then, again, the experts are wrong a lot.

But, if it was exactly 36% of the best teams that won, and those are exactly the cases where the experts were right ... then the experts are perfect predictors.

So you can't just look at a number. 36% may be bad, but it might be awesome. It depends what actually happened.

-------

However: while this logic applies to picking outright winners of games or divisions, it doesn't apply to picking against the spread. Why not? Because, against the spread, the presumption is that the odds are close to 50/50.

In the Patriots/Bills game, the odds were roughly 77/22. Some experts might have pegged the Bills as having a 25% chance of winning, while some may have estimated only 20%. Still, both pundits would have obviously still bet on the Patriots. The fact that New England ended up losing isn't really relevant.

But against the +9 spread, you might have a reasonable difference of opinion. One expert might figure the true spread should be +8.5, and another might figure +9.5. So the first guy takes the Bills, and the second takes the Patriots.

On bets that are approximately 50/50, reasonable experts can disagree. On bets that are 77/22, they cannot. So, when it's 50/50, a higher percentage could, in fact, mean better predictions.

Still, there's lots of luck there too. If a pundit predicts all 256 games in a season, the (binomial) standard deviation of his success rate will be a little over 3 percentage points. So one predictor in 20 will be over 56%, or under 44%, just by luck.

That means it's still hard to figure out who's a "better" expert and who's not.

--------

The post goes on to criticize the pickers for risk aversion. Why? Because, it seems, the experts tended to pick the same teams that won last year.

Um ... why is that risk aversion? It stands to reason that the teams that won before are more likely to still be pretty good, so they're probably reasonable picks. But, the author says,

"Over the last fifteen seasons, the NFL has averaged exactly six new teams in the playoffs every year, meaning that half of the playoff picture is completely different from the year before. ... Given that information, a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs."

That doesn't follow at all! Just because I know six favorites will lose doesn't mean I should pick six underdogs! That would be very, very silly.

It's like predicting whether John Doe will win the lottery this week. The odds say no, and that's the way I should bet. And it's the same for Jane Smith, or Bob Jones. If there are a million people in the lottery, I should pick them all to lose. I'll be right 999,999 times, and wrong once.

But, according to Freakonomics, I should arbitrarily pick one person to win! But that's silly ... if I do that, I'll almost certainly be right only 999,998 times!

It's not exactly the same, but this logic reminds me of the Jeff Bagwell prediction controversy. In the fall of 1990, Bill James produced forecasts of players' batting lines for 1991, and it turned out that Bagwell wound up with the highest prediction for batting average. It was said that Bill James predicted Jeff Bagwell to win the batting title.

But, obviously, he did not. At best, and with certain assumptions, you might be able to say that James had Bagwell with the *best chance* of winning the batting title. But that's different from predicting outright that he'd win it.

Back to the lottery example ... if I notice that John Doe bought two lottery tickets, but the other 999,999 people only bought one, I would be correct in saying that Doe has the best chance to win. That doesn't mean I'm *predicting* him to win. He still only has a 1 in 500,000 chance.

--------

Finally, in their introduction to their post, Levitt and Dubner say,

" ... humans love to predict the future, but are generally terrible at it."

I disagree, especially in sports. Yes, very few people can beat the point spread year after year. But that doesn't show that the experts don't know what they're doing. It shows that they DO! Because, after all, it's humans that set the Vegas line, the one that's so hard to beat!

I'd argue 180 degrees the opposite. In sports, humans, working together, have become SO GOOD at predicting the future, that nobody can add enough ingenuity to regularly beat the consensus prediction!

Packers win, casinos barely profit

It turns out that Las Vegas casinos didn't make a whole lot of money on this year's Super Bowl. According to the State of Nevada Gaming Board (.pdf), the sports books' profit was a mere 0.83 percent of the $87,491,098 wagered on the game. That's a total profit of only $724,176.

For the last ten years, the average profit was about 8 percent, or about 10 times higher.

If you make the assumption that all bettors need to wager $6 to win $5, then, on average, the casino keeps $1 of every $12 bet, which is 8.3%. That's not too far off from the actual amounts for the last decade, so let's assume that's the case, and see where it leads us.

Making that assumption and doing a bit of algebra, I get that the relationship between the amount of profit and the percentage bet on the winning team is

Percent Winners = (6 - 11 * profit percentage) / 6

Plugging 0.83% into the equation gives that 54.09% of bettors won their bets on Sunday.

Here are the numbers for all 10 years:

2011: 54.90% winners, 0.83% profit
2010: 50.01% winners, 8.30% profit
2009: 50.07% winners, 8.20% profit
2008: 56.07% winners, 2.90% loss
2007: 46.96% winners, 13.9% profit
2006: 49.47% winners, 9.30% profit
2005: 45.27% winners, 17.0% profit
2004: 46.20% winners, 15.3% profit
2003: 50.56% winners, 7.30% profit
2002: 52.74% winners, 3.30% profit

The highest proportion of winners was 56.07 percent, in 2008 (when the Giants beat the undefeated Patriots). That's pretty high. Put into baseball team terms (which I think all percentages should do), it's almost 91-71.

However, we need to take those percentages with a grain of salt, for a couple of reasons.

First, we assumed all bettors are betting $6 to win $5. That's not necessarily true. Big bettors probably get better odds than that. And some proposition bets probably pay worse odds than $5 to $6. Without knowing the expected percentage the house takes, the percentages of winners in the table are only rough estimates.

If you change your assumption to assume that bettors get a better deal than 5:6, the percentage of winners will move closer to 50% in every case.

UPDATE: here are the numbers assuming 10:11 odds:

2011: 51.94% winners, 0.83% profit
2010: 48.03% winners, 8.30% profit
2009: 48.09% winners, 8.20% profit
2008: 53.84% winners, 2.90% loss
2007: 45.10% winners, 13.9% profit
2006: 47.51% winners, 9.30% profit
2005: 43.48% winners, 17.0% profit
2004: 44.37% winners, 15.3% profit
2003: 48.56% winners, 7.30% profit
2002: 50.66% winners, 3.30% profit

Second: the usual assumption is that the casinos want to eliminate risk by having the same amount of money on both sides of the bet. That way, the bookies are certain to win a fixed amount: no matter what happens, they pay the winners with $5 of the losers' money, and keep the remaining $1.

So, one naive assumption is that the sports books weren't that great in predicting how bettors would behave. They obviously had the spread too low if 55% of bettors successfully picked the Packers -- and they nearly lost money because of it.

That might be true: the original line favored the Packers by 2.5 points. In response to the Packers attracting too much action, the bookmakers could have moved the spread to -3. But, because so many games are won by a field goal, 3 points might have been too big a gap from 2.5, and the pendulum might have swung too far towards the Steelers.

However, it's also possible that the bookmakers have an excellent idea of the "true" odds, and are willing to take a certain amount of additional risk if it's in their favor. For instance, suppose the casinos realized that the chance the Packers would beat the spread was only 45 percent. In that case, they might have been happy to take a bit more action on the Patriots. They assumed a bit more risk for that game, and it cost them -- but, over time, they make more money on average by going with the odds.

So, we can't really draw any detailed conclusions, because of our assumptions. However, we CAN say that:

1. If all bets were taken at 5:6 odds, then almost 55% of "pick 'em" bets on the Super Bowl were winners last Sunday.

2. Regardless, there was much more winning than usual on Sunday, enough to almost wipe out the oddsmakers' profits.

I know there are lots of gambling experts out there who might have enough information to explain what really did happen on Sunday (and correct any bad assumptions I may have made). Anyone?


Hat Tip: The Sports Economist

The regression equation versus r-squared

OK, I hope I'm not beating a dead horse here, but here's another way to think of the difference between r-squared and the regression equation.

The r-squared comes from the standpoint of stepping back and looking at the distribution of wins among teams in your dataset. Some teams have over 60 wins, some teams have under 20 wins, and some teams are in the middle. If you look at the standings, and ask yourself, "how important are differences in salary to how we got this way?", then you're asking about r-squared.

The regression equation matters more if you're interested in the future, if you care about how much you can influence wins by increasing payroll. If you ask yourself, "how much do I have to spend to get a few extra wins?", then you want the regression equation.

The r-squared looks at the past, and asks, "was salary important to how we got to this variance in wins?". The regression equation looks to the future, and says, "can we use salary to influence wins?"

It's very possible, and very easy, to have two different answers to these two questions. Here's an example.

Suppose you're trying to see what activities 25-year-olds partake in that affect their life expectancy. You might discover that the average 25-year-old lives to 80, but you want to try to figure out what factors influence that. You run a multiple regression, and you figure out that if the person smokes at 25, it appears to cut five years off his life expectancy. If he eats healthy, it adds four years. If he commits suicide at 25, it cuts off 55 years (since he dies at 25 instead of 80).

Your regression equation would look something like:

life expectancy = 80 - (5 * smoker) + (4 * eats healthy) - (55 * commits suicide).

We should all agree that committing suicide has a big effect on life expectancy, right?

Now, let's look at the r-squared. To do that, look at all the 25-year-olds in the sample (which might be several thousand). You'll see a few that live to 25, some that live to 45, a bunch that live to 65, a larger bunch that live to 80, and some that live to 100. The distribution is probably bell-shaped.

For the r-squared, ask yourself: how much did suicide contribute to the curve looking like this? The answer: very little. There are probably very few suicides at 25, and even if you adjusted for those, by taking those points out of the left side of the curve and moving them to the peak, the curve would still look roughly the same. Suicide is not a very big factor in making the curve look like it does.

And so, you get a very low r-squared for suicide. Maybe it would be .01, or even less.

See the apparent contradiction?

-- suicide has a HUGE effect on lifespan.
-- r-squared for suicide vs. lifespan is very low

And, again, that's because:

-- the regression equation tells you what effect the input has on the output;
-- the r-squared tells you how important that input was in creating the distribution you see.

The regression equations tell you that having a piano drop on your head is very dangerous. The low r-squared tells you that pianos haven't historically been a major source of death.

----

Here's a different way to explain this, which might make more sense to gamblers:

Suppose that you had to predict the lifespan of a random 25-year-old. Obviously, the more information you have, the more accurate your estimate will be. And, imagine the amount you lose is the square of the error in your guess. So if you guess 80, and the random person dies at 60, you lose $400 (the square of 80 minus 60).

Without any information, your best strategy is to guess the average, which we said was 80. Your average loss will be the variance, which is the square of the SD. Suppose that SD is 15. Then, your average loss would be $225.

Now, how valuable is knowing the value of whether or not the guy committed suicide? It's probably not that valuable. Most of the time, the answer will be "no", and you're only slightly better off than when you started (maybe you guess 80.05 now instead of 80). A tiny, tiny proportion of the time, the answer will be "yes," and you can safely guess 25 and be right on. On balance, you're a little better off, but not much.

On average, how much less will you lose given the extra information? The answer is given by the r-squared. If the r-squared of the suicide vs. lifespan regression is .01, as estimated above, then your loss will be reduced by 1%. Instead of losing $225, on average, you'll lose only about $222.75.

Again: the r-squared doesn't tell you that suicide is dangerous. It just tells you that, because of *some combination of dangerousness of suicide and historical frequency of suicide*, you can shave 1% off your error by taking it into account.

----

Now, let's reapply this to basketball. The r-squared for salary vs. wins was .2561. The SD of wins was 14.1, so the variance was the square of that, or 199.

If you took a bet where you had to guess a random team's wins, and had to pay the square of the difference, you'd pick "41" and, on average, owe $199. But let's suppose someone tells you the team's payroll. Now, you can adjust your guess, to predict higher if the team has a high payroll, or lower if the team has a low payroll. If you adjust your guess optimally -- by using the results of the regression equation -- you'll cut your average loss by 25.61%. So, on average, you'd lose only 74.39% as much as before. That works out to $148.11.

What Berri, Brook and Schmidt are saying, in "The Wages of Wins," is, "look, if you can only cut your losses by 25.61% by knowing salary, then money can't be that important in buying wins." But that's wrong. What they should conclude is that "how important money is, combined with how often it's been used to buy wins," isn't that important.

And, really, if you look at the full results of the regression, it turns out that money IS important in buying wins, but that not too many teams took advantage of that fact in 2008-09.

The equation shows that every $1.6 million dollars in additional salary will buy you a win -- so if you want to go 61-21, it should only cost you $32 million more than the league-average payroll of $68.5 million.

That's pretty important, and so the low r-squared must be that not a lot of teams varied much in salary. If you look at the salary chart, there's a huge group bunched near the average: there are 18 teams between $62mm and $75mm, within $6.5 million of the average. Those teams are so close together that there's not much difference in their expected wins.

If you have to bet, and the random team you pick turns out to be the lowest-spending in the league, you'll reduce your estimate. You would have lost a lot of money guessing 41, so the information that you picked a low-spending team will cut your losses a lot. If it turns out be be one of the highest-spending in the league, same thing. But if it turns out to be one of the 18 teams in the mdidle, the salary information won't help you much. And why the r-squared is only about 25% -- for many of the teams in the sample, knowing the salary doesn't help you cut your losses much.

What if we take out those 18 teams, and regress only on the remaining 12? Well, the regression equation stays almost the same -- $1.5 million per win instead of $1.6. But the r-squared increases to .4586. Why does the r-squared increase? Because salary is much more significant a factor for those 12 teams than for the ones in the middle. Before, knowing the salary might not do you much good for your estimate if it's one of the teams bunched in the middle. But, now, those teams are gone. Your random team is much more likely to be the Cavaliers or the Clippers, so knowing the salary is a much bigger help, and it lets you cut your betting losses by almost half.

----

One last summary:

1. The regression equation tells you how powerful the input is in affecting output -- is it a nuclear weapon, or a pea-shooter?

2. The r-squared tells you how powerful the input is, "multiplied by" how extensively the input was historically used. That is: a nuclear weapon used once might give you the same r-squared as a pea-shooter used a billion times.

So a low r-squared might mean

-- an input that doesn't have much effect on the output (e.g., shoe size probably doesn't affect lifespan much);

-- an input that has a big effect on output but doesn't happen much (e.g., suicide curtails 100% of lifespan but happens rarely); or

-- an input that doesn't affect output and also doesn't happen much. (e.g., fluorescent purple shoes' effect on lifespan).

In the case of the 2008-09 NBA, the regression equation shows that salary is a fairly powerful bomb. And the moderate r-squared shows that not every team uses it to its full potential.

Bottom line: salary can indeed very effectively buy wins. The r-squared is as small as it is because, in 2008-09, NBA teams differed only moderately in how they chose to vary their spending.

NFL home underdogs win late in the season

A while ago, Steven Levitt pointed out that betting on NFL home underdogs is a winning strategy. In the sample he covered, home dogs won more than 53% of the time.

In the comments to my last post on the subject, Brian Burke pointed me to an academic study that found the effect is almost completely caused by late season effects.

That paper is called "The Late-Season Bias: Explaining the NFL's Home Underdog Effect." It's by Richard Borghesi of Texas State University.

In a series of tables, Borghesi shows us that in weeks 15-18 of the NFL season, home teams tend to overperform relative to the spread. First, the raw numbers. In weeks 1-14, home teams were favored by an average 2.57 points, and actually won by 2.48 points. In the later weeks, the spread dropped slightly to 2.40 points, but the home teams won by 4.46 points. That is, in the 704 late-season games in the sample (1981-2000), the home team beat the spread by an average of 2.04 points.

The effect was even more extreme in the 188 playoff games, where the home team beat the 5.75 point spread by 2.85 points.

Here it is in an easier-to-read font:

Weeks 1-14: visiting team beat spread by 0.09 points
Week 15-18: home team beat spread by 2.06 points
Playoffs: home team beat spread by 2.85 points

If you consider only home underdogs, the effect persists. This is just the home dogs, from Borghesi's Table 3:

Weeks 1-14: home underdogs beat spread by 0.29 points
Weeks 15-18: home underdogs beat spread by 3.13 points
Playoffs: home underdogs beat spread by 11.33 points

The playoff effect is a huge 11.33 points, but there were only 18 games in the sample.

In Table 4, Borghesi shows that the effect is getting stronger over time. Here are the "late home underdogs" by era:

1981-1985: late home underdogs beat spread by 1.28 points
1986-1990: late home underdogs beat spread by 2.95 points
1991-1995: late home underdogs beat spread by 2.42 points
1996-2000: late home underdogs beat spread by 6.92 points

Why is the effect confined to the later weeks of the season? Could it be the weather? In Table 5, Borghesi defines "cold weather advantage" as a game in which a northern team hosts a southern team. The home advantage is greatest in the cold months:

Aug-Sept: cold weather teams failed to cover spread, by 1.36 points
October: cold weather teams beat spread by 0.84 points
November: cold weather teams beat spread by 1.49 points
Dec-Jan: cold weather teams beat spread by 1.93 points

Remember, none of these tables tell us anything about the relative skills of the teams – just how they did against the spread. Since the spread could (and probably does) adjust somewhat for these effects, we don't actually know whether warm weather teams play worse in the cold. It's possible that, say, the Dolphins actually play a touchdown *better* in Green Bay in December, but the spread corrects by 10 points, making the Packers three points better than the spread. Admittedly that's unlikely, but the point is that we can only draw conclusions about the betting line, and not about the teams themselves.

How much of the "home underdog" effect is caused by the weather? To find out, you'd have to take the total home underdog games, subtract out the weather-related home underdog games, and see what's left. Borghese doesn't give enough data for us to be able to do that, I don't think.

Borghesi then posits a few simple betting rules, and figures out how you'd have done if you'd followed them:

.4905 -- bet on early-season home teams
.5144 -- bet on early-season home underdogs

.5422 -- bet on late-season home teams
.6000 -- bet on late-season home underdogs
.6071 -- bet on late-season home 2+ (points) underdogs
.5789 -- bet on late-season home 8+ (points) underdogs

And, finally, Borghesi runs a couple of regressions. The first one tries to predict the winner based on which team is at home, and the spread. Its predictions are 55% accurate early in the season, and 57% accurate late in the season. However, the regression is run separately for each year, which means the results can take advantage of pretending to "understand" random year-to-year fluctuations. When the same regression formulas are used to predict *next year's* winners, the results drop back to 50%.

However: Borghesi finds that if you run the regression every week, based only on the last month's games, you can predict next week's winners late in the season at a 53% clip. But that's only a 361-317 record, which is less than 2 SDs from .500.

One last regression in the paper tries to retrospectively predict winners based not only on the spread and the home team, but various performance statistics of the two teams. This gives 67% accuracy, but so what? Again, the regression might just be picking up randomness and spurious correlations.

(Why are the regressions there? Maybe (he said, mischievously) you can't get a paper published just by counting and tabulating things, even if you answer the question better than by more involved methods.)

If you ignore the regressions, the paper does present very good evidence that the entire "home underdog" effect happens late in the season, and that weather is at least partially responsible for it.

The home underdog effect isn't holding up

A year ago, I posted about a Steven Levitt study showing that NFL bettors don't seem to like home underdogs. In the online tournament Levitt studied, it turned out that when the home team is the dog, only 31.8% of bettors backed it – the other 69.2% bet on the road favorite.

And, as it turned out, the home underdogs would have been good bets; they beat the spread almost 58% of the time. Levitt hypothesized that because bookies know that their customers prefer road underdogs, they shade the point spread in favor of the home team, in order to win more of those bets.

But in a post last week on Freakonomics, Levitt revisited the home underdog effect, and found that, over the past two seasons, it no longer held. In 2007, home underdogs were 44-45-1 against the spread in 2007, and 32-45-2 against the spread in 2008.

Why the change? Levitt thinks it's just luck, and not a change in the way bookmakers set their lines.

Thinking about the issue again, I wonder if the original finding, that 58% of home underdogs beat the spread, might not itself have been just luck. In Levitt's original paper, there were 2,286 bets on home underdogs. But that's the number of bets, not the number of games. There were only 85 NFL games studied in total. That means that there were probably less than 40 home underdogs to bet on. 58% of 40 games is only 23-17, which is really not that big a deal, is it?

When Levitt looked at 21 seasons' worth of NFL betting lines, instead of just those 85 games, home underdogs were still a good bet, beating the spread 53.3% of the time. That's significant, but not as impressive as 58%. Assuming 53.3% is the "real" figure, that means that in 2007-8, home dogs should have gone 90-79. They actually went 76-90-3, or 13 games below what they "should" have. That's almost exactly two standard deviations.

If you don't assume 53.3% is the "real" number, and instead you test, statistically, for the difference between the two means (53.3% earlier, and 45.9% later), you wind up with a bit less than two standard deviations. I'd agree with Levitt that it's probably just luck.

An NCAA basketball pool for experts

Carl Bialik, the Wall Street Journal's "Numbers Guy," is running a March Madness pool for scientific prognosticators of college basketball.

It's an entertaining exercise, but I wouldn't make too much of the results.

For one thing, as Tom Federico says in the article, there's a lot of luck involved in any small sample, and the ultimate winner would probably have been aided more by good fortune than by ability. Second, the sample is in a sense actually "smaller" than it looks at first glance, because the results of one round affect the next. So a single upset could actually cost you several games' worth of losses.

I'd call this more a publicity exercise than a true test of an expert's abilities. The ultimate test of a forecaster is still how well he does against the Vegas betting market – the best aggregate predictor of sporting events known to humankind.

Pete Rose and betting market efficiency

Courtesy of Zubin Jelveh's blog, a link to a short paper analyzing Pete Rose's 1989 bets (from the Dowd Report). It's called "Market Efficiency in the Baseball Betting Market: The Case of Pete Rose," by Douglas Coate.

The paper is only two pages long; it lists Rose's bets, both on the Reds and on other teams, from April 8 to May 12, 1987. Rose placed 190 bets, typically of $2,000 each. He lost $4,200 on the Reds (27 bets), $36,000 on other NL teams (72 bets), and $7,000 on his AL wagers (89 bets), for total losses of $47,200. (My numbers add up to 188 bets instead of 190 because I probably counted them wrong from the table.)

Coate describes Rose's bookie's odds as even money, with a 10% fee payable on losses. This is equivalent to 10:11 odds. He says the losses "include about $20,000 to $25,000 in transaction fees," so even if Rose were able to have gotten even money, he still would have lost.

From this, Coate argues that these results are "consistent with an informational efficient [betting] market," and says that Rose's expertise "was not an advantage."

I'd agree that it's interesting that Rose didn't win, and I do think the betting market is very efficient, but I think that looking at Rose's bets is a very weak way of testing for strong market efficiency.

First, why presume Rose is an expert? It's pretty well established that most professional baseball managers work from their gut, not from science or empirical evidence. Rose may have inside information, but professional gamblers have analyzed the market, and, in some sense, have a lot more information impacting their bets than Rose does.

Second, during the period in question, Rose bet on every Reds game except one. It's not very plausible to assume that he would have had inside information impacting every game, would it? Even if he did, somehow, know that over the entire month, the Reds were better than could be predicted from publicly-available information, that would have become apparent as they started winning games and the odds adjusted.

Third, any advantage you'd gain by having inside information isn't worth that much. For instance, suppose Rose knew that Eric Davis would be out of the lineup. If Davis was 50 runs better than his replacement over a season, that's 0.3 runs over a game. That's 0.03 wins, or one extra win every 33 games. Knowing about Davis's absence is a fairly big piece of insider information, and even if Rose had one of those every game he bet on, he'd only win one extra game out of 33 (perhaps going 17-15 with a rainout instead of 16-16).

Suppose, on average, Rose had this kind of insider information one out of every three games (which still seems like a lot, especially considering he bet on so many different teams). In that case, instead of going 95-95, he'd be expected to go 97-93. Rose's results aren't significantly different from either of those numbers, so if you say there's no evidence that Rose could beat the market, you have to also admit that there's no evidence that he didn't have insider information and *could* beat the market. Rose's bets just aren't a powerful enough test to tell us much about market efficiency.

Straight-up picks can't distinguish good pundits from bad

Experts are no better at picking winners of NFL games than a simple algorithm, "Numbers Guy" Carl Bialik reports in his latest Wall Street Journal blog posting.

The easy method is dubbed the Isaacson-Tarbell postulate, after the two readers who proposed it. Pick the team with the better record; if the two teams have the same record, choose the home team. According to ESPN.com's Gregg Easterbrook, no pundit was able to beat Isaacson-Tarbell. Only one was able to tie.

Easterbrook writes,

"You don’t need incredible insider information, you don’t need to spend hours in fevered contemplation … Whatever you do, don't think!"

While normally I love to join the "Super Crunchers"-esque refrain that formulas often know better than "experts," I don't think it really applies in this case, where you have to pick winners straight-up.

NFL matchups are often lopsided. If an .800 team is playing a .300 team, it's obvious that you have to pick the .800 team. No matter how expert you are, no matter how much insider knowledge you have, you simply aren't going to be able to know that the .300 team is better, because it *isn't* better. The same is true for a .700/.400 matchup, or even a .650/.450 matchup. You may be more expert than the rest of the world, but the rest of the world isn't dumb. Everyone picks the .650 team over the .450 team, so your insider knowledge doesn't do you any good. The best you can do is to *tie* the rest of the dumb-but-not-that-dumb punditocracy.

It's only when it's a close matchup that expertise can come into play. Suppose that two evenly-matched teams are going at it, and most experts think team A has a 52% chance of winning. So, they pick team A. For you to outpredict those guys, you have to have insider knowledge, and that knowledge has to be in the direction that leads you to believe that team A has *less than a 50% chance*. That's the only way you'll predict team B will win, and the only way you'll be the rest of the (A-picking) experts.

How many close matchups are there in a season? Maybe one or two a week? Suppose there are 30 close games a season. In those, the best predictor might be more accurate than the pack, say, 50% of the time (to be generous). That's 15 games. In those 15 games, 7.5 will be more accurate go the "wrong" way -- that is, the additional expertise will confirm the pack's pick, not contradict it. That leaves 7.5 games left. Again to be generous, call it 8.

So, in eight games a season, the expert predicts a different team than the pack. But those eight games are already pretty close, almost 50/50. It's probably the case that the pack thinks they have a .520 pick, but they only have a .480 pick. So the expert has a .040 edge for eight games a year. Over 256 games, the best, most expert pundit has an advantage of about a third of a game.

Is it any wonder you can't find out who the experts are by picking straight-up?

If you want to evaluate the experts, just have them pick against the spread. Now *all* the games are close to 50/50, not just 30 of them. Now, the expert has a fighting chance to emerge from the pack.

Most of the touts I've heard of *do* pick against the spread. I haven't seen how they've performed, long-term, but I bet most of them are pretty close to 50%. And, if so, *then* you can conclude that those so-called insiders can't beat a simple algorithm.

But looking at straight-up picks? That's like trying to find the best mathematician in a crowd by asking them what 6 times 7 is. Under those circumstances, the Ph.D. will do about as well as a sixth-grader. It doesn't mean the guy with the doctorate in mathematics doesn’t know more than the eleven-year-old. It just means you asked the wrong question.