Monday, September 10, 2007

Consumer Reports is starting to bug me

Consumer Reports magazine has been frustrating me recently. They seem to know what they’re doing when it comes to testing products, but in anything involving numbers or analysis – consumer sabermetrics, say – they come up short.

Here are three examples from the
October, 2007 issue, which arrived in my mailbox last week. These aren’t the only things in the issue that bother me, but they’re the three biggest.


Jack is a hypothetical retiree in a
sidebar in CR's "Your Money" column. The question: should Jack start collecting Social Security at age 62, or should he wait until age 70?

If he starts at 62, he'll receive $19,320 a year (adjusted annually for inflation). Or, if he waits until age 70, he'll start receiving $35,240 per year. Which is better, assuming Jack will live to age 90?

According to the "CR Money Lab," the first option gives Jack total payments of $542,570 (in 2007 dollars). That's 28 years times $19,320. The second option gives him 20 years times $35,240, which works out to $709,745. (These numbers aren't exactly correct; I assume there are rounding errors somewhere.)

28 years times $19,320 = $542,570
20 years times $35,240 = $709,745

Therefore, CR says, the second option is better.

Has CR never heard of the time value of money? A dollar today is worth more than a dollar eight years in the future, because of interest. CR’s calculation treats them as equal, in violation of basic principles of Finance 101. As Steven E. Landsburg writes (perhaps a little too harshly): "when college sophomores treat a dollar paid 20 years from now as the equivalent of a dollar paid today, we usually advise them that they have no talent for economics."

Assuming a real discount rate of 3%, the first option gives Jack $381,842 (in 2007 dollars, adjusted for inflation and time). The second option is worth $314,033. It’s actually the first option that’s better!

The higher the discount rate, the better the first option looks. If you use 4%, it wins by $341K to $237K. To get the two options equal, you have to lower the discount rate to about 1.8%.

Which option to choose depends on your personal discount rate, which is another way of saying it's your personal situation and preference. As CR does admit, your economic circumstances are a factor in which option is better for you.

Here’s one last way of looking at it. Suppose you don’t need the money until age 70, but decide on the first option anyway. From age 62 to 69, you invest all your benefits at a real rate of 4%. At age 70, those eight years of benefits will have grown to $185,139.

Now, even after 70, you’ll still be receiving only $19,320 a year, as opposed to the $35,240 you would have got had you waited. Your shortfall is $15,920 per year. But the lump sum of $185,139 should cover the shortfall for quite a few years, especially if invested well. You could probably even buy an annuity with that $185K that would top up your annual benefit, bringing it close to what you would have got by waiting.

Indeed, this kind of actuarial logic is probably how the government sets the benefits in the first place: so no matter which of the two options you take, the expected total payout, adjusted for time value, is roughly the same. CRs oversimplified calculation is simply not correct.


In the
October issue’s letters section, a doctor writes in to explain that when drug companies send him free samples, he uses them "to let patients try the medication for free to see if it is effective and tolerable."Sounds like a win/win, right? Not to CR. An unnamed editor replies,

"Free samples are not free. They are part of the drug company’s advertising budget and contribute to the high cost of drugs. The free sample is a tool to tune the patient in to brand-name recognition, so that when it runs out they will stick with the same brand, despite the expense. There might be less-expensive drugs that are just as effective."

This is wrong.

First, the free samples might actually lower the cost of the drug. Most of the drug companies’ costs are in R&D; it can take many millions of dollars in research to come up with one marketable drug. But once the drug is approved, the cost of actually making the pills is minimal. So the more patients taking the drug, the lower the drug company can price it and still recoup their costs. If the free samples prove to patients that the drug will help them, the drug sells more, and prices can come down.

Suppose that without free samples, the demand curve is such that 1,000 people will use the drug at $100, but 1,500 will use the drug at $60. The company would make more money pricing the drug at $100. Suppose that after giving out samples, 2,000 will use the drug at $100, but 5,000 will use the drug at $60. Now, the company will drop the price to $60, so it can earn $300,000 instead of $200,000.

CR should look at it this way: if I were the only reader of their magazine, could they afford to sell it for $3 a copy?

Second, we patients aren’t as dumb as CR thinks we are. If we like our free sample of drug X, we might stick to it even though we know that there might be cheaper drugs that work too. But, in most cases, it’s just not worth the risk. If drug X is $100 a month, and drug Y is $80 a month, it’s perfectly reasonable to pay the extra $20 for a drug you know will work, instead of taking a chance on an untried alternative. (This arguably doesn't apply to generics, which are actually the same compound as the brand name. But CR isn’t talking about generics.)

Medicine is not a game, where the objective is to experiment with treatments to make the drug companies’ profits as low as possible. Switching to a new drug, if the old one works just fine, can be rather unpleasant. If a $1 pill keeps my diarrhea under control, I ain’t switching to a different compound just because it only costs 90 cents. At least not before a big date, or a job interview.

And keep in mind that these are prescription drugs we’re talking about, drugs the patient can’t get without a doctor’s approval. If there are cheaper alternatives, I expect my doctor to recommend them. If my insistence on the same drug doesn’t make medical sense, I expect my doctor to try to talk me out of it. But if I’m happy to get a free sample, and the doctor is happy to offer it to me, and my doctor agrees with my decision to stay on it after the sample runs out - well, who is Consumer Reports to overrule my relationship with my doctor? It’s my money and my suffering, and not CR’s.

Indeed, can you imagine what CR would say if HMOs started forcing patients to give up their preferred medication for a different, but cheaper, one?

Strangely enough, CR’s argument doesn’t seem to apply to their own products. In this very same issue, on page 38, they offer me a "risk-free sample issue" of "CR on Health." If I like it, I have to pay for more.

What makes CR’s offer so much more reasonable than a drug company’s? Isn't it true that free issues are not free? Once CR stops with the expense of giving away free issues, won't the price of a subscription drop? And how come they're not worried that "when my free sample issue runs out I will stick with the same magazine?" After all, there are many less-expensive alternatives than $39 a year.


In an article entitled “Make your car last 200,000 miles” (
subscription required), CR compares the costs of keeping a new car 15 years, as opposed to replacing your new car every five years. Here are their numbers. The first column is one car over 15 years; the second column is three cars over 15 years.

$19,000 $66,400 Total car price (Honda Civic EX)
$14,900 $35,500 Depreciation
$18,300 $ 6,200 Maintenance/Repairs
$ 3,100 $ 8,500 Finance/Interest
$ 2,200 $ 4,800 Fees/Taxes
$18,500 $22,500 Insurance
$57,000 $77,500 Total

$20,500 savings
$10,300 potential investment savings
$30,800 total potential savings

What CR is saying is that, by keeping your Civic for 15 years instead of trading in every five years, you can save $20,500. Plus, if you invest that $20,500, you can earn an extra $10,300 from the interest (at 5% per annum). So the total savings are $30,800.

But there are a couple of things here that don’t make sense. First of all, CR claims that if you pay $19,000 for a Civic, and keep it for 15 years, it will only depreciate $14,900 – which means it’s still worth $4,100. Could that be right? A lot of Civics will rust out over 15 years, and be worthless. Even if similar cars for sale right now are worth $4,100 (or about $2,600 before 15 years’ inflation), there’s a selection bias – most ‘92 Civics are long dead, especially around here with our salty winter roads. (I can tell you for sure that my 8-year-old Sunfire is going to be worth a lot less than 21% of its original price by the time it hits 15.)

Second, where do the investment returns come from? CR is calculating $10,300 in interest on $20,500 principal, which is a 50% return. At 5%, the average dollar of the $20,500 savings would have to compound for 8 years.

But that’s not the case. Over the first five years, both options are absolutely identical, so there’s no savings then. All the savings accrue over the next ten years. If you assume the money is saved equally over each month of those ten years, the average compounding time is five years. For repairs, the savings might slightly skew to the middle five years, since breakdowns are probably more frequent in years 11-15. On the other hand, insurance is lower in years 11-15. Also, the absolute numbers are higher in years 11-15 because of inflation. All these seem to roughly balance out, so let’s assume that our orignal 5-year number is correct.

Five years compounding brings the interest only to about $5,700, far less than the $10,300 claimed.

(Update: this method is an approximation: a better method gives $5,921 instead of $5,700. See comment 5 to this post.)
I could be wrong about all this, of course, but if I’m right, CR is making overstated financial projections that no used-car dealer would be allowed to get away with.


So what’s the deal with CR? They seem to have experts who understand how to deal with testing and evaluating real products, but when it comes to finance and economics, where you have to use logic instead of instruments ... well, if there were a Consumer Reports for Consumer Reports, I think they’d rate a “Not Acceptable.”


UPDATE, 9/12

Here's one more.

In their “Claim Check” feature this month, CR investigates a claim by Tide that you can save “up to $65 a year” in energy costs by using its Tide Coldwater detergent. The savings accrue because the product lets you wash in cold water instead of warm.

CR verifies that Proctor & Gamble are telling the truth: $65 is indeed correct.

Case closed? Nope. CR complains anyway:

“But hold on. With the qualifier 'up to [$65],' the claim is valid even if you save a penny a year.”

That’s literally true, but any reasonable consumer knows what “up to” means. It means "as much as, under the right conditions, but yours will probably be less." Here, the conditions are very reasonable: indeed, if you live in a city with higher-than-average electric rates, you’ll save *more* than $65.

Besides, if the claim was way off, and P&G defended it on the "penny a year" grounds, CR would be all over them, accusing them of misleading the public.

So I would have thought this to be a perfectly legitimate and ethical use of “up to.” Sniping at Tide here is completely inappropriate.

Speaking of energy savings: three pages later in the magazine is an ad for a new book, “Consumer Reports Complete Guide to Reducing Energy Costs.” Why should you buy this book? Because CR says you can

"SAVE UP TO $1000 A YEAR."

Seriously. Three pages after gratuitously bitching about "up to," they use it in an ad for their own product!

Labels: ,


At Monday, September 10, 2007 1:02:00 PM, Blogger Tangotiger said...

Using the calculator here:
Setting my variables to: Oct 1, 1946, and earnings at 97,500$ (the maximum), the monthly payouts, in 2007 dollars, at age 62 is 1437$, compared to the 2007 dollars at age 70 of 2700$. So, I don't see a problem here, since both are in 2007 dollars. The 2700$ in 2007 dollars is actually 3551$ in 2016 (and will continue to grow based on cost-of-living). There is no potential for "earnings" beyond the cost-of-living, since the money is basically used to live with, and therefore, the discount rate would be just that. The breakeven age is set at 79 years old, which seems about right to me. That is, expect half the people at age 61 to live beyond 79 years. If you think you will outlive your peers, wait until you are 70, if you can. If you don't think you will outlive them, then start the payout at age 62.


The depreciation of the car must be almost the full purchase value. The salvage value of the car will likely be under 1000$. The savings of the car, as you said, can only kick in after 5 years, since the first 5 years, it's equal.

Also, don't forget that we are paying for advances in safety and performance. A 1992 Civic probably won't have passenger side air bags, won't have side impact protection. And, the miles per gallon (or km per L) must get worse as the car gets older, thereby having extra costs. Maintenance cost should also include your time to bring the car in/out of the shop.

I once made a simple calculation that showed that the break-even point was about 10 years for my Accord. At that point, I figured I'm better off getting a new car than trying to keep my old car going. But, I should revist those calculations at some point.


As for your price of CR, how much do they charge? Over here, it's 26US$ for the paper subscription, and 19US$ for the online (for paper subscribers).

At Monday, September 10, 2007 1:15:00 PM, Blogger Phil Birnbaum said...


I agree that CR calculated the cost-of-living factor (inflation) correctly. However, the discount rate is not equal to the inflation rate. For instance, you can probably get 5% on a CD while inflation is at 3% for a real rate of 2%. For typical investments, I think the real rate of 3% that I used (ie, a nominal rate of 6%) is reasonable.

You're arguing that if you need every penny of Social Security to live on, then the effective real discount rate is 0%. I agree with that. However, if that's the case, there is NO choice between the option of taking payments now or waiting until 70 -- if you wait until 70, you die!

If you DO have the option of waiting until 70, you also have the option of taking payments at 62 and investing them. If you do that, then the 62 is equal to the 70, not worse.

In either case, comparing the sum of the nominal payments is not correct. You can make up different sequences of payments that add up identically, but are unfair to either side.

For instance, suppose you give me $100,000, and I agree to pay you $100 a month until you die, then return your $100,000. If you live another 60 years, the sum of your payments is $172,000, but I'm still ripping you off, paying you only 1.2% on your money.

When trying to compare two different payment streams, you have to use the market discount rate.

At Monday, September 10, 2007 1:19:00 PM, Blogger Phil Birnbaum said...

Agreed that on the car, you'll get a much better car five years later and ten years later. And you might get better mileage too, for the same performance. I didn't think of the idea that the old car's mileage would drop, or the time spent in the shop, but you're right, of course.

Agreed that CR should have mentioned all these things.

I was giving them the benefit of the doubt that they understood that, but were concentrating on the savings instead of the improvement in the product.


I think I paid $31 CDN per year for a five-year subscription. It was more than that for a one-year.

At Monday, September 10, 2007 1:29:00 PM, Blogger Tangotiger said...

That'd be 5% minus taxes paid on earnings, which would drop it to 3.5% or so. Your point however is valid.

At Tuesday, September 11, 2007 9:21:00 AM, Blogger Phil Birnbaum said...

On the car cost issue: my logic, that the average interest-earning period is 5 years, yields the correct calculation of interest only for simple interest, not compound.

For compound, instead of taking the average compounding time, I recalculated using 1/10 of the principal compounding for 9.5 years, another 1/10 compounding for 8.5 years, and so on, up to the last 1/10 compounding for 0.5 years.

The result: $5,921 in interest.

I've put an update in the original post.

At Tuesday, September 11, 2007 9:30:00 AM, Blogger Phil Birnbaum said...

By the way, if you use the above compounding method, but for 15 years instead of 10, you get a number close to CR's.

That suggests that they just didn't realize that the all the savings accrue during the last 10 years of ownership.

At Thursday, September 13, 2007 12:52:00 AM, Anonymous Anonymous said...

"Most of the drug companies’ costs are in R&D"

Not true. Where did you get this idea? That's just pharmaceutical company propaganda.

In reality, R&D costs are about par with marketing costs, and significantly less than the profit.

Consider also that most R&D cost is allocated towards "me too" drugs, where a small change in the molecular or chemical composition results in new or extended patent protection for another 17 years.

Check your facts. Pharmaceutical revenue is mostly driven by marketing and profit, than by R&D spending.

At Thursday, September 13, 2007 12:54:00 PM, Anonymous Anonymous said...

Something that a lot of people also forget about the drugs that are used in these "free samples" is that we're not talking about simply allergy or cough medications in most cases. A large number of these samples are for anti-depressant and anti-psychotic medications which are used to treat everything from nicotine withdrawals to serious chemical imbalances. In many cases, there IS no cheaper alternative. In fact, these drugs are often the ONLY alternative, but the free samples allow mental health professionals and doctors to test the effectiveness of a medication on a patient BEFORE writing a prescription for them. Thus, if a patient tries one medication but does not respond well, they do not need to buy a new prescription 2 weeks later. The doctors can test several options without incurring a cost to the patient. For that small fact, I think the samples are a great idea.

The sad reality that doctors actually get a kick-back from the pharmaceutical company for each prescription they write for certain newer drugs is sickening. My doctor, despite our eloquent author's comments on the sanctity of dr-patient relationships, is taking a bribe to put me on a drugs I may or may not need.

Anyone who wants to know what they're taking, or how well it works, or any other options should not talk to their doctor, but to their pharmacist. Doctors don't know what's in these things. Pharmacists do!

At Thursday, September 13, 2007 3:44:00 PM, Blogger Unknown said...

I'm wondering if you could use the same car comparison for leasing...right now in my area you can lease a new Civic for $200 a month with no money down. It would seem to me that would be the best option over 15 years...because repairs and maintenence would be minimal since the car is still under warranty the entire time you have it under lease.

At Thursday, September 13, 2007 4:57:00 PM, Anonymous Anonymous said...

Regarding drug samples:
I am a doctor, and our clinic has decided to ban samples, with which I agree, even though we have many uninsured patients. The economic rationale for refusing to participate in sampling is that they harm patients and mis-allocate healthcare system resources.
1) Sampled meds are usually new, similar to existing meds, but at higher cost. A short trial of a low-cost alternative with longer track record and better safety data is typically not financially ruinous.
2) Sampled meds affect prescriber behavior. Although poster is mistaken (MD's do not get kickbacks) there is excellent scientific evidence that MD's prescribe meds more when they are sampled (and their logos appear on pads and pens, which we also refuse.)
3) Pharma does not sample meds for time-limited indications. I would welcome samples of antibiotics or skin-condition topicals, but I have never seen such a sample. You can imagine why...
4) Blogger mistakes the economic implications of Pharma's marketing strategy. The goal is not to expand the market to maintain margins while distributing fixed costs across growing sales volume despite lowering prices. Rather it is to solidify increasingly inelastic demand by promotions including direct-to-consumer advertising (and samples) and protection of monopoly status with endless extensions of patent protection through bogus "new indications" approvals through complicit FDA. They're shifting the demand curve, Sir.
5) When we did stock samples, we were never provided with a reliable regular supply of initially-sampled drugs for chronic conditions that patients must take regularly, without missed doses, for years. Patients were changed monthly from one sampled drug to another similar one based on what the reps brought us, harming health.

Conclusion: the goal and effect of samples is to recruit paying inelastically-
demanding customers, and steer doctors and patients to meds with no proven benefit over existing ones, and dollars to less health-enhancing expenditures than optimal. Examples: statins, proton-pump inhibitors, ACE inhibitors, ARB's.


Post a Comment

<< Home