Earlier this month, I wrote briefly on how the relationship between high cholesterol and heart disease is growing murkier than has been traditionally assumed. Today, by way of Gary Schwitzer’s Health News Blog, I came across a recent BusinessWeek article by John Carey that cracks this story wide open—in part by addressing an incredibly important, but often misunderstood and misused—statistic: the “number needed to treat.”
The succinctly titled piece, “Do Cholesterol Drugs Do Any Good?,” notes that “Americans are bombarded with the message from doctors, companies, and the media that high levels of bad cholesterol are the ticket to an early grave and must be brought down. According to these ubiquitous messages, statins [cholesterol-lowering drugs like Lipitor] “are the most potent weapons in that struggle.” Carey notes that Lipitor advertisements claim that the drug “reduces the risk of heart attack by 36 percent…in patients with multiple risk factors for heart disease.” Sounds pretty effective, right?
Hold the phone—there’s more to that number than meets the eye. Carey notes that the 36 percent is accompanied by an asterisk stating that “in a large clinical study, 3 percent [or three out of every hundred] of patients taking a sugar pill or placebo had a heart attack compared to 2 percent [or two out of every hundred] of patients taking Lipitor.”
Now, Pfizer’s number isn’t an outright lie. Pfizer, Lipitor’s manufacturer, says its potion reduces risk by 36 percent because the difference between two patients getting a heart attack on Lipitor and three patients getting a heart attack on placebos is one patient—or about a third the number of heart attacks that would have happened without Lipitor.
But Carey looks closer at the math to find that this calculation is
ultimately unimpressive—and worse, misleading. He notes that “to spare
one person a heart attack, 100 people had to take Lipitor for more than
three years [i.e. the duration of the trial].” Think about it—if the
trial only tells us that Lipitor only helps one out of a 100 people who
take it faithfully for three years than working with a pool of less
than 100 would produce no benefits. You can’t help one-third of a
person not have a heart attack. Conversely, a success rate of one out
of 100 means that “the other 99 got no measurable benefit” from
These numbers are a lot less comforting than the more superficial
statistic of cutting heart attacks by 36 percent. But this data is what
ultimately matters, because it represents the number needed to treat
(NNT) for one person to benefit. For Lipitor, that number is 100—in
other words, an honest doctor would have to tell her patient that “only
1 in 100 [people who take Lipitor] is likely” to be positively affected
by the drug. Of course, doctors are subjected to the same lobbying from
drug companies as consumers, and more often than not, according to Dr.
Darshak Sanghavi from UMass Medical School, “many physicians don’t know
the NNT” of specific drugs. This isn’t information we hear very
often—even from medical experts.
Worse still, even an NNT of 100 is probably a low-ball figure. Statin
trials sponsored by the drug industry paint an even worse picture,
pegging the NNT “at 250 and up for lower-risk patients, even if they
take it for five years or more.” Dr. Jerome R. Hoffman, professor of
clinical medicine at the University of California at Los Angeles,
clarifies the unpromising reality behind this statistic: “What if you
put 250 people in a room and told them they would each pay $1,000 a
year for a drug they would have to take every day, that many would get
diarrhea and muscle pain (the side effects of statins), and that 249
would have no benefit? And that they could do just as well by
exercising? How many would take that?”
That’s right: exercise and diet changes—primarily a greater consumption
of fruits, grains, fish, and olive oil—have been proven to reduce
cholesterol more than statins. Statin supporters shrug off criticisms
by saying that a success rate of one out of a hundred is significant
when the drug is taken on a wide enough scale. After all, one-one
hundredth of one million is ten thousand.
As Carey notes, that’s a fair point—but where to draw the line? What’s
the right balance between low individual benefit and personal
risks—including muscle pain that can lead to kidney failure, and liver
problems whose symptoms include the yellowing of your skin—and the
possible benefit to the population?
On the one hand, this is an interesting philosophical question; but
ultimately it’s a moot point. Regardless of where you fall on the
issue, this sort of watered-down communitarian benefit is not what
Pfizer has been peddling to America.. Instead, the company has been
spinning the information so that it maximizes the individual benefits
but diffuse the risks of the drug. For example, the company picks and
chooses when to pitch numbers in NNT format or in percentage points.
Despite ignoring the NNT stats for Lipitor benefits, Pfizer uses it to
play down the risks of the drug, bluntly stating that “only 1 in 100
people suffers a side effect”—even though this is exactly the
prevalence of the drug’s success! Just as many people suffer
side-effects as benefit—this is strategic selectivity at its best.
For the sake of context, it’s worth thinking about how Lipitor compares
to other drugs in terms of NNT. Below is a great insert (click to enlarge in a new window) that
accompanied Carey’s Business Week piece, and gives you a sense of how
NNT puts our understanding of drugs in a new light.
The best way to read this table is to keep in mind something that
Merrill Goozner said in a recent post: “the better [a drug] works, the
fewer people you need to enroll in a clinical trial since the
statistical significance needed to convince the FDA will be easier to
reach.” A lower NNT means that a drug is more reliably effective and
that its benefits are significant.
So, for example, the antibiotic cocktail above eliminates bacteria in
almost everyone who uses it; this means both that the treatment more
effective, and that it’s also cheaper to develop. That’s because the
trials require testing the drug a smaller number of people for a
shorter amount of time before a benefit is apparent.
If the NNT is high, however, than in reality a drug is just a “minor
innovation or no innovation at all,” Goozner notes. Related costs
reflect this fact: the trials take longer and require more subjects;
the analysis is more laborious and time consuming because results are
not easily detected; and once the drug rolled out, marketing will
likely be more intensive to compensate for questionable efficacy.
So when drug companies tell you that, unless we pay exorbitant prices
for their products, they won’t be able to afford to research develop
new drugs you might keep this fact in mind: it costs far more to
develop a drug that represents only a tiny improvement over less
expensive drugs that are already on the market.
Goozner sums up the argument: “Why does industry spend $20 billion a
year on clinical trials? Is it because the cost of trials has
skyrocketed? Or is it because the new drugs that industry is bringing
to market are such minor innovations or no innovation at all compared
to previous drugs that it takes trials with literally thousands of
people in them to prove something works.” Ultimately a more honest use
of NNT isn’t just an issue of forthrightness—but also of
Unfortunately, the obscurity of NNT might also be related to how easy
it is to fool patients. Those who lament the ubiquity of misleading
statin claims note that “Americans have come to rely too much on
easy-to-grasp health markers.” As one University of Texas expert puts
it to Carey, “the American cultural norm is that doing something makes
us feel better than just watching and waiting.” We want the eye-popping
statistics, the quick fix, the miracle cure.
But perhaps what’s even more troubling is that this ethos is
institutionalized in our health care system. The system works on the
notion that more is better. Direct to consumer advertising encourages
patients to ask for more drugs, more procedures, or more implants;
fee-for-service schedules reward volume over effectiveness; and the
hierarchy of reimbursement rates encourages complex solutions over
simple ones. We’re fueled by volume, not even-keeled assessments of
There’s a lot of sleight of hand in the prescription drug industry. But
the misunderstanding and misuse of NNT statistics is perhaps one of the
most important, least recognized, and most emblematic distortions you
can find. Kudos to BusinessWeek and Merrill for shedding some light on