How to Mislead with Graphs: Health Care Edition

People don’t like looking at big tables of numbers, which is why people invented graphs. Looking at a graph can often make the meaning behind a large set of numbers become much clearer. However, we mustn’t confuse clarity with accuracy. A graph can make a set of numbers much more easily understood, but if those numbers don’t tell us the whole story, then the graph doesn’t either.

Recently, I came across this stunning graph. Take a few moments to absorb its beauty:

According to this graph, the measure of the goodness of the health care system correlates with the slope of the line. If the line goes up, that’s good, and the more it goes up, the better it is. If the line goes down, then… you get the picture.

Before I get to the criticism, let me say that I agree with the general point: health care costs too much in the US, and I believe we do need to do something about it. But scale matters. It matters whether we think the current system is so bad that we need to completely gut it and start over, or whether we think it just needs a few tweaks (for the record, I think it’s somewhere in between). Looking at this graph might lead you to believe that our system is entirely without merit and should be discarded immediately, so it’s worth considering whether that graph really does represent an accurate picture of the situation.

First, since we are to measure a health care system based on the slope of the line, that implies that there is some way to balance the value of lifespan against the value of dollars. The graph implies that every year of lifespan is worth about $200 a year for that person in health care spending. Is that about right? Is the relationship even linear? That’s a tough thing to measure, but that’s the implication of the graph. As a thought experiment, imagine that the average lifespan in the US was 85, putting us higher than any other nation. What would the slope look like? It would still be the most negative slope on the graph. But would our extra spending on health care be worth it then? Maybe. What if our average lifespan were 90? Would it be worth it? According to the graph, we would still be the most inefficient system, since our slope would be the most negative. To further illustrate the point, look at Mexico’s line. Their efficiency is pretty impressive. Should we be switching to their model? Thus we see the difficulty in trying to equate dollars with life-years, and we see that we can’t really judge the scale of our problem based on the slope of these lines, because the authors of the graph have arbitrarily decided how much a life-year is worth.

Second, looking only at the left side of the graph, we have to ask: why does health care cost so much in the US? Is it all just waste and inefficiency? Should we really be shooting to spend the average amount of $3,000 per person? It turns out that it may not be fair to compare countries in this way, because richer nations tend to spend more on health care. Of course that makes sense: if you have more money, you’ll spend more on everything. But the effect is even greater than that because if you have more money, you are likely to increase your health care spending by an even greater degree than other spending. The additional spending is less likely to have a major impact on life expectancy because of the principle of diminishing returns, but that doesn’t mean it isn’t worth it to the person doing the spending. In the US, we like our technology and advanced tests, even if that technology and those advanced tests only buy us a marginal gain over less expensive alternatives. Whether that is good or bad is a debate we ought to have (or better yet, let’s just let people decide for themselves), but the graph above ignores the question entirely. In addition, wealthier countries have to pay their medical professionals more, because there is more competition for high-skill labor. And the same principles applies to prices of medical supplies and pharmaceuticals, as I’ve described before. The bottom line is that you can’t just take the total spending per capita as a comparison between nations without considering all of these complicated and inconvenient factors that don’t fit so nicely into a pretty graph.

According to the report from OECD linked above, the US is overspending by $2,500 per person, when adjusting for our wealth. So, we’re still overspending, but by much less than the $4,000 depicted in the graph. To be a fairer comparison, you’d have to adjust the left-hand side of the US line down a good bit, thus decreasing the slope of the line quite a bit. Remember, scale matters.

Lastly, let’s look at the right half of the graph. I think it’s pretty obvious that health care spending is not the major factor in average lifespan. Sure, it is *a* factor, but not a major one. This graph itself is evidence of that. With lines crossing every-which-way, there appears to be almost no correlation (although of course there is some). It’s not a stretch to argue that a good portion of our lower lifespans is due to our unhealthy lifestyles and risky behavior. How much would our lifespans go up if Americans made the same lifestyle choices as Europeans or (better yet) Asians? Who knows, but it would certainly go up, thus decreasing the downward slope of the line.

So, am I arguing that we have nothing to worry about? Of course not. We need to reform our system to be more efficient and less expensive so that more Americans can afford it. But understanding the scale of the problem matters. It matters when figuring out how to fix it and what we are willing to sacrifice to do it.

Lies, Damn Lies, and Obama’s Healthcare Statistics

When Republicans tried to explain the size of the stimulus bill based on the amount of money it amounted to per day since the birth of Jesus, I objected: “I’ve never really thought these sorts of number analogies all that useful. I can make just about any number look large or small using some visualization.”

I had a similar reaction when Obama said that healthcare costs cause a bankruptcy every thirty seconds. But I have ever greater reason to be annoyed, because it turns out Obama’s number is not only a meaningless statistic, but is actually dead wrong:

“The cost of health care now causes a bankruptcy in America every thirty seconds,” Obama said at the opening of his White House forum on health care reform. The problem: That claim, based on a 2001 survey, is simply unsupportable.

The figure comes from a 2005 Harvard University study saying that 54 percent of bankruptcies in 2001 were caused by health expenses. We reviewed it internally and knocked it down at the time; an academic reviewer did the same in 2006. Recalculating Harvard’s own data, he came up with a far lower figure – 17 percent.

…

Himmelstein tells me that the reason for the difference is a change in federal law that sharply reduced the number of bankruptcies. In 2005, the year he and Warren wrote their op-ed, there were just over 2 million bankruptcies. Data out just today say that in 2008 there were 1.1 million (up sharply, by the way, over 2007). So this error in the White House claim stems simply from the fact that it’s using out-of-date information. The next question is whether the estimate of “medical bankruptcies” is reliable in the first place.

A good part of the problem is definitional. The Harvard report claims to measure the extent to which medical costs are “the cause” of bankruptcies. In reality its survey asked if these costs were “a reason” – potentially one of many – for such bankruptcies.

Beyond those who gave medical costs as “a reason,” the Harvard researchers chose to add in any bankruptcy filers who had at least $1,000 in unreimbursed medical expenses in the previous two years. Given deductibles and copays, that’s a heck of a lot of people.

Moreover, Harvard’s definition of “medical” expenses includes situations that aren’t necessarily medical in common parlance, e.g., a gambling problem, or the death of a family member. If your main wage-earning spouse gets hit by a bus and dies, and you have to file, that’s included as a “medical bankruptcy.”

You might think, “So what? Healthcare reform is important, so why so much fuss over a statistic?” I would agree that healthcare reform is important, as I’ve blogged about before. But to illustrate why I think this mischaracterization (and the many that take place every day in the world of politics) is important, let’s do a little thought experiment:

Imagine that you are an HR rep, and you come to your boss with a new brilliant idea to increase employee retention:

You: “Sir, I believe we should give away free soda in the break rooms.”

Boss: “Why’s that?”

You: “Well, did you know that someone quits every five days because there is no free soda in the break room?”

Boss: “Every five days? Wow, that’s a lot. Where did you get that number?”

You: “Well, in 2001, we gave everyone who quit a survey, and some said that no soda in the break room was a reason they quit. And actually it’s closer to one person every 9 days, but that’s not the point…”

Boss: “Wait, we’ve cut the number of people who quit in half since 2001 due to other policy changes. And just because someone says it’s a reason they quit, doesn’t mean it was the main reason. But even so, that number seems high.”

You: “Yes, well I also included people who said they liked soda even if they didn’t list it as a reason they quit. I assume that if we had free soda, they might not have quit.”

Boss: “I see. How much do we pay you again?”

I hope that makes it clear. We would never tolerate this sort of fudging in our real lives, but for some reason we tolerate it from our politicians. I don’t mean to single out Obama. Almost every politician does this. But of all the politicians that have come along, Obama was the one who had the most power to change the norms that we have come to accept. Looks like the opportunity will be wasted.