What do education reform and health-care reform have in comon?

I've been reading (listening to podcasts?) about health-care reform and education reform lately, and both issues have one thing in common: We measure the inputs but not the outputs.

What does it mean to measure inputs but not outputs? Basically we measure what goes into the system, typically in terms of dollars, but we don't do a very good job measuring what comes out of the system.

In health care, this means that we measure health treatments (tests, medicines, doctor visits, specialists and so on) but we don't measure health outcomes (does the patient get better?). For schooling, we measure spending on education, but we don't have fine-grained data on student outcomes. It's true that NCLB has forced all states to use standardized testing, but only at a very course-grained level. We could do much, much better.

Why do we measure inputs? There are a number of reasons, but the most obvious, and probably most important reason, is that measuring inputs is fundamentally easier than measuring outputs.

Why is this the case? Well, consider the task of measuring health outcomes for a heart surgeon. A crude approximation is to measure the percentage of successful surgeries performed. Except maybe the best surgeons perform the most difficult surgeries, so they're going to fail more because they're better. So we need to take into account the difficulty of the procedure being done, which makes measuring the outcomes more difficult. Furthermore, some successful surgeries are more successful than other successes (and some failed procedures are worse than other failures), so we need to adjust for that as well. At the very least, we need standards for evaluating degrees of difficulty of the surgery to be performed, and we need to follow-up for years after the surgery to see how well the surgeon's work holds up.

Now consider measuring health outcomes for a general practitioner responsible for caring for people for many years. How do we measure a successful outcome? Average lifespan of the patient? Quality of life of the patient? What does a successful outcome even look like?

The point isn't that we can't measure health outcomes. We can track degree of difficulty for surgeries, we can do follow-up with the patient. We have the technology to do that. We can compare a GP's patients against the general population, while controling for factors like the region of the country, the age of the patients, their income, their heritable disease risks, and so on. We have the technology to store and analyze all this data, we just don't do it aggressively enough.

Once we can track health outcomes, we can evaluate doctors based on health outcomes. And once we can do that, we can stop paying doctors based on treatments administered, which can be very expensive, and start paying doctors for better health outcomes. So long as doctors are paid for treatments, there is incentive to order every test that a patient's insurance will pay for. I don't see how this can not lead to over-consumption of health care for people with insurance, since they're not paying for the extra tests, and litigation-averse doctors will happily order every test covered by insurance.

We have a very similar problem in education, namely that we measure how much money goes into the system, but we have crude measurements of what comes out of the system. And it's hard to evaluate teachers for the same reasons that it's hard to evaluate doctors: some teachers have wealthy students with highly-involved parents, and some teachers don't. Sometimes the best teachers educate the worst students, where 50% of the students passing a basic proficiency exam is a big success.

As in medicine, these challenges don't mean that it's impossible to evaluate student learning in a more fine-grained way. It just means that it's going to take some more work to collect better data.

One intriguing idea is Value-Added Testing (VAT), where you measure a class at the beginning of the year to establish a baseline of what they know, then again at the end of the year to see what they've learned---in other words, what value has the teacher added to the students' knowledge? This controls for some of the problem of diverse student bodies.

Another exciting trend is performance-based pay, or merit pay, for the best teachers. Right now, faculty salaries are determined by seniority and not much else. So if you work for 10 years as an industrial chemist, then decide to teach high school chemistry, you make the same money as a 22-year old college graduate. In fact you probably make less because you don't have the teaching certificate that the 22-year old has, and you have to go back to school to earn that certificate. And while I'm not very familiar with the literature, I'm pretty sure that there's very little correlation between teaching certifications and performance in the classroom.

So you can't switch into teaching and make good money, and if you happen to be an excellent 22-year-old teacher, the only way to get a raise is to wait until you've been there long enough. So long as you don't get fired, it doesn't matter how well or how poorly you teach, you get the same raises everyone else does. It doesn't matter if the teacher next door is terrible and you're great; you get paid the same. Very few competitive industries pay everyone the same; why do we expect it to work in education? I don't think there are enough selfless individuals who will work long hours for little respect and no chance at a raise.

Are there problems with merit-based pay? Certainly. Basing pay entirely on standardized testing or value-added testing puts an awful lot of pressure on tests, and tests never tell the whole story in education. Plus it creates a huge incentive to teach to the test, to the exclusion of everything else. Merit-pay will have to include other metrics, like classroom observation and follow-up studies on how well the students do down the road. But it's hard to imagine a worse system given the amount of money we spend on education in the US.

The point of all of this: Statistics matter. In 1986, we thought that batting average and home runs were the most important statistics; most fans didn't understand the value of on-base percentage. Hell, most general managers didn't understand the critical importance of OBP. Baseball stat geeks have revolutionized our understanding of the game of baseball by looking at the data in great detail.

How the heck can we understand baseball so well, but we don't know nearly enough about health care or education?

In defense of partisan politics

I highly recommend listening to EconTalk, hosted by Russ Roberts.

Bipartisanship has been the buzzword since Barack Obama's election, and bipartisanship is generally considered to be a good thing.

In a recent EconTalk, David Brady, a political scientist at Stanford's Hoover Institute, made a strong and provocative defense of partisans politics. (He also enunciated several clear reasons why bipartisanship isn't really feasible given the current dynamics of US elections, but I'm focusing on his intriguing defense of partisanship).

Brady claims, in essence, that major changes have always been partisan. The elimination of slavery was not a bipartisan (or bi-regional) compromise, it was a unilateral partisan decision enforced at gunpoint, and that was arguably the only way it was going to get done.

There are other, less dramatic examples as well. Social Security was pushed through a Democratic congress by a Democratic president. Republicans kept the US on the gold standard in the 1890s, and isolationists were swept from power after WWII and the debate in Washington has never seriously returned to that position.

The key is that all of these decisions were highly partisan and involved little compromise with the other side. And that's how things often have to change.

Another thing these decisions did, according to Brady, is shift the terms of the debate. Prior to the civil war, the debate was how to preserve the union and to preserve slavery; afterwards, slavery was off the table and the debate was about reconstruction. Prior to Social Security, the debate was welfare VS no welfare; now it's about how much welfare. It used to be isolationism VS engagement; now it's how much and what kind of engagement.

All of this reminds me of the classic problem of mediating disputes through compromise: The older brother asks for the whole cookie, the younger brother wants half, so the bipartisan compromise is to give the older brother 3/4 of the cookie. That's clearly not the correct compromise, unless you're the older brother.

I don't know the examples of great bipartisan legislation, so I don't know how properly to compare partisanship with bipartisanship. But listening to the podcast has me thinking more about why my gut instinct has always been to value bipartisanship, and whether I should be more careful about when I should do so.

The mean face of mental illness

I'm the trustee for my disabled aunt, who hasn't worked for over 30 years due to severe anxiety. It's a gut-wrenching, challenging job for me because she's mentally ill, but not obviously so, and I don't always know how to treat her. She has a college degree, she's a good artist, and she can pass for a perfectly functional person at first glance. But if you look a little deeper, you see a messy picture. She lives in a filthy condo packed full of junk, she has no friends, and if she doesn't get what she wants, she lashes out as fiercely and as viciously as a cornered animal. I've heard her, as an grown woman, say to her own mother, "I hate you and I wish you had died instead of Dad."

The hard part is that I know that she's mentally ill, that she has anxiety, that telling her to just relax is like telling somehow with diabetes to just produce more insulin. She also has a host of other health problems, and is in the process of having her knees and hips replaced.

But at the same time, when she says the cruelest and most hurtful thing she can think of, is that simply mental illness? Does she get a free pass to hurt people whenever she wants? It's not like she doesn't know that she's trying to hurt someone, because if one hurtful tactic doesn't work, she's intelligent enough to try something else. She knows exactly what she's doing.

For example, when I was younger I used to get upset and defensive when she would tell me that my mother was no-good. Then I got a little older and started disregarding her opinions on the matter, so she tried other tactics. She would tell me that I was selfish. That my best friend confessed to her that he didn't like me. That my nieces don't like me. That my father is a no-good drunk. That I come from bad genes. That I was brought up without any class. That I'm spoiled. When all of that doesn't work, she'll instantly start crying because of all of her health issues, and beg me for sympathy. If that doesn't work, she'll go back to anger. She will never, under any circumstances, admit she was wrong, nor will she apologize, nor will she accept responsibility for any of her actions.

One night I tried to get her to admit that she was at least 1% responsible for the rift between her and her twin sister (my mother). She went through a litany of reasons why she was not responsible for any part of it. She was brought up to act that way. Her Dad was sick when she was young and that screwed her up. She hates my step-dad. We went in circles for over an hour, and in the end she was unwilling or unable to admit any responsibility for anything. She is beyond reproach for anything.

What sucks about the situation is that I have to treat her like a child, because she is unable to act like an adult, and that feels very condescending. It's not natural for me to treat adults like children, and even with children I try to teach them responsibility for their actions. She can't be taught anything, so I constantly have to remind myself that she's not an adult but that she's not exactly a child either. It's hard for me to take her verbal abuse when I have agreed to be her trustee for free, and paid over $2000 out of my own pocket to hire lawyers to handle her disability case. Like hell I'm selfish! Like hell I don't do anything for her! But it doesn't help me to explain that to her because she either doesn't care or isn't capable of understanding.

And what about her outbursts? Every single lie, outburst, nasty personal attack or bad decision she's made was not a choice but rather the result of mental illness? I don't know enough about the subject, but it's hard to accept that she has no ability to make choices. It's not natural for me to treat her, not quite as a person, but as a special type of person who is never responsible for their hurtful actions. That's not something I have much experience doing, and it's very draining.

What's especially hard is that her constant attempts to hurt me whenever she feels like lashing out have greatly diminished the amount of sympathy I can muster for her. She's bitter, alone, and nasty, and I can't even feel sympathy. All I feel is pity, and that doesn't feel very good.