A paper published last year by Stryjewski et al. found that Massachusetts health reform was not associated with health improvements for patients with certain chronic conditions. A different paper, published earlier in the year by Sommers et al., found that Massachusetts health reform was associated with reductions in mortality.* How could both findings be true?
The answer: Different samples yield different results. Often such a glib response would be a cop out. But here it isn't. Let's look carefully at the samples to see why.
I summarized the sample for the Sommers study in my editorial.
[F]indings [are] based on county-level data from the Centers for Disease Control and Prevention and other sources. These data permit a much larger sample (270 000 adults gaining coverage) than work relying on smaller natural experiments; they also allow a longer follow-up. These reasons may have accounted for the investigators’ finding of a statistically significant reduction in all-cause mortality (2.9%) associated with reform in Massachusetts counties compared with propensity score–selected control counties in other states.
The key here is that the sample included data from all Massachusetts counties, comparing them to matched counties from other states. Hold that in your head as we consider the sample in the Stryjewski study. Quoting the paper:
We examined uninsured and insured patients seen in the Partners HealthCare network, the largest delivery system in Massachusetts. It includes Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), their outpatient departments, and their 20 outpatient community health centers and satellite locations. [...] At Partners, patients with no insurance had their hospital and outpatient services billed to the uncompensated care pool and were not required to make copayments. Physicians waived their portion of the total care charges for uncompensated care pool patients on a pro bono basis. Uninsured patients were required to pay a $1–$3 monthly copayment for prescription drugs, but a voucher to waive payments was available.
There are two things to note. (1) The study compared "uninsured" to insured patients from before and after health reform in Massachusetts. But, in this sample, "uninsured" patients weren't that uninsured. They had their care billed to an uncompensated care pool, without copayments! (2) Moreover, the sample is drawn from patients seen at Partners and who had received blood sugar and cholesterol tests. So they were already getting some care for the specific conditions in question before the reform took effect. This is very different from a sample of genuinely uninsured patients not seen by health care providers. Such individuals would have been represented in the data used by Sommers.
To use a technical term, there's an ascertainment bias in the Stryjewski study. The investigators could only measure characteristics of an "uninsured" (but not really) population that received care. Given the nature of "uninsurance" among those in the sample, that care may have been about as comprehensive as that of insured patients. If the groups basically look the same in that regard, one shouldn't expect to find a contrast.
This is about all I need to know to explain a big reason why the study authors didn't observe any improvement for the chronically ill populations it studied (those with high cholesterol, diabetes, or hypertension). The authors turn to the study limitations in their discussion:
Generalizability of our findings to other states should be considered carefully. Compared with Massachusetts, other states might not have as generous a compensated care pool, might not have baseline rates of uninsured patients as low, and might not have as high a concentration of health providers. It is possible that patients with more severe access limitations in other states would have improved outcomes after receiving insurance. On the other hand, it is also possible that other states will have fewer resources for enrolling, subsidizing, or treating uninsured patients after health reform, so that outcomes could be worse compared to those we detected. [...][The study design] limits generalizability of our findings to uninsured patients who have contact with the health care system.
Having said all that, what the Stryjewski et al. does tell us is that, for the population studied, the (potentially) chronic care coordination functions of insurance did not seem to make a difference. Maybe those functions weren't in place or maybe they didn't work. This is not an insubstantial finding. Chronic disease management is precisely where we should be targeting our efforts to improve health and reduce spending. If insurers aren't playing a role, why do we have them?
* This post begins with links to many others about it.
Austin B. Frakt, PhD, is a health economist with the Department of Veterans Affairs and an associate professor at Boston University’s School of Medicine and School of Public Health. He blogs about health economics and policy at The Incidental Economist and tweets at @afrakt. The views expressed in this post are that of the author and do not necessarily reflect the position of the Department of Veterans Affairs or Boston University.