In response to an increased demand to include individuals with lived experience, or individuals who have personal experience with the research topic, in the research process, health services researchers have begun to shift away from quantitatively centered study designs to mixed-method study designs that capture qualitative data, particularly from those with lived experience. When health services researchers engage stakeholders early, intentionally, and consistently throughout the entire project, the quality of science improves, the relevance of interventions is strengthened, and the potential for lasting change increases. When you build science around the people who need the intervention, you don’t just improve the quality of the intervention, you heal historic mistrust and create local champions who can advocate for its implementation.
However, if stakeholders (i.e., patients, families, community leaders, policy makers, clinicians) are engaged at the wrong time, in the wrong capacity, or their knowledge and experience is dismissed, this results in interventions that do not work. We have a term for this kind of dismissal now: epistemic injustice. How stakeholders are engaged is even more critical than if they are engaged, because, on paper, engagement done poorly can look identical to engagement done well. A project can check the box, hold the listening sessions, and still end up with an intervention that misses the mark, resulting in a program or tool that doesn’t work.
This is especially true in Quality Improvement (QI) work. QI projects are designed to improve systems, and every stakeholder occupies a different role, sees a different problem, and therefore offers a different perspective. Clinicians see workflows. Payers see utilization. Patients see what happens when systems fail. None of these perspectives alone can provide the full context. However, when stakeholder engagement happens properly (i.e., community-led needs assessments, community-defined evaluation metrics) health service researchers can prevent the breakdown that often occurs somewhere between identifying a problem and developing or recommending a specific intervention.
That breakdown is usually not a failure of effort. It is usually a failure to sequence properly. QI projects use the Plan-Do-Study-Act (PDSA) cycle, that breaks problem solving down into four distinct, repeatable steps. Essentially, the PDSA cycle is a four-step process by which a change is tested. While that sounds like a simple way to improve quality, it only works if the people closest to the problem help define what the problem actually is before anyone starts to test solutions. Bringing stakeholders in after the intervention has already been designed, is not engagement, it is consultation. You are asking them to react to something that has already been decided. While the cycle looks participatory, it is not. The most important decision, identifying the problem, was already made without them.
To avoid this pitfall, researchers first need to assess their own rationale for including stakeholders, and more importantly the value they place on stakeholder perspectives. Do you value stakeholders as co-investigators or merely as subjects? Do you consider their knowledge as legitimate or as complimentary to “real” data? Are stakeholders’ inclusion intentional, or something of convenience that you add to secure funding? These are important questions researchers must sit with honestly before starting a project, because if the answers are not right, the project will miss the mark no matter how well-intentioned it is.
In QI, this plays out in small but telling ways, like whose definition of improvement gets used. One example is measuring quality in a clinical setting. Clinicians may prefer metrics that are easily measurable, like appointment availability, while patients may measure quality by whether a provider actually listens to them, which is not easily measured. Both are valid, but if one voice is credited with knowing what quality is over the other, only one version of quality gets pursued. This can result in the measurable data looking great, while the system fails the people it was meant to serve.
Longstanding approaches like community-based participatory research (CBPR) offer practical steps for engaging stakeholders early, as partners in defining the problem, not just as consultants reviewing solutions. What CBPR cannot do, however, is address a researcher’s underlying assumptions about who is capable of producing legitimate knowledge. No framework can fix that on its own. Still, CBPR is a good starting place to have conversations with yourself, your research team, and then with your stakeholders.
The critical decisions in any improvement effort are undoubtedly made at the beginning, when the project is conceptualized, the team is assembled, gaps and blind spots are identified, and adequate resources are secured. It is that time that determines whether the work ahead will make a meaningful impact on the lives and intended communities or if it will be just another research project. Quality improvement gives us the tools necessary to plan-do-study, and act as many times as necessary to get it right. It does not automatically give us the humility to ask the right people what needs fixing in the first place. That part is still our responsibility.
A few takeaways to sit with:
- Engage before you define the problem, not after. If stakeholders only see the intervention, they have missed the decision that mattered the most.
- Ask yourself who counts as credible. If a finding only becomes real once an “expert” says it, that is worth looking at more closely.
- Let stakeholders define “improvement,” too. A dashboard with data can look great and still miss what actually matters to the people it serves.
This blog post is the first in a series from the Medicaid Outcomes Distributed Research Network: Examining Quality Improvement for Medicaid Programs Project (MODRN EQUIP). Led by Multi-Principal Investigators Julie Donohue, Ph.D. and Andrew Barnes, Ph.D., MODRN EQUIP aims to develop and describe provider-level OUD treatment quality measures, measure OUD patient-reported outcomes and experiences in treatment, and implement an OUD quality improvement initiative in Medicaid.