This year’s 16th Annual Conference on the Science of Dissemination and Implementation in Health, with the theme of Raising Expectations for D&I Science, was a unique opportunity to showcase the Agency for Healthcare Research and Quality (AHRQ) EvidenceNOW Managing Urinary Incontinence (MUI) initiative. Reflecting the conference theme of further exploring implementation practices, the MUI initiative presented both on creating tools to understand the interactions of interventions at a project level and gave a deep dive presentation on the partnerships that facilitate this work within diverse health care systems. This year’s conference provided a platform to bring colleagues from all levels of the MUI initiative together, from the funder and evaluator to the technical support center and implementation grantees, which showcased their combined contribution to the science of D&I.
As part of this initiative, five AHRQ grantees are testing novel ways to disseminate and implement evidence-based urinary incontinence (UI) care for women with primary care practices in different regions in the United States. Theory of Change Models (ToC), described below, were designed to capture the unique interventions of the grantees. Grantees also highlighted the processes for developing cohesive partnerships as a way to scale and spread these interventions.
Theory of Change Models (ToC) for Program Implementation Evaluation
ToC models are a visual way to represent research intervention approaches and outcomes, distinguish the interrelated components, and analyze implementation and evaluation processes. ToC models distill the inter-workings that must come together in an intervention, within a particular context, to achieve set outcomes.
ToC models of the grantee projects were developed through a documentation review process to decipher key elements, qualitative-coding analysis to better understand interwoven pathways, and validation through consensus building with grantee teams, project officers, and the technical assistance team. ToC models can be used in four main ways:
1. Orientation, to help evaluators and technical assistance providers immerse themselves in the intervention;
2. Technical assistance, by supporting grantees to refine their strategies for implementation,
3. Monitoring and evaluation, where the evaluation team will look at implementation approaches over time;
4. Collaboration, by providing grantees a platform for communication on lessons learned.
TOCs were shared with grantees during an annual meeting. Key themes that emerged for the MUI initiative from this discussion included:
1. Recruitment of practices is a challenging and time-intensive process that required more intermediate steps and groundwork than was anticipated;
2. Continuing Medical Education credits did not motivate providers to engage in the project as was expected in original intervention designs;
3. Several project leaders relied on personal connections for practice recruitment, which was acknowledged as a factor that limited scalability and sustainability.
Grantees shared that the ToC model served as a strategic tool and helped refine their own initiatives, better convey connections between intervention components, and further make connections between their own interventions and those of other grantees. ToC models can further function as a dynamic representation of how implementation strategies evolve through iteration to document and disseminate intervention adaptations.
Partnership Development for Promoting Intervention Scalability and Sustainability
As represented by the ToC models, implementation of UI interventions relies on effective partnerships. Grantees built partnerships across a diverse array of stakeholders, including patients, primary care providers and clinic staff, urogynocologists and OBGYNs, geriatricians, health system leaders, practice improvement experts, internal and external practice facilitators, informaticists, IT specialists, mobile health vendors, and community-based organizations. Further, the diversity of primary care practices which are engaged range from federally qualified health centers (FQHCs) to Veterans Affairs (VA) ambulatory centers, to private primary care practices, to large university-affiliated practices.
Grantees described multiple enabling factors that led to sustained partnership engagement as outlined in the list below. Specifically, the University of Wisconsin detailed the engagement process of primary care clinics over their eight-month intervention to improve UI screening and care. In measuring clinic engagement over time, practice facilitators assessed that while most clinics start with low engagement during the assessment and planning phases, there is an increase in engagement, and thus ownership during the launch and adaptation phases of the project. This highlights the evolution of partnerships over time and the primary care clinic’s progress towards independent ownership and maintenance.
Sustainability Enabling Factors
- Mobile health application with clinical pathways
- Proven local/regional engagement before scaling
- Multiple direct outreach options for women to engage in intervention
- Streamlined practice facilitation with consistent contact
- Providing a higher dose of support to higher-need sites
- Incorporating interventions into primary care EHR
- Understanding contextual staffing and workflow barriers
- Understanding clinic priorities and projects
Audience members raised questions about eliciting buy-in in a characteristically overburdened setting. Several themes emerged in grantees’ responses, including meeting practices where they are and tailoring interventions to sites. For example, allowing variations to workflows or building off existing screening protocol processes are strategies that can facilitate a practice’s ownership of the process.
In discussion with the audience, grantees noted one of the biggest barriers to providers addressing UI in the primary care setting is lack of awareness of first-line treatment options, which is why several resources created by the grantee teams include UI educational materials or clinical decision support tools for treatment algorithms. When the audience inquired about the uptake of these types of resources in the primary care setting, the panel emphasized the importance of actively promoting these tools to providers. Grantees further emphasized the value of direct-to-patient communication, such as sending portal messages or providing patients with resources.
Implications and Resources
As the field of implementation science considers challenges and opportunities to capitalize on progress and raise the bar on expectations, the MUI initiative provides two exemplar approaches to fill gaps in care: ToC models ensure that interventions capture all interrelating components and partnerships capture voices so that interventions are authentically meeting the needs of the population served. For more information on up-to-date resources for those working with primary care settings, please visit the AHRQ website.