Background

The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.'s "ten key ingredients" and was developed to support the assessment of DIS research proposals. We describe how we adapted INSPECT and used it in combination with the NIH scoring system to evaluate pilot DIS study proposals through our DIS Center.

Methods

We adapted INSPECT to broaden considerations for diverse DIS settings and concepts (e.g., explicitly including dissemination and implementation methods). Five PhD-level researchers with intermediate to advanced DIS knowledge were trained to conduct reviews of seven grant applications using both the INSPECT and NIH criteria. The INSPECT overall scores range from 0 to 30 (higher scores are better), and the NIH overall scores range from 1 to 9 (lower scores are better). Each grant was independently reviewed by two reviewers, then discussed in a group meeting to compare the experiences using both criteria to evaluate the proposal and to finalize scoring decisions. A follow-up survey was sent to grant reviewers to solicit further reflections on each scoring criterion.

Results

Averaged across reviewers, the INSPECT overall scores ranged from 13 to 24, while the NIH overall scores ranged from 2 to 5. Reviewer reflections highlighted the unique value and utility for each scoring criterion. The NIH criteria had a broad scientific purview and were better suited to evaluate more effectiveness-focused and pre-implementation proposals not testing implementation strategies. The INSPECT criteria were easier to rate in terms of the quality of integrating DIS considerations into the proposal and to assess the potential for generalizability, real-world feasibility, and impact. Overall, reviewers noted that INSPECT was a helpful tool to guide DIS research proposal writing.

Conclusions

We confirmed complementarity in using both scoring criteria in our pilot study grant proposal review and highlighted the utility of INSPECT as a potential DIS resource for training and capacity building. Possible refinements to INSPECT include more explicit reviewer guidance on assessing pre-implementation proposals, providing reviewers with the opportunity to submit written commentary with each numerical rating, and greater clarity on rating criteria with overlapping descriptions.

Background

Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy - and its inseparable politics, polity structures, and policymakers - in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based "thing" to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers' evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy's role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I.

Recommendations

(1) Specify dimensions of a policy's function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy's form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success.

Conclusion

Researchers should use TMF to meaningfully conceptualize policy's role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.

Background

Policy is a powerful tool for systematically altering healthcare access and quality, but the research to policy gap impedes translating evidence-based practices into public policy and limits widespread improvements in service and population health outcomes. The US opioid epidemic disproportionately impacts Medicaid members who rely on publicly funded benefits to access evidence-based treatment including medications for opioid use disorder (MOUD). A myriad of misaligned policies and evidence-use behaviors by policymakers across federal agencies, state Medicaid agencies, and managed care organizations limit coverage of and access to MOUD for Medicaid members. Dissemination strategies that improve policymakers' use of current evidence are critical to improving MOUD benefits and reducing health disparities. However, no research describes key determinants of Medicaid policymakers' evidence use behaviors or preferences, and few studies have examined data-driven approaches to developing dissemination strategies to enhance evidence-informed policymaking. This study aims to identify determinants and intermediaries that influence policymakers' evidence use behaviors, then develop and test data-driven tailored dissemination strategies that promote MOUD coverage in benefit arrays.

Methods

Guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, we will conduct a national survey of state Medicaid agency and managed care organization policymakers to identify determinants and intermediaries that influence how they seek, receive, and use research in their decision-making processes. We will use latent class methods to empirically identify subgroups of agencies with distinct evidence use behaviors. A 10-step dissemination strategy development and specification process will be used to tailor strategies to significant predictors identified for each latent class. Tailored dissemination strategies will be deployed to each class of policymakers and assessed for their acceptability, appropriateness, and feasibility for delivering evidence about MOUD benefit design.

Discussion

This study will illuminate key determinants and intermediaries that influence policymakers' evidence use behaviors when designing benefits for MOUD. This study will produce a critically needed set of data-driven, tailored policy dissemination strategies. Study results will inform a subsequent multi-site trial measuring the effectiveness of tailored dissemination strategies on MOUD benefit design and implementation. Lessons from dissemination strategy development will inform future research about policymakers' evidence use preferences and offer a replicable process for tailoring dissemination strategies.

Background

In 2016, the California Department of Healthcare Services (DHCS) released an "All Plan Letter" (APL 16-014) to its Medicaid managed care plans (MCPs) providing guidance on implementing tobacco-cessation coverage among Medicaid beneficiaries. However, implementation remains poor. We apply the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to identify barriers and facilitators to fidelity to APL 16-014 across California Medicaid MCPs.

Methods

We assessed fidelity through semi-structured interviews with MCP health educators (N = 24). Interviews were recorded, transcribed, and reviewed to develop initial themes regarding barriers and facilitators to implementation. Initial thematic summaries were discussed and mapped onto EPIS constructs.

Results

The APL (Innovation) was described as lacking clarity and specificity in its guidelines, hindering implementation. Related to the Inner Context, MCPs described the APL as beyond the scope of their resources, pointing to their own lack of educational materials, human resources, and poor technological infrastructure as implementation barriers. In the Outer Context, MCPs identified a lack of incentives for providers and beneficiaries to offer and participate in tobacco-cessation programs, respectively. A lack of communication, educational materials, and training resources between the state and MCPs (missing Bridging Factors) were barriers to preventing MCPs from identifying smoking rates or gauging success of tobacco-cessation efforts. Facilitators included several MCPs collaborating with each other and using external resources to promote tobacco cessation. Additionally, a few MCPs used fidelity monitoring staff as Bridging Factors to facilitate provider training, track providers' identification of smokers, and follow-up with beneficiaries participating in tobacco-cessation programs.

Conclusions

The release of the evidence-based APL 16-014 by California's DHCS was an important step forward in promoting tobacco-cessation services for Medicaid MCP beneficiaries. Improved communication on implementation in different environments and improved Bridging Factors such as incentives for providers and patients are needed to fully realize policy goals.

Plan language summary

In 2016, the California Department of Healthcare Services (DHCS) in California released an "All Plan Letter" (APL 16-014) to its Medicaid managed care plans (MCPs) providing guidance on implementing tobacco-cessation coverage to address tobacco use among Medicaid beneficiaries. We conducted semi-structured interviews with health educators in California Medicaid MCPs to explore the barriers and facilitators to implementing the APL using the Exploration, Preparation, Implementation, Sustainment framework. According to MCPs, barriers included a lack of clarity in the APL guidelines; a lack of resources, including educational materials, infrastructure to identify smokers, and human resources; and a lack of incentives or penalties for providers to provide tobacco-cessation materials to beneficiaries. Facilitators included collaboration between MCPs and state and/or national public health programs. Overall, our findings can provide avenues for improving the implementation of tobacco-cessation services within Medicaid MCPs.

Increasing calls within the field of implementation science (IS) research seek to promote active engagement of diverse and often disenfranchised stakeholder voices to increase buy-in, fidelity, outcome relevance, and sustainment of evidence-based practices (EBPs). Including such voices requires cultural humility and the integration of multiple perspectives and values among organizations, groups, and individuals. However, the IS field lacks guidance for researchers on structuring collaborative approaches to promote a co-created process (i.e., synergistic approach to goal attainment). We contend that improved operationalization of co-created implementation collaborations is critical to sparking synergy and addressing differentials based on power, privilege, knowledge, and access to resources among stakeholders. These differentials can undermine future implementation and sustainment efforts if not addressed early in the research effort. An insufficient understanding of the guiding principles of co-created implementation collaborations may limit the scientific value of evaluation processes, and researchers' ability to replicate outcomes. We propose a perspective foregrounded in the concept of co-creation to guide the structuring of implementation collaboratives through five principles. We offer three case examples informed by the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework to illustrate the application of these co-creation principles. Lastly, we offer recommendations for promoting co-creation in IS research moving forward.

Background

Xylazine is an increasingly common adulterant in the North American unregulated drug supply that is associated with adverse health outcomes (e.g., skin infections, overdose). However, there are significant knowledge gaps regarding how xylazine was initially identified and how syringe services program (SSP) staff and clients (people who use drugs) responded to its emergence.

Methods

From June-July 2023, we conducted qualitative interviews with medical (e.g., clinicians) and frontline SSP staff (e.g., outreach workers) and adult clients with a history of injection drug use at a Miami-based SSP. Inductive memos identified emergent codes; thematic analysis involving team consensus established final themes.

Results

From interviews with SSP staff (n = 8) and clients (n = 17), xylazine emergence was identified at different times, in various ways. Initially, during summer 2022, clients identified a "tranquilizer-like substance" that worsened sedation and withdrawal and caused wounds. SSP medical staff later identified this adulterant as xylazine by treating new medical cases and through diverse information-sharing networks that included professional societies and news sources; however, frontline SSP staff and clients needed additional educational resources about xylazine and its side effects. With limited guidance on how to reduce harm from xylazine, SSP clients altered their drug consumption routes, reduced drug use, and relied on peers' experiences with the drug supply to protect themselves. Some individuals also reported preferring xylazine-adulterated opioids and increasing their drug use, including the use of stimulants to avoid over sedation.

Conclusions

Xylazine's emergence characterizes the current era of unprecedented shifts in the unregulated drug supply. We found that xylazine spurred important behavioral changes among people who use drugs (e.g., transitioning from injecting to smoking). Incorporating these experiences into early drug warning surveillance systems and scaling up drug-checking services and safer smoking supply distribution could help mitigate significant health harms caused by xylazine and other emergent adulterants.