Name
#21 - Secondary Analysis of Leading Practices Program (LPP) Submission Data to Inform on Program Improvement
Date & Time
Tuesday, February 13, 2024, 12:00 PM
Description

Introduction. A Leading Practice Program (LPP) that requested submissions from the Military Health System (MHS) field for piloted, local clinical solutions for review and potential spotlighting and award was implemented. It was delivered in partnership between Defense Health Agency (DHA) Clinical Quality Management and DHA Research and Engineering (R&E) Implementation Science Branch (ISB). An array of information was requested from the program’s submission application, all developed for the primary purpose of assisting submission evaluators understand the basics of these solutions, gauge their potential impact, and score their worth for program spotlighting and reward. We leveraged this submission data for the secondary purpose of continuous program improvement, exploring for opportunities to improve the application’s entry items so that they could potentially be better indicators of solution understanding, impact, and worth. Multiple observations, interpretations, and resultant way-forward recommendations were reported. The current presentation shares a select set of those observations and recommendations. Methods. Disseminations reaching across the MHS to spread awareness of the LPP and its request for leading practice submissions occurred in the last quarter of CY2020. The submission cycle ended in late December 2020. In total, 76 separate submissions were received for evaluator review and scoring – 86% from continental US (CONUS) military treatment facilities (MTF) and multiple submissions delivered from several MHS markets. As part of the LPP’s principal purpose, submission data was collated and distributed to evaluators in a planned procedure to score and select practice submissions submitted for program spotlighting. As a secondary effort, pulling emergent themes from qualitative data items and exploratory analysis (mostly in the form of unstratified and stratified frequency reports) were performed to search for potential areas of submission application improvement. Results. Analysis revealed several application items that likely could be modified to enhance both applicant and reviewer needs. Among several observations, we noted likely ambiguity in how submitters may have interpreted the purpose and response expectations for some items, weaknesses on informing on program characteristics due to close-ended items’ exclusivity options, relationships between qualitative response length and meaningful program details, and ambiguous anchor qualifications for some interval items. Afterward, we delivered several recommendations, including re-considering the meaningfulness of entry items on informing evaluators’ assessments, splitting ambiguous entry items into multiple ones, hoping to capture more explicit foundation concepts, considering non-mutually exclusive response options (e.g., check all that apply) to multiple response items, requesting a minimal reporting standard for submitters entering qualitative information, informing reviewers of narrative length as an indicator of quality informing, and considering item-specific anchor qualifications across entry items formatted for interval data. Conclusions. The purpose of this exercise was to apply data collected for one purpose (i.e., reviewing, scoring, and selecting practices for LPP spotlighting) to address a secondary pursuit (i.e., continuous program improvement regarding the LPP’s application construction). We found many observations that resulted in recommendations for future iterations of this program. We believe these recommendations can produce more readily informing submissions and support evaluators in their challenging mission to assess, score, and select submissions for program spotlighting.

Location Name
Prince Georges Exhibit Hall A/B
Content Presented on Behalf of
DHA
Learning Outcomes
1. Be informed of a DHA program that intends to systematically canvas leading clinical practice solutions across MHS Direct Care for spotlighting and potential implementation and/or scale support.<br />
2. Appreciate secondary data analysis on a dataset beyond its principal purpose, but to mine for lessons in continuous program improvement.<br />
3. Be informed of recommendations to improve information collections.
Session Type
Posters