New possibilities in performance reporting are emerging due to artificial intelligence (AI), and among its compelling applications are predictive analytics and natural language processing (NLP).
The education technology company DeepLearning.AI describes NLP as a discipline that enables machines to understand, interpret, and generate human language. NLP can also analyze unstructured data from diverse sources. On the other hand, predictive analytics, according to IBM, uses historical data, statistical modeling, data mining techniques, and machine learning to make predictions about future outcomes.
Case Study: AI-Powered Performance Reporting in Healthcare
In its case study about UHS, Nuance stated that Dragon Medical One allows doctors to verbally record patient updates, current illness history, and treatment plans directly into their electronic health record (EHR) from almost any location. Using software like Microsoft Power BI, Dragon Medical One can also provide detailed insights into performance metrics at various levels. The reports include productivity forecasts, dictation quality data, and industry-wide comparisons.
Nuance pointed out the growing need for efficient documentation in healthcare as doctors are now being evaluated on quality metrics, which are publicly reported to the government. The software company also explained that the scores of physicians and hospitals are determined based on the accuracy of the documentation of patient conditions and the care provided. “Physicians, however, don’t always give themselves credit for the real medical complexity of the patient because of the extra time it takes to fully document it,” the report states.
In light of this, Nuance emphasized that CAPD aims to strengthen human intelligence by providing automatic suggestions to the doctor during patient care, but only when there is an alternate diagnosis or additional medical data that needs to be taken into account. This fully integrated system also allows UHS to analyze patient interactions in real time through the use of NLP.
Results of UHS’ initiatives show that there was a 12% increase in the case mix index (CMI) when physicians agreed with the CAPD clarifications and updated patient documentation. The healthcare AI system also improved the identification of “extreme” cases of disease severity by 36% and mortality risk by 24%. In addition, they recorded a 69% reduction in transcription costs year over year, resulting in $3 million in savings.
Next Steps
Companies planning to use AI in performance reporting can start by identifying areas in their operations where unstructured data is prevalent and manual processes are time-consuming. Next, it is important to develop an AI strategy that aligns with the company’s objectives.
Moreover, organizations should consider forming partnerships with AI solution providers due to their specialized expertise, experience, and ability to provide customized solutions more quickly and cost-effectively than developing in-house capabilities from scratch. Lastly, companies should invest in training their staff to work alongside AI technologies to cultivate a culture of innovation and continuous improvement.
While integrating AI into performance reporting is promising, it requires alignment with organizational objectives and flexibility from stakeholders. Click here if you’re interested in more practical applications of AI in strategy and performance management.
**********
Editor’s Note: This article was written with the help of Francesco Colamarino, a former Management Consultant at The KPI Institute.
The case below explores the implementation of a standardized Operational Deployment System (ODS) at Corewell Health West, a healthcare system in West Michigan. The goal of the system was to align operational processes and improve efficiency across physician and non-physician stakeholders. By implementing ODS, the organization aimed to enhance quality, increase patient satisfaction, optimize operational efficiency, and reduce costs while ensuring staff and physician satisfaction.
Background
Corewell Health West is a complex large healthcare system in West Michigan with 31,000 employees (4600 providers). Due to its large footprint in West Michigan, it aims for transformation to improve quality, increase patient satisfaction, deliver operational efficiency, and reduce costs. Foundational to all this work is staff and physician satisfaction. There was a need for shared language to communicate critical goals in a way that allowed us to be efficient while creating a standard approach to work. To move such a large team in one coordinated direction, Corewell Health needed to engage in focused efforts in a way that was respectful to its teams and leaders.
The ODS was designed to help leaders clarify what is most important and align the right resources to meet the goals set. This system, composed of best practices from individual project management and process improvement methodologies, was implemented to provide clarity, cascade goals appropriately, and help prevent employee burnout by creating a system of intentional alignment.
ODS Implementation Process
Implementing the Operational Deployment System begins with an annual goal-setting process led by the executive team and subject matter experts in the areas of cost, quality, people, and value. There is then a multi-week process of cascading these goals from the executive team through various levels of physician and operational leadership to front-line staff. Subsequent conversations called “catch-ball” follow in which each level of leadership discusses and eventually finalizes goals in each of the four categories. This process culminates with executive sign-off, confirming the roll-up of goals at each level to ultimately achieve the system goals. These goals are captured in a document called an Operational A3 (see sample). Each level of leadership, starting at the director level, has an OA3 that outlines the annual goal in each category and provides space for monthly data updates and explanations.
The manager level of leadership does not have an OA3 but instead utilizes a reporting tool called a gate chart (see sample). Each goal has a separate gate chart featuring a leading metric (the metric that aligns with the director OA3), a lagging metric, and specific tactics and timelines for impacting performance.
Following this goal-setting process and after populating the OA3 and gate charts, weekly report-outs begin each week focused on one of the four priority areas. Report-outs take place in a virtual meeting with managers reviewing the gate chart performance with front-line staff. This is followed by managers reporting their gate chart update to directors, who then provide a similar report to Physician and Operations Vice Presidents (VPs), and so on. Each of these report-outs follows the TAPE methodology, which stands for Target (what was the goal), Actual (what is the actual performance metric), and Please Explain (what were the actions or factors that contributed to that month’s performance).
Change Management
The ODS process inherently supports change management surrounding efforts to meet annual goals by engaging the front-line staff and every level of physician and operational leadership in goal setting, action plan development, and performance tracking. A key component of successful implementation is training leaders and teams in the ODS process. Training sessions for all levels of leaders included a review of the principles of ODS, the OA3 and gate chart templates, and the TAPE reporting format, and included time for discussion and questions. Implementing operational goals, management for daily improvement and cascade reporting, and communication were key areas of discussion during these training sessions.
Stakeholder Experience
To gauge the stakeholder experience, VPs and Director-level physician and operational leaders were surveyed about their experience with ODS. Among the 54 respondents, 61% agreed or strongly agreed that ODS has allowed them and their upline to focus on key areas for operational success. Moreover, 69% agreed or strongly agreed that ODS effectively aligns operational tactics with system strategy.
Lessons Learned and Next Steps
The ODS at Corewell Health initially faced challenges as leaders at all levels adjusted to this new form of tracking and presenting metrics. As the process matured, these perceived notions morphed into support, engagement, and eagerness to introduce new ideas.
Survey results indicate that the leaders perceive improved focus in key operational areas due to ODS. The system has been adopted outside of service lines as well. Hospital medical staff leadership embraces value in aligned goals and now reports on the executive dashboard. Independent physicians are looking at ways to use ODS to improve their private practice structure and function.
Implementing an Operational Deployment System at Corewell Health has been thought-provoking, enlightening, and rewarding. Previously top-down leadership in this space has moved to shared decision-making. As ODS progresses through year three, physician and operations leaders will build on lessons learned and broaden skills to make ODS an even richer process and a model for other organizations to follow.
********
Editor’s Note: The authors are Aiesha Ahmed MD, MBA (VP, Population Health, and Chief of Neuroscience); Rashelle Ludolph (Operations Director, Medical Specialty Services); Cheryl Wolfe MD, MBA (VP, Chief of Women’s Health), and Sonja Beute (Director of Strategic & Operational Deployment).
This article has been updated as of September 17, 2024.
Competency-based education (CBE), according to the non-profit organization EDUCAUSE, enables students to advance their development based on their learning capacity, skills acquisition, or competency improvement at their own pace. This learning journey could happen in almost all types of environments.
CBE creates opportunities for learners and employees to nurture integrated performance-oriented capacities that can help them handle different challenges. This strategy is designed to accommodate various learning styles and can result in more efficient learner outcomes.
CBE would not work well without high-quality trainer’s training. Trainer’s preparation is a critical component of successful CBE deployment. Trainers must learn how to assess learners and tailor learning experiences to their needs. They must have a thorough understanding of the CBE process, techniques for tailoring learning experiences, mastery-based assessment, and the critical role of technology in learning customization.
Developing competencies standards is a crucial part of CBE. It helps sustain skills at the enterprise level by defining an ideal competent performance to measure a worker’s actual performance. The competency criteria can be tailored to a specific country’s setting and confirmed through a local method. TheILO (2021) recommends considering the following factors when creating a local validation strategy:
The industry’s size and geographic dispersion (so representative businesses can participate)
The industry’s diversity (i.e., the technology utilized and the products produced)
The worker’s profile (to guarantee that all competencies are included)
Validation costs and validation time
They must be aware of the differences between CBE and traditional education (TE) while developing competency maps, assessment goals, learning speed, grading, and promotion. They must also understand the importance of assessment in adapting training.
CBE in Healthcare
Healthcare institutions and organizations are being confronted with a number of challenges, including developing clinical approaches and a scarcity of resources. On the other hand, healthcare providers are expected to use care skills effectively and put theoretical knowledge into practice (Eijkenaar et al., 2013; Goudreau et al., 2015). However, a growing amount of research from several countries demonstrates that the clinical performance of healthcare systems is not up to par (Eijkenaar et al., 2013).
Below is a case study demonstrating how CBE works, its effectiveness, and its application to professional practice. It includes a population of physicians, nurses, medical students (residents or interns), and nursing students. CBE courses were offered to improve the clinical performance of medical and nursing students, physicians, and nurses. Traditional education was provided to the control group, and the clinical performance of the study population was the outcome.
Thestudy “The effect of competency-based education on clinical performance of health-care providers: A systematic review and meta-analysis” aimed to evaluate the influence of competency-based education on the clinical performance of healthcare providers. The study takes into consideration the need to develop an intervention to improve the clinical performance of healthcare providers.
The criteria for inclusion were clinical or quasi-experimental trials; physician/nurse or medical/nurse participants; an age range of 18–65 years; adoption of the CBE approach; measurement of clinical performance of the participants using concrete tools of performance assessment; presentation of the results quantitative scoring; and the use of traditional teaching methods for the control group. The exclusion criteria are the use of a combination of CBE and other educational modalities, the lack of a control group, and the measuring of the participants’ theoretical knowledge or non-clinical abilities.
The study used six international electronic databases: PubMed, Ovid, Cochrane Library, Web of Science, ProQuest, Scopus, and Google Scholar. The national electronic databases IranMedex and SID were also used to conduct a systematic review and meta-analysis. In addition, the study comprised other studies that were related to the research goals. The Cochrane Collaboration’s Risk of Bias Tool was applied to assess the studies’ quality.
According to the random-effects model, CBE could improve the clinical performance of health care professionals in the intervention group compared to the control group (SMD = 2.717, 95 percent CI: 3.722 to 1.712).
Competency-based education can help health care practitioners improve their clinical performance. Meanwhile, high turnover rates, decreased job satisfaction, increased presenteeism, poor patient safety, and increased medical errors are consequences of a lack of clinical skills and competencies
