By Annie Callanan and Frank Mazza, MD
Measurement has proven foundational to advancing individual and collective performance in every business endeavor, vocational pursuit, professional sport, and recreational hobby. People do not always appreciate being measured, and some fear the implication more than others. But every successful advancement over the past century has been aided, and ultimately affirmed, by metrics that authenticate comparative achievement.
Measurements serve as foundational pillars underlying performance in every facet of our lives. They form the basis of compensation structures, bragging rights, passions, and failures. Measurements define individuals for whom they are and how they evolve and progress. A systematic approach to measurement has even permeated 21st-century philanthropy, thanks in no small part to the Bill and Melinda Gates Foundation, which counts deep inroads toward the ultimate elimination of malaria as a victory of actionable assessment (Bill & Melinda Gates Foundation, 2010; What we do, n.d.).
Healthcare would seem an unlikely candidate to eschew measurement as a validating force, given the industry’s penchant for randomized control trials, double-blind formats, and statistical process control charts. Given its rightful place in assessing virtually every manner of human performance, one might expect that measurement of all things safety-related would be pervasive and regarded as a core operating value. However, the reality of measurement in healthcare has proven to be more complex than originally envisioned. Staggering growth in the number of quality and patient safety indicators imposed upon healthcare organizations and the resulting burden on available resources have resulted in at least one high-profile recommendation that measurement be reduced by 50% (Stempniak, 2015). But is measurement the problem or the solution in helping make care safer for patients? Some argue that, when it comes to patient safety, we need to measure what matters in order to make a difference (Thomas & Classen, 2014). Herein lies our view.
Safety in healthcare suffers from a scarcity of comprehensive, credible, and sustainable measurement.
Safety is not impaired by excessive measurement. On the contrary, accurate safety measurement is ubiquitously lacking in healthcare. Safe care remains an aspirational goal that has yet to be precisely calculated, despite nearly two decades of effort. Progress has been made in defining and enabling Just Culture over the past 15 years. Nevertheless, credible comparisons of safety performance among physicians, nurses, other providers, and health systems are still elusive. Despite some declarations of major success in the effort to make care safer for patients (Rice, 2014), others would argue that improvement has been minimal at best (Waters et al., 2015; Lee et al., 2012).
Hospital systems measure what can be readily counted in small, fragmented doses. Knowledgeable safety gurus decry the “check the box” mentality that has consumed the quality and safety movements. Meanwhile, claims of large-scale improvement have been questioned as largely reflecting better documentation and coding on the part of healthcare organizations (Rice, 2014). With actual aggregate data, categories of known harm continue to expand as exemplified by electronic medical record–associated errors, with little hope for imminently identifying strategies aimed at their systematic prevention and mitigation.
Improving safety in healthcare requires a systematic approach to measuring, monitoring, and mitigating safety risks.
To bring true accountability and cost containment to healthcare while genuinely advancing patient safety, stakeholders must take a systematic approach to safety improvement.That implies a more exact science of unobtrusive collection of credible data and verification of care performance (i.e., actions) that is subsequently compiled within a dynamically structured and precise taxonomy. That also includes systematically measuring actions and comparing them against evidence-based standards, and aggregating those measures across all care settings. It involves identifying and measuring gaps between actual and optimal care, giving precise feedback on performance against those gaps over periods that demonstrate long-term sustainability. Finally, it requires establishing benchmarks for safety and metrics that deliver market transparency across a spectrum, from the individual provider to the organization.
Ironically, the technology and data to facilitate a verifiable approach to improving safety exist today. Incident reporting systems capable of aggregating, monitoring, and analyzing performance attributes in real time are already deployed in one form or another in nearly every healthcare facility in the country. Unlike EHRs, incident reporting systems were conceived as modern rapid-capture devices on the front end and dynamic intelligence-gathering data-marts on the back end. Capturing an adverse event, a near-miss hazard, or a procedural gap in care requires little or no training and minimal time away from patient care, and can be classified in a structured taxonomy to facilitate comparative analysis in as little as two minutes. When appropriately designed, these systems enable frontline staff to instantaneously record errors of commission and omission, as well as adverse events, without neglecting patient needs. When combined with an effective strategic approach to performance improvement, clinical leaders and patient safety experts can derive the sophisticated analytical insights they need to drive tangible, measurable improvements.
Incident reporting systems can enable prospective as well as retrospective improvements.
While widely recognized as instrumental in retroactively identifying and documenting contributing factors that lead to adverse events, incident reporting systems can also dynamically prompt preventive actions to reduce harm at the point of care. For example, a provider about to administer a narcotic can be prompted to verify whether her patient carries a diagnosis of sleep apnea or has physical features of that disorder, placing him at risk for respiratory complications. Proactively encouraging the provider to undertake and document appropriate mitigative strategies before she administers care associated with known safety risks can foster transparency and accountability while preventing the pursuit of a well-meaning but risky course of action.
Incident reporting systems have the capacity to record and measure actions associated with safe care performance.
Through event reporting, process and procedural actions that correlate to safe care performance can be systematically captured and comparatively scored against a set of evidence-based standards. But validating safe care and certifying related improvement requires establishing safety process and outcome baselines that can be captured broadly and assessed dynamically over time. To derive such baselines, safe care must be abstracted and captured as a series of performance actions across one or more procedures and/or care processes. The cumulative composite of these actions and any related omissions can be rapidly aggregated electronically, scored against precise standards of care, and compared on a risk-adjusted basis to form baseline performance standards. Safe care performance can then be comparatively assessed on an individual or systemwide basis against such standards, and the effect of efforts intended to improve performance can be measured over time.
Modern incident reporting systems can pinpoint procedural safety gaps in care.
Safe practices that mitigate certain risks have already been defined, established, and recognized as evidence-based within a growing cadre of tools. Precious time and resources are dedicated to repeated education and training of clinicians, but adherence to best practices is rarely tracked or systematically measured. Procedural gaps correlated to central line–associated bloodstream infections (CLABSI) and catheter-associated urinary tract infections (CAUTI), for example, can be captured electronically in real time and at the point of care by documenting performance actions against evidence-based checklists entered into an event reporting system. The systematic mash-up of technology, clinical best practices, and human factors can materially advance precision, accuracy, and recall around performance standards without alienating the cadre of well-intentioned frontline staff or weighing them down with onerous and nonintuitive intake mechanisms, particularly when done in the context of just culture.
Systematic measurement will facilitate proactive monitoring and mitigation of safety risks for patients receiving established procedural care.
Compliance with evidence-based practices necessitates instant recall at the point of care by those tasked with following such practices. Data suggests that barriers to compliance with best practices often stem from caregivers who don’t know or can’t recall what they should be doing, while laboring within a system of relentlessly increasing clinical and operational complexity. Smart integration of incident reporting systems with EHRs can aid in the proactive identification of high-risk conditions and patients, with the intent of mitigating preventable harm. Matching risk profiles from incident reporting systems with patient data from EHRs can direct decisions on when, and in what manner, to enable precise application and documentation of care in the spirit of real-time decision support. Capturing provider performance for high-risk or error-prone conditions and patients will accelerate the identification of both individual and system failure modes, and present clear, unambiguous targets for focused improvement efforts. As an example, while checklists can successfully remind providers of rule-based tasks that need to be done and capture their successful performance, real-time or retrospective review of actions taken and compliance with evidence-based practice can prevent harm from occurring in the first place.
Stakeholders who care about advancing patient safety should demonstrate a commitment to promoting systematic solutions.
Incident reporting systems were built to identify and categorize errors and adverse events so that appropriate improvement efforts can be undertaken, but they are still only fulfilling a small fraction of their potential. Modern technology can deliver the way to cost-effectively systematize and measure adherence to evidence-based practices. The question is whether the will exists to do so. Patient safety experts continue to call for the development of comprehensive patient safety systems to identify, report, and analyze data. These systems already exist. What is missing, however, is systematic commitment to explicitly, transparently, and in a nonpunitive manner capture all actions associated with care performance in a way that truly analyzes and codifies failure points in care. Pay-for-performance and the demand for reliable execution around numerous quality and patient safety indicators may perversely cause frontline providers to labor single-mindedly on specific tasks rather than focus on system deficiencies that lead to errors and adverse events. Senior leaders maintain overall accountability for creating and supporting an environment where the pursuit of safe care becomes the highest priority, but commonly fail to appropriately message and model necessary behaviors that assure providers they truly mean what they say. The failure of leadership to understand and follow through with their supportive role in this regard represents the greatest contribution to the normalization of behaviors practiced among providers that prevent safer and more reliable care.
Government incentives should foster innovation around the systematic advancement of safety performance.
As aggregate safety data suggest (Lee et al., 2012; Waters et al., 2015), the federal dollars applied to date have had minimal impact on containing the problem of preventable harm and have not identified collective failure points or rewarded systematic remediation. Short-term government grants, such as the Partnership for Patients’ HEN 2.0 grants, require healthcare organizations to perform tests of change meant to improve care, collect data, and demonstrate measurable improvement—all within a one-year time frame. That favors a human capital and consulting approach. Hiring temporary staff members to “nag” providers to do the right thing is neither sustainable nor cost-effective. Broadly introducing new and innovative safe practices to transform systems is more likely to be successful.
Meaningful use incentives applied to incident reporting systems can be one effective catalyst for aligning around new and innovative priorities. System enhancements to incident reporting must shift away from what is today largely an “intake orientation” that supports reactive reporting of incidents of harm to an outcomes-based orientation centered on the proactive reporting of actions that constitute safe performance.
Measuring care performance will lead to a service-oriented culture committed to embracing accountability and improvement.
The time has come for healthcare to affirm the intelligent, efficient, and systematic capture of actions compared to precisely defined safety standards. Doing so will accelerate the ability to proactively monitor hazards in real time and mitigate risks to both patients and providers. Incident reporting systems programmed to track process and outcome measures derived from performance monitoring will accelerate proactive analysis of gaps in care across a broad range of settings. Transparency of performance and continuous improvement will be properly centered on the people who deliver care each day, for the benefit of both givers and receivers of care. Performance measurement is the enabler, not the enemy. It can be done effectively in healthcare. The time to do so is now.
Annie Callanan, president and CEO of Quantros, has more than 25 years of senior-level IT experience. Prior to joining Quantros, she served as chief operating officer of Systech International and of ProQuest, a private equity–backed information company, where she spearheaded major strategy and technical shifts across the company’s portfolio of products and services. She may be contacted at firstname.lastname@example.org.
Frank Mazza is chief medical officer of Quantros. He is a physician by training (pulmonary, critical care, and sleep disorders) and still practices medicine part time. Prior to joining Quantros, he held several executive positions within the Seton Healthcare Family in Austin, Texas, including system-level chief patient safety officer and associate chief medical officer, as well as vice president of medical affairs at Seton Medical Center, Austin. He may be contacted at email@example.com.
Bill & Melinda Gates Foundation. (2010). A guide to actionable measurement. Retrieved from https://docs.gatesfoundation.org/documents/guide-to-actionable-measurement.pdf
Lee, G. M., Kleinman, K., Soumerai, S. B., Tse, A., Cole, D., Fridkin, S. K. … Jha, A. K. (2012, October 11). Effect of nonpayment for preventable infections in U.S. hospitals. New England Journal of Medicine, 367, 1428–1437.
Rice, S. (2014, December 2). 1.3 million adverse events prevented in U.S. hospitals since 2010, feds say. Modern Healthcare. Retrieved from http://www.modernhealthcare.com/article/20141202/NEWS/312029936
Stempniak, M. (2015, December 11). Don Berwick offers health care 9 steps to end era of “complex incentives” and “excessive measurement.” Hospitals & Health Networks. Retrieved from http://www.hhnmag.com/articles/6798-don-berwick-offers-health-care-9-steps-to-transform-health-care
Thomas, E. J., & Classen, D. C. (2014, May 6). Patient safety: Let’s measure what matters. Annals of Internal Medicine, 160(9), 642–643.
Waters, T. M., Daniels, M. J., Bazzoli, G. J., Perencevich, E., Dunton, N., Staggs, V. S., … Shorr, R. I. (2015). Effect of Medicare’s nonpayment for hospital-acquired conditions: Lessons for future policy. JAMA Internal Medicine, 175(3), 347–354. doi:10.1001/jamainternmed.2014.5486
What we do: Malaria: Strategy overview. (n.d.) Retrieved from http://www.gatesfoundation.org/What-We-Do/Global-Health/Malaria#OurStrategy