Accounting for Patient Risks: A Prerequisite to Quality Improvement
September / October 2012
American College of Surgeons
Accounting for Patient Risks: A Prerequisite to Quality Improvement
Taking into account the health risks posed by the condition of each individual patient is not just critical to improving quality—it is a prerequisite.
To fairly measure a hospital’s performance, any assessment must consider the health risks posed by the condition of each individual patient, as well as the hospital’s case mix, given that some hospitals may take on more complex surgical cases than others.
If you do not adjust for the risks of patients facing a particular type of surgical procedure, you do not have a fair or valid basis on which to assess and compare the quality of the outcomes. For example, the risks of a surgical procedure faced by a chronically ill 75-year-old are likely to exceed those faced by a healthy 21-year-old. To compare their surgical outcomes without taking into account the pre-existing condition of the patients is like comparing apples to oranges—neither fair, valid nor, ultimately, useful.
The very terms “quality” and “improvement” presume the ability to compare. “Quality” presumes the ability to say that one surgical outcome is better or worse than another, while “improvement” presumes the ability to compare surgical outcomes over time.
Any quality measurement program that does not adjust for the patient’s condition prior to an operation cannot offer a basis for accurately benchmarking and comparing surgical outcomes among hospitals. In fact, public reporting of outcomes, without adjusting for the risk of the patient, can penalize physicians for taking on the toughest cases. For instance, studies have shown that public reporting of cardiac surgery procedure outcomes in New York and Pennsylvania has prompted some physicians in those states to avoid difficult, complex cases. Unfortunately, most quality improvement programs could not properly adjust for the patient’s risks even if they wanted to.
As we saw in the previous article in this series (“The Critical Importance of Good Data to Improving Quality,” July/August 2012), most quality improvement programs collect patient data from medical bills and insurance claims—in other words, they use administrative data. While some of these quality improvement programs may try to use administrative data to risk adjust, studies show that administrative data is not up to the task (Best, 2002; Ghaferia, 2009).
The desire for data that would adjust for a patient’s risk and a hospital’s case mix helped spur the American College of Surgeons to develop the National Surgical Quality Improvement Program (ACS NSQIP®). ACS NSQIP collects information from patients’ medical charts—in other words, it uses clinical data. With roots in a similar program developed by the Department of Veterans Affairs, ACS NSQIP is the nation’s first and only risk-adjusted, clinical, outcomes-based program to measure and improve the quality of surgical care across specialties in the private sector. One study shows that the average ACS NSQIP hospital prevents, on average, from 250 to 500 complications and from 12 to 36 deaths per year (Hall, 2009).
Using data from patient charts, ACS NSQIP can account for each patient’s age, presence of chronic conditions, and other risk factors. Logistic regression models are used to construct patient-level predicted probabilities for outcomes across all hospitals to arrive at an “expected” outcome for the type of patient and procedure. These probabilities are summed to estimate each hospital’s risk-adjusted probability of the outcomes of interest and are reported as odds ratios. An odds ratio of less than 1.0 indicates better-than-expected outcomes, while an odds ratio greater than 1.0 indicates worse-than-expected outcomes. These models have been in use for more than 20 years and are regularly updated as additional data is collected and reviewed by expert surgeons. In this way, ACS NSQIP’s risk-adjusted data enables any one hospital’s outcomes to be meaningfully calibrated against those at other similar hospitals.
Risk-adjusted data enables surgeons and hospitals to compare their outcomes with other surgeons and hospitals to help them understand where they are doing well and where they need to improve. For instance, the University of Utah’s academic medical center hospital in Salt Lake City was surprised to find its urinary tract infection (UTI) rate was twice as high as it should be from its ACS NSQIP semi-annual report. The center instituted a quality improvement program, and found that by better managing its Foley catheters, it was able to significantly reduce its UTI rate to below average levels.
“We wouldn’t have noticed that problem without risk-adjusted data,” observed Sean Mulvihill, MD, FACS, Huntsman Cancer Institute, University of Utah. “You don’t really know what’s acceptable or not without good, comparable data.”
Such data also helps surgeons to do a better job when it comes to prescribing, preparing for, and following up on any given procedure for any given patient. After all, to properly apply best practices, surgeons must consider the condition of the patient. No longer can surgeons dismiss potentially substandard outcomes by claiming “my patients are just sicker than your patients.” Better yet, ACS NSQIP offers evidence-based guidelines that can guide improvement. For instance, consider surgical site infections (SSI) – ACS NSQIP guidelines list surgical patient risk factors for SSI, and pre-, intra- and post-operative strategies to help prevent infection.
Using risk-adjusted data, ACS recently developed a risk calculator tool that can be used in pre-surgical consultation to inform patients about their individual risk of a postoperative complication for a variety of procedures. In the past, surgeons have assessed the risks of surgery through the lens of their own experience, and shared this information with their patients verbally. Now ACS NSQIP data has been analyzed and compiled into a risk calculator tool that allows clinicians to use an individual patient’s risk factors, such as age, sex, and BMI, to make more informed decisions about that patient’s risk of experiencing various surgical outcomes. The tool is an easily understandable aid for the surgeon to discuss risk with the patient, empowers the patient to make a more informed decision about a given procedure, and helps the surgeon set reasonable expectations.
The calculators provide better predictive ability than most other models across a range of outcomes, including morbidity, serious morbidity, and mortality. By comparing a given patient’s condition against outcomes from ACS NSQIP’s extensive database, the calculators provide surgeons with an easy-to-understand chart that compares their patient’s risks from a given procedure, such as colorectal surgery, to those of composite sick and healthy patients. There are risk calculators for common procedures such as pancreatectomy, colorectal and bariatric surgeries, and calculators for additional procedures are in development.
ACS NSQIP’s ability to adjust for patient risks was essential to a new quality improvement effort called the Florida Surgical Care Initiative (FSCI). To combat perceptions of Florida as a higher-cost state, the ACS partnered with the Florida Hospital Association to introduce a focused version of ACS NSQIP, using four measures targeted to a state with an older, higher-risk patient population. To date, 67 hospitals have joined FSCI, making it the largest statewide surgical quality initiative in the nation. The initiative will enable Florida to not only get a fair assessment of the quality of its surgical care, but it will also provide the basis to improve that care.
| In this series, Dr. Clifford Ko of the American College of Surgeons discusses the key tenets of quality improvement and then specifically breaks down an important element of any quality program—data collection. Data collection is one of four key principles to continuous quality improvement, but many healthcare organizations today may not be collecting the best data or using their data effectively. This series will focus on the elements of robust data and what they mean for quality improvement: clinical versus administrative data sources, risk-adjustment, post-discharge outcomes measurement, and national benchmarking, as well as the power of collaboration and sharing data to improve care. This is the third article in the series. |
The effectiveness of any quality improvement program depends on its ability to compare hospitals to each other and to national benchmarks. The ability to compare, in turn, depends on the ability to risk-adjust patient data and to account for a hospital’s case mix.
It’s worth repeating: risk-adjusted patient data is not just critical to improving quality, it is a prerequisite. ?
Clifford Ko serves as director of the American College of Surgeons Division of Research and Optimal Patient Care, which administers ACS NSQIP. He is a practicing general and colorectal surgeon, who serves as professor of surgery and health services at UCLA Schools of Medicine and Public Health, director of UCLA’s Center for Surgical Outcomes and Quality, and a research scientist at RAND Corporation. He holds a medical degree, BA in biology and MS in biological/medical ethics from the University of Chicago, and a MSHS in health service research from the University of California. Ko can be contacted at cko@facs.org.
References