Editor’s Notebook: Evidence and Criteria
September / October 2008
Editor’s Notebook
Evidence and Criteria
For as long as healthcare professionals have worked to improve patient safety, they have debated which criteria are appropriate for evaluating improvement initiatives and what evidence is required before programs are widely implemented. Some of the most prominent leaders in safety and quality improvement — Don Berwick, Bob Wachter, Peter Pronovost, Paul Batalden, and others — have debated publicly and in print about the role of criteria and evidence. The debate has intensified, with positions taking shape around the choice between “action” and “evidence,” or between the urgency of action in the face of unacceptable lapses in safety versus the importance of caution in the face of insufficient scientific evidence. The use of Rapid Response Teams is one example of a hotly debated program. Most stakeholders believe that RRTs save lives, but clinical studies have yielded contradictory results.
AHRQ has issued a Request for Proposal (RFP) for a grant that should help resolve this debate. The RFP is titled “Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria.” With it, AHRQ aims to
…benefit researchers and others seeking to evaluate the results of patient safety practice implementation, as well as those who seek to synthesize the evidence on the effectiveness and safety of patient safety practices within specific organizational and systems contexts.
The details of this debate may sound academic, but the issues have profound implications with broad effects. Later in the RFP, under the heading “User-friendliness/plain language,” AHRQ acknowledges that a diverse community of individuals will benefit from this project:
…explicit consideration must be given to how the approaches and results of the effort will be made understandable and user-friendly for a broad range of stakeholders, including patients, providers, health plan administrators, and policymakers.
Unfortunately, there is no equivalent in quality improvement for the randomized controlled trial (RCT), which sets a universally accepted benchmark for clinical interventions. Indeed, some safety improvements seem so obvious as to render RCT inappropriate — a point made with elegant humor by Smith and Pell in their article, “Parachute use to prevent death and major trauma related to gravitational challenge: Systematic review of randomized controlled trial,” published by the BMJ in 2003.
Some who work in quality improvement argue that it is irresponsible to promote interventions that have not proven themselves through rigorous study, especially when those programs are implemented widely, at great expense, without clear understanding of all of the consequences — intended and especially unintended. Others point out that appreciation of context — the bane of RCTs — is often the crux of improvement strategies. Usually improvement cannot not be accomplished or sustained without giving the messy business of social interactions, communication, power, and organizational context its due. Gray area abounds and fuels the debate.
Reading the RFP is valuable for anyone with more than a passing interest in safety and quality improvement, which is to say for all readers of PSQH. The RFP is available at http://www.ahrq.gov/fund/contarchive/rfp0910001.htm. For those interested in applying for the grant, notice-of-intent letters are due on September 13, and the deadline for proposals is October 13.