Developing Standardized Peer Review Guidelines For Adverse Events

Introduction: Adverse events (AE) are common in medical practice and are associated with poor patient outcomes, including prolonged hospitalizations, increased readmission rates, and reduced quality of life, which lead to increased hospital expenditures. Peer review is a cornerstone to evaluating AEs in the hospital setting and contributes valuable information to root-cause analyses. When executed effectively, peer review programs help identify system-based failures contributing to adverse clinical outcomes which can then be used to prevent patient harm on an institutional scale.

Many hospitals utilize standard forms when an AE review is required to establish standard of care. However, no evidence-based guidance exists describing what case data should be presented or what types of case assessments should be made to highlight system versus human factors that can inform root-cause analyses and systems safety solution development. The purpose of this study is to develop a consensus guideline for a standardized, health system-wide AE peer review tool as part of an AE analysis process.

Methods: The consensus guidelines were developed using a two-step modified Delphi method. This method uses successive rounds of questionnaires to obtain consensus from an expert panel and is frequently used in healthcare settings to develop guidelines for defined clinical problems where little definitive evidence exists.

Participants: Our expert panel included 30 Mount Sinai Health System faculty and staff. There were 6 nurse leaders, 4 participants from community hospitals, and 4 system leaders. All were required to have a background in AE peer review and a stake in the outcomes.

Survey Construction: Our study incorporated two rounds of email surveys. The initial survey was developed through a systematic literature review for existing peer review structure and design. Results from our review were used in conjunction with Mount Sinai’s existing peer review forms to develop a list of 15 multiple-choice survey questions organized into three content domains: peer review process/procedure (n = 3), relevant case details/information (n = 6), and event evaluation criteria (n = 6).

Survey Implementation: Surveys were disseminated via email. In round 1, panelists could provide written feedback to questions. Round 1 responses were collected and answer frequencies were calculated for each question. Question consensus was defined by answer choice agreement of ≥51%. Questions that were below this threshold were recirculated on round 2. In round 2, survey questions were presented alongside the panelist’s initial response and majority group response. Questions without round 2 consensus were excluded from the final consensus tool.

Results: The group agreed to the inclusion of 16 items for a peer review. Reviewers overwhelmingly agreed that both system and individual factors should be assessed during the review process. Some of the system factors that received high levels of consensus included workplace culture (87% agreement), workflow process (90% agreement), resource management (93% agreement), and work environment (77% agreement). The individual and team factors that were commonly favored included communication (100% agreement), supervisory availability (93% agreement) and quality (87% agreement). There was also a strong consensus among reviewers that peer review should include a summary assessment of the standard of care (96.7% agreement). The experts concurred that assessments should evaluate for delays in diagnosis or intervention, lapses in documentation, and failure to perform necessary testing or treatment.

Conclusion: Despite the recognized importance of peer-review, there are currently no evidence-based guidelines describing elements of a successful review process. This study provides a unique, consensus-derived peer review tool that incorporates input from multiple clinical and administrative stakeholders across the Mount Sinai Health System. When implemented, this tool will help standardize peer review, facilitate objective AE case assessments and root-cause analyses, and generate targeted solutions.