Effects of Checklists in Surgical Care - a Study on Complications, Death and Quality of Patient Administrative Data

1.0 Background Surgical procedures are high risk events and patients may suffer complications or die post operatively. A report from an on-going patient safety campaign "In Safe Hands" lead by the Norwegian Knowledge Centre for the Health Services reveals that approximately 16 % of all Norwegian hospital admissions in 2010 involved an adverse event (AE) (Deilkås, 2011). A review study on AEs in 2008 included a wide range of in-hospital patients from Australia, Canada, New Zealand, the United Kingdom, and the United States of America (US) (de Vries et al., 2008). 9 % of the patients experienced an AE, with 7. 4 % of these ending fatally. The majority of the AEs occurred during surgical treatment or was related to drug administration. The authors claimed that almost half of these could have been prevented if checklists covering the entire surgical pathway had been used (de Vries et al., 2008). Implementation of such a system, called the Surgical Patient Safety System (SURPASS) did in fact result in a reduction of in-hospital morbidity (from 27.3% to 16.7%) and mortality (from 1.5 % to 0.8 %) (de Vries, 2010).

Patient safety checklists have been introduced and recommended as a standard of surgical care (Birkmeyer, 2010; de Vries et al., 2011). Studies based on data from electronic patient administrative systems show that checklist use may reduce mortality and morbidity in surgery (de Vries et al., 2010; van Klei et al., 2012; Haynes et al., 2009). Safe Surgery checklists have been recommended by the World Health Organization (WHO) since 2008 as a strategy to avoid adverse events (AE) during surgery. More than 6000 hospitals have implemented Safe Surgery checklists in their operating theatres (OTs) (http://www.who.int/patientsafety/safesurgery/en/), including Haukeland University Hospital (HUH).

This multicentre research project will also introduce a system of patient safety checklists at each point of care during the surgical patients' stay, not only in the operating theatres (OTs). The system combines new checklists on patient care (parts of SURPASS) with the already established Safe Surgery checklist (WHO) in the OTs. At the same time securing reliability, validity and quality of the patient, morbidity and mortality data will be an essential part of the study.

Today the discharging physician reviews the medical journal and makes a medical summary including coding diseases and complications relevant for the current admission. International Classification of Diseases (ICD-10) codes are used to set diagnoses for clinical, epidemiological and quality purposes (http://www.who.int/classifications/icd/ICD10Volume2_en_2010.pdf). The ICD-10 codes are also used for registrations on national mortality and morbidity in the Norwegian National Patient Register (NPR). Questions have been raised as to the accuracy and quality of the data in such registers in Norway, e.g. in patients with sepsis (Flaatten, 2004), and intensive care patients (Aardal et al., 2005). In a Danish study on relations between ICD-10 coding in the National Registry of Patients and the hospitals' discharge summary and medical records, a high reliability between ICD-10 scores and co-morbidity was found (Thygesen et al., 2011). To our knowledge similar studies have not been done in Norway. As a crucial part of this investigation we concurrently will evaluate the reliability and validity of our patient administrative data by comparing the post discharge ICD-10 codes to actual data available directly from medical journal systems as documented by health care personnel in the journal texts.

2.0 Objective

The main objectives of this study are to:

• Perform a systematic review of published studies on effects of safety checklists in medicine.
• Explore effects on morbidity and mortality after implementing a system of patient safety checklists at each point of care during the surgical patients' stay (elements of SURPASS and the WHO Safe Surgery list combined), in a cohort of surgical patients in different surgical departments in one hospital, with patients from departments not having the system introduced serving as controls from three hospitals.
• Investigate the validity of the post discharge ICD-10 codes for complications compared to actual information found in medical journal systems texts.

3.0 Methods 3.1 The projects and design

1. Systematic review A systematic review on effects of safety checklists in medicine was done on May 29th, 2012 in the databases MEDLINE, Cochrane library, EMBASE and Web of Science, limited to only humans. The criteria were pre-set and included all time published quantitative studies in any language in the in-hospital and pre-hospital setting where safety checklists were the sole intervention, and effects of using checklists, generated as measurable outcomes. 7408 singular articles were found. To ensure the transparency of the reviewing process we used the PRISMA guidelines (Liberati et al., 2009). 34 studies met our inclusion criteria. The majority of the included studies measured effects pre-and post-intervention and was classified as having an observational design. . This systematic review has identified that safety checklists can be effective safety tools in various clinical settings. Their use has reduced patient mortality and morbidity. In addition, safety checklist use has been associated with better human performance, improved compliance with evidence-based practices, promoted consistency of care, and reduction of technical omissions. None of the included studies reported that safety checklists have negative effects on patient safety issues.
2. Implement the new patient safety checklist system and measure effects on morbidity, mortality and length of hospital stay. A prospective stepped wedge trial design (Brown \& Lilford, 2006; Brown et al., 2008) will be used when implementing the validated patient safety checklist system in the Neurosurgical Department, the Orthopaedic Clinic and the Department of Gynaecology and Obstetrics at HUH. Patients from departments not using the patient safety checklist system serve as controls, this includes the Head and Neck Clinic (HUH), the Thoracic Surgery Section of the Heart Department (HUH) and two hospitals outside our own municipality (Health Trust Førde, and Health Trust Fonna - Haugesund Hospital). Primary end-points to be measured prospectively include length of hospital stay and morbidity and mortality utilizing the ICD-10 codes for complications collected electronically from the hospital patient administrative systems.
3. Validation of morbidity and mortality data Today ICD-10 codes are produced by discharging physicians to summarize diagnoses at discharge and any complications having occurred during patient stay. In order to validate HUH's and Health Trust Førde's ICD-10 coding on patient morbidity and mortality we will randomize inclusion for quality check comparing the ICD-10 codes used at discharge to all actual information on morbidity and mortality as documented in the electronic patient journal (EPJ) - DIPS. This validation should include approximately 700 patients, all having undergone major surgery. Such a comparison is essential to gain knowledge on the quality of generated ICD-10 data and thus important to the quality of results in this study.

3.2 Intervention study sample Three surgical units at HUH (Department of Neurosurgery, Orthopaedic Clinic, and Department of Gynaecology and Obstetrics) will have the checklist system implemented. Approximately 3700 patients will be included before and 3700 patients after checklist implementation. The Control Group includes 7400 patients.

3.4 Data collection For the study on mortality and morbidity we will extract ICD-10 codes used at discharge from the hospitals NPR file, as all Norwegian hospitals report their ICD-10 codes and procedure codes to NPR. In addition to registering all ICD-10 codes on each patient, we will collect demographic data (age, gender, height and weight), American Society of Anaesthesiologists Physical Health Classification (ASA), dates of admission and discharge, and all surgical procedures and major treatments. Data will be processed through Webport using a system previously developed locally for the WHO Surgical Safety Checklist project.

The primary end points, morbidity and mortality, are registered during hospitalization and postoperatively up to 30 days. Morbidity will be registered as major complications according to the American College of Surgeons' National Surgical Quality Improvement Program (http://www.facs.org/cqi/outcomes.html): organ/space surgical site infection, wound dehiscence, deep vein thrombosis, pulmonary embolism, pneumonia, re-intubation, ventilator use longer than 24 hours, cardiac arrest, myocardial infarction, sepsis, shock, coma longer than 24 hours, prosthetic/graft failure, and bleeding. Additional complications to these, as reported by de Vries (2010) will be included in order to make comparisons possible.

The study investigating reliability and validity of the ICD-10 codes will be done in detail: A prospective random selection of 700 patients, 200 patients from Health Trust Førde and 500 patients from the HUH, all having undergone major surgery. Present knowledge should suggest one or several major complications caused by procedures or iatrogenic causes in at least 17 % the surgical patients (de Vries, 2010). Then an inclusion of 700 patients is needed in order to find such complications in 119 cases. We will identify all post discharge ICD-10 codes for each patient. These codes will be thoroughly reviewed for accuracy and completeness by comparing to the actual information as documented by physicians and nurses in the EPJs throughout the total hospital stay. Primary outcome is here to investigate that registered ICD-10 codes have adequate sensitivity and specificity compared to the information in the patients' medical journal.

3.5 Statistics Descriptive and inferential statistical methods will be used to analyse data. Confidence intervals (95% CI) for sensitivity and specificity will be calculated using the normal approximation for the standard error of proportions.

Mortality and morbidity will be analysed as to time of measurement, e.g. pre and post intervention, and surgical unit, i.e. using or not using the checklist. Multiple regression analysis and other appropriate statistical tools will be used to adjust for covariates to mortality and morbidity. Calculation of sample size and power, with an expected mortality rate decrease (0.015 vs. 0.008) requires a sample size of 3641 patients in both baseline and post intervention groups with an alpha (0.05, 2-tailed), power is 80%. To calculate sample size and power for morbidity mitigation from 27% to 17% (de Vries et al., 2010) requires a much smaller sample size of 234 in baseline and post intervention groups to constitute an 80% power with alpha at 0.05, 2-tailed. Statistical analysis will be conducted with appropriate statistical software e.g. Statistical Package for the Social Sciences, Stata or R.