A REVIEW ON DIFFERENT OPERATIVE RISK CALCULATORS Dr Kitty Lo Queen Elizabeth Hospital

C ASE S CENARIO 88/M PMH: DM, HT, COPD, IHD, CRF E admitted with fever & severe abd pain BP 100/70 P125 Hb 8.5, WCC 22, Urea 18, Cr 250, K5.5 ABG pH 7.20, PaO 2 9, HCO 3 13 CT: Perforated CA caecum, no distant metastasis

W OULD Y OU O PERATE ON T HIS P ATIENT ?

WHY DO WE NEED OPERATIVE RISK CALCULATORS?

1. Guide decision making & informed consent and improve treatment planning 2. For surgical auditing & comparison of outcomes to improve quality of care

HOW TO DEFINE IF A PREDICTION MODEL IS ACCURATE? Discrimination Calibration Observed: Estimated Ratio

DISCRIMINATION Ability of the model to assign higher probability of outcome to patient who actually die than those who live Area under the receiver operative characteristic curve (c-index) Value between 0.5 (random)-1.0 (perfect) Values : reasonable discrimination Values >0.8: good discrimination

CALIBRATION Ability of the model to assign the correct probabilities of outcome to individual patients Hosmer-Lemeshow χ ² statistic The smaller the value, the better the calibration

O:E MORTALITY RATIO Ratio of observed no. of deaths to expected no. of deaths as calculated by the prediction model In validation studies: 1.0 = perfect prediction

COMMONLY USED MODELS

RISK SCORING SYSTEMS APACHE SAPS POSSUM & its variations ACS NSQIP Dedicated for general surgery patients

APACHE Acute Physiology & Chronic Health Evaluation

APACHE- ACUTE PHYSIOLOGY & CHRONIC HEALTH EVALUATION Developed for use in ICU setting APACHE II: 12 physiological parameters in the first 24h after hospital admission Score modification available for surgical patients undergoing elective or emergency surgery

APACHE II Pros: Reproducible & objective Can be applied prospectively & retrospectively Cons: Too complex for general surgical use Not all parameters are routinely measured

APACHE II Validity: Extensively validated in ICU patients all over the world 1-3 Tend to overpredict mortality in surgical patients 4 Trend to underpredict mortality in high risk cases and overpredict in low risk cases 1- Hariharan S, Moseley HSL, Kumar AY. Outcome evaluation in a surgical intensive care unit in Barbados, Anaesthesia. 2002; 57: Beck DH, Taylor BL, Millar B, et al. Prediction of outcome from intensive care: a prospective cohort study comparing Acute Physiology and Chronic Health Evaluation II and III prognostic systems in a United Kingdom intensive care unit. Crit Care Med. 1997;25: Gianguiliani G, Mancini A, Gui D. Validation of a severity of illness score (APACHE II) in a surgical intensive care unit. Intensive Care Med, 1989; 15: Colpan A, Akinci E, Erbay A, et al. (2005). Evaluation of risk factors for mortality in intensive care units: a prospective study from a referral hospital in Turkey. Am J Infect Control 33:42-47.

SAPS II Simplified Acute Physiology Score II

SAPS II- SIMPLIFIED ACUTE PHYSIOLOGY SCORE Variant of APACHE 17 variables Use data obtained during the intensive care/ early post-op period

SAPS II Validity: Extensively validated in ICUs over the world Not as commonly used as APACHE II Conflicting results Some studies found SAPS II as a better predictor when compared with other scoring systems 1-2 Other studies have found it to be less effective with relatively poor goodness-of-fit 3 1- Can MF, Yagci G, Tufan T, Ozturk E, et al. Can SAPS II predict operative mortality more accurately than POSSUM and P-POSSUM in patients with colorectal carcinoma undergoing resection? World J Surg. 2008; 32: Capuzzp M, Valpondi V, Sagarbi A, et al. Validation of severity scoring systems SAPSII and APACHE II in a single centre population. Intensive Care Med. 2000; 26: Sculier JP, Paesmans M, Markiewicz E, Berghmans T. Scoring systems in cancer patients admitted for an acute complication in a medical intensive care unit. Crit Care Med. 2000; 28:

POSSUM Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity

POSSUM- P HYSIOLOGICAL AND O PERATIVE S EVERITY S CORE FOR THE EN U MERATION OF M ORTALITY AND MORBIDITY Initially developed in 1991 for surgical audit 12 Physiological and 6 Operative factors Can predict both 30 day mortality and morbidity Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg. 1991; 78:

POSSUM Validity: Extensively studied and applied internationally over the general surgical spectrum POSSUM mortality equation found to consistently overpredict deaths; up to 7-fold in low risk patients 1-3 POSSUM was accurate in predicting post-op morbidity 1 1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post-operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14: Whiteley MS, Prytherch DR, Higgins B, Weaver PC, Prout WG. An evaluation of the POSSUM surgical scoring system. Br J Surg Sep; 85 (9): POSSUM, p-POSSUM and Cr-POSSUM: Implementation issues in a United States health care system for prediction of outcome for colon cancer resection. Dis Colon Rectum 2004; 47:

POSSUM Inherent problems: The equation gives a minimum risk of death of 1.1%, far too high for minor procedures and fit patients Physiological parameters are measured at the time of surgery, scores will vary depending on aggressiveness of resuscitation given pre-op Operative severity scores includes blood loss and need for reoperation, which may be surgeon- dependent Morbidity and complication is not easy to define

P-POSSUM – P ORTSMOUTH POSSUM A modification of the POSSUM system Uses the same physiological and operative variables as POSSUM Recalibrated to a different regression equation Only predicts in-hospital mortality 1- Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg. 1998; 85:

P-POSSUM Developed by analyzing surgical procedures 1 Using the first 2500 patients as a training set to produce the P-POSSUM predictor equation Then applied prospectively to the remaining 7500 patients for validation 1- Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg. 1998; 85:

CR-POSSUM- C OLO R ECTAL POSSUM Developed specifically for colorectal surgery Predicts in-hospital mortality Developed from data from almost 7000 patients Tekkis PP, Prytherch DR, Kocher HM, et al. Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br J Surg 2004;91:

A S YSTEMATIC R EVIEW OF POSSUM, P-POSSUM & CR-POSSUM IN C OLORECTAL C ANCER 19 studies 6929 patients To compare the predictive value of POSSUM, P- POSSUM and CR-POSSUM in colorectal cancer surgery Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post- operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:

S UMMARY D ATA 1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post- operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:

P-POSSUM was most accurate model for predicting post-operative mortality after colorectal cancer surgery POSSUM was accurate in predicting post-op complications A S YSTEMATIC R EVIEW OF POSSUM, P-POSSUM & CR-POSSUM IN C OLORECTAL C ANCER 1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post- operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:

O THER S PECIALTY -S PECIFIC POSSUM MODELS O-POSSUM (oesophago-gastric) V-POSSUM (vascular)

ACS NSQIP American College of Surgeons National Surgical Quality Improvement Program

ACS NSQIP - A MERICAN C OLLEGE OF S URGEONS N ATIONAL S URGICAL Q UALITY I MPROVEMENT P ROGRAM Data from 393 US hospitals, 1.4 million patients Developed a universal surgical risk estimation tool Use 21 preoperative factors to predict 8 outcomes Mortality Morbidity Pneumonia Cardiac event Surgical site infection Urinary tract infection DVT Renal failure Bilimoria KY, Liu YM, Paruch JL, et al. Development and evaluation of the Universal ACS NSQIP surgical risk calculator: A decisional aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217:

A CS NSQIP Validity: Excellent discrimination for mortality (c-statistic 0.944), morbidity (c-statistic 0.816) and the 6 additional complications (c-statistic >0.8) Limitation: Does not account for the indication of the procedure Application: Calculated prospectively as a decision-support tool Quality indicator for surgical auditing Bilimoria KY, Liu YM, Paruch JL, et al. Development and evaluation of the Universal ACS NSQIP surgical risk calculator: A decisional aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217: Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery

CASE SCENARIO Proceeded to EOT with Right hemicolectomy performed Post op not extubated to ICU, required mechanical ventilation & CVVH & inotropic support Complicated by chest infection and AMI in early post op period After a stormy initial period, finally made a full recovery and discharged home on day 30

P REDICTED R ISK

C ONCLUSION No scoring system is perfect Serve as a guide to informed consent and manage expectations Identify high risk patients preoperatively & facilitate decision making regarding intensity of post-op monitoring Role in surgical auditing

T HANK Y OU

WHAT IS A SURGICAL AUDIT? A systematic appraisal of the implementation and outcome of any process in the context of prescribed targets and standards Commonly assessed outcomes include 30 day or in-hospital mortality and morbidity

Crude morbidity & mortality affected by many factors Account for the variation in the physiological condition of the patient and the severity of the procedure (Case-mix) for fair comparison Allows for comparison of performance between individual hospitals and surgeons

A PPLICATION OF ACS NSQIP Annals of Surgery 2015

ACS NSQIP Prediction models for mortality & morbidity recalibrated every 6 months Each hospital given risk adjusted O/E ratios that permit comparison & for targeting quality improvement efforts Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery

Improving performance over time Estimate 0.8% reductions in mortality, 3.1% reductions in morbidity and 2.6% reductions in surgical site infection annually The longer the duration of time in the program, the greater the magnitude of quality improvement Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery

P OSSUM M ORBIDITY Haemorrhage Wound and deep Infection Chest, UTI, Deep, Septicaemia, PUO Wound dehiscence Superficial and deep Anastomotic leak Thrombosis DVT, PE, CVA, MI Cardiac failure Impaired renal function Hypotension Respiratory failure

WHAT IS PREREQUISITE FOR A GOOD MODEL? Quick and easy to use Small number of variables Reproducible Applicable across the general surgical spectrum, both elective & emergency Validated internationally by a large number of centres