Leapfrog’s “Survival Predictor”: Composite Measures for Predicting Hospital Surgical Mortality May 7, 2008.

Slides:



Advertisements
Similar presentations
How would you explain the smoking paradox. Smokers fair better after an infarction in hospital than non-smokers. This apparently disagrees with the view.
Advertisements

Statistical Analysis and Data Interpretation What is significant for the athlete, the statistician and team doctor? important Will Hopkins
Logistic Regression.
Departments of Medicine and Biostatistics
Background: Self-rated health (SRH) is widely used in research on health inequalities by socioeconomic status. However, researchers must be certain that.
Inappropriate clopidogrel adherence explains stent related adverse outcomes Leonardo Tamariz, MD, MPH University of Miami.
Gender Differences in Hospital Survival Rates For Medicare Beneficiaries Undergoing Coronary Artery Bypass Graft Surgery: Does Hospital Performance Ranking.
EVIDENCE BASED MEDICINE
CORRELATIO NAL RESEARCH METHOD. The researcher wanted to determine if there is a significant relationship between the nursing personnel characteristics.
Public Employees Benefits Board April 23, PEBB Procurement Bid Alternatives.
Multiple Choice Questions for discussion
 Ascertainment of cardiovascular (CV) outcomes via physician review of medical records using a standardized case definition aims to maximize data quality,
Ultra long term outcomes in adult survivors of tetralogy of Fallot and the effect of pulmonary valve replacement Dobson R1,2, Danton M2, Walker N2, Tzemos,
Decision Analysis. Real Case 63 year old housewife with 6 grown children. 10 year history of stable angina pectoris. 8 years ago she had total hip replacement.
An Overview of NCQA’s Relative Resource Use Measures.
The effect of surgeon volume on procedure selection in non-small cell lung cancer surgeries Dr. Christian Finley MD MPH FRCSC McMaster University.
Implementing the Leapfrog Standard for β-Blocker Use during AAA Repair in California Hospitals: Translation of Evidence-Based Process Measures to Improve.
The Standardized Infection Ratio Linda R Greene, RN, MPS,CIC Rochester General Health System Rochester, NY
High expenditure diseases and interventions Variation of inpatient expenditure rates.
A Novel Score to Estimate the Risk of Pneumonia After Cardiac Surgery
Healthcare Workforce and Regionalization of Services: Lung Cancer Resections Stephen C. Yang, M.D. Chief of Thoracic Surgery The Arthur B. and Patricia.
Andreas A. Rostved, MD Research assistant Department of Surgical Gastroenterology and Transplantation Rigshospitalet – Copenhagen University Hospital Denmark.
Teaching Intensity, Race and Surgical Outcomes Jeffrey H. Silber The University of Pennsylvania The Children’s Hospital of Philadelphia.
Is Veteran User Status an Independent Risk Factor for Mortality After Private Sector CABG? William B. Weeks, MD, MBA Dorothy A. Bazos, Ph.D. David M. Bott,
Inpatient Quality Reporting In Colorado Sept HCUP User Group Meeting.
EVEREST II Study Design Multicenter Randomized in a 2:1 ratio to either percutaneous or conventional surgery for the repair or replacement of the mitral.
Impact of Perceived Discrimination on Use of Preventive Health Services Amal Trivedi, M.D., M.P.H. John Z. Ayanian, M.D., M.P.P. Harvard Medical School/Brigham.
Coronary Artery Surgery 2003 Coronary Artery Surgery Results 2003 Coronary Artery Surgery Results 2003 The Japanese Association for Coronary Artery Surgery.
Peripheral Artery Disease in Orthopaedic Patients with Asymptomatic Popliteal Artery Calcification on Plain X-ray Adam Podet, MS; Julia Volaufova, phD,;
Date of download: 5/30/2016 Copyright © 2016 American Medical Association. All rights reserved. From: Elective Intra-aortic Balloon Counterpulsation During.
Date of download: 6/1/2016 Copyright © The American College of Cardiology. All rights reserved. From: Early and long-term clinical outcomes associated.
Ten Year Outcome of Coronary Artery Bypass Graft Surgery Versus Medical Therapy in Patients with Ischemic Cardiomyopathy Results of the Surgical Treatment.
Date of download: 6/25/2016 Copyright © The American College of Cardiology. All rights reserved. From: Medical Therapy With Versus Without Revascularization.
Date of download: 6/27/2016 Copyright © The American College of Cardiology. All rights reserved. From: Percutaneous Coronary Intervention Complications.
Do JCAHO accredited hospitals perform better on quality measures? An analysis of process-of-care measures and surgical indicators. William B Weeks, MD,
Single IMA {Single Arterial}
When Using DOPPS Slides
Ruyun Jin, MD, Gary L. Grunkemeier, PhD, Albert Starr, MD 
Successful CTO PCI Associated with Lower Mortality Risk
Is Hospital Procedure Volume a Reliable Marker of Quality for Coronary Artery Bypass Surgery? A Comparison of Risk and Propensity Adjusted Operative and.
Fig. 3 Summary estimates of the relative risks for the different surgical techniques for late mortality. Comparing the two main techniques, EVR showed.
Transient postoperative atrial fibrillation after abdominal aortic aneurysm repair increases mortality risk  Anai N. Kothari, MD, Pegge M. Halandras,
The prevalence of abdominal aortic aneurysm is consistently high among patients with coronary artery disease  Jussi A. Hernesniemi, MD, PhD, Ville Vänni,
Cyrus J. Parsa, MD, Linda K. Shaw, MS, J. Scott Rankin, MD, Mani A
Mean eGFR among survivors after CABG surgery vs all-cause deaths
Robert A. Meguid, MD, MPH, Benjamin S. Brooke, MD, Bruce A
Predicting In-Hospital Mortality After Redo Cardiac Operations: Development of a Preoperative Scorecard  Sebastian Launcelott, BA, BComm, Maral Ouzounian,
Cumulative sum techniques for assessing surgical results
Factors influencing early and late outcome of the arterial switch operation for transposition of the great arteries  Gil Wernovsky, MD* (by invitation),
Shikhar Agarwal, MD, MPH, Aatish Garg, MD, Akhil Parashar, MD, Lars G
The prevalence of abdominal aortic aneurysm is consistently high among patients with coronary artery disease  Jussi A. Hernesniemi, MD, PhD, Ville Vänni,
Ruyun Jin, MD, Gary L. Grunkemeier, PhD, Albert Starr, MD 
The Vascular Study Group of New England Cardiac Risk Index (VSG-CRI) predicts cardiac complications more accurately than the Revised Cardiac Risk Index.
Prediction of in-hospital mortality after ruptured abdominal aortic aneurysm repair using an artificial neural network  Eric S. Wise, MD, Kyle M. Hocking,
Raghavendra Paknikar, BS, Jeffrey Friedman, BS, David Cron, BS, G
Risk Stratification and Interventional Cardiology: Robert L
The impact of endovascular aneurysm repair on mortality for elective abdominal aortic aneurysm repair in England and the United States  Alan Karthikesalingam,
Global patient outcomes after elective surgery: prospective cohort study in 27 low-, middle- and high-income countries    British Journal of Anaesthesia 
Nader N. Massarweh, MD, MPH, David R. Flum, MD, MPH, FACS, Rebecca G
A decade after the Surgical Treatment for Ischemic Heart Failure (STICH) trial: Weaving firm clinical recommendations from lessons learned  Robert E.
Terry Shih, MD, Justin B. Dimick, MD, MPH 
National trends in utilization, mortality, and survival after repair of type B aortic dissection in the Medicare population  Douglas W. Jones, MD, Philip.
Aortic Valve Replacement for Patients With Severe Aortic Stenosis: Risk Factors and Their Impact on 30-Month Mortality  Edward L. Hannan, PhD, Zaza Samadashvili,
Logistic Regression.
ST-segment elevation myocardial infarction in China from 2001 to 2011 (the China PEACE-Retrospective Acute Myocardial Infarction Study): a retrospective.
A personal experience with factors influencing survival after elective open repair of infrarenal aortic aneurysms  Norman R. Hertzer, MD, Edward J. Mascha,
Trends in Patient Characteristics and Outcomes of Coronary Artery Bypass Grafting in the 2000 to 2012 Medicare Population  Christian McNeely, MD, Stephen.
Effect of Coronary Artery Disease Extent on Contemporary Outcomes of Combined Aortic Valve Replacement and Coronary Artery Bypass Graft Surgery  Zhongmin.
Influence of Hospital Volume on Outcomes of Thoracic Endovascular Repair in Vascular Quality Initiative Database: 5-year National Study. Presenting Author:
Gary L Grunkemeier, PhD, Ruyun Jin, MD  The Annals of Thoracic Surgery 
Presentation transcript:

Leapfrog’s “Survival Predictor”: Composite Measures for Predicting Hospital Surgical Mortality May 7, 2008

2 Townhall Call Overview Introductions Need for better measures What is the “Survival Predictor”? Methods Reporting to Leapfrog & Scoring Details Resources Q & A

3 Need for a better measures Individual quality measures have limitations –Mortality rates are often too “noisy” –Hospital volume is a weak proxy for performance Growing interest in composite measures but methods are not well established –Not clear on how to best weight measures when creating composite or interpret them when they conflict

4 What is the “Survival Predictor”? A simple composite score based on a combination of surgical mortality and hospital volume Designed to optimize the prediction of future risk- adjusted mortality based on these two inputs Applicable to six of Leapfrog’s EBHR high-risk surgeries (bariatric surgery excluded)

5 Validation Study: Data Sources & Study Population Used data from national Medicare population (MEDPAR files) Identified all patients aged 65 to 99 undergoing six operations –Coronary artery bypass –Percutaneous coronary interventions –Abdominal aortic aneurysm repair –Aortic valve replacement –Pancreatic cancer resection –Esophageal cancer resection

6 Development of Measure Used empirical Bayes approach to combine mortality rates with hospital volume Observed mortality rate is weighted according to how reliably it is estimated, with the remaining weight placed on the information regarding hospital volume Composite mortality prediction = (weight)*(observed mortality) + (1-weight)*(volume-predicted mortality)

7 Weight = Reliability (precision) or the mortality measure More weight placed on observed mortality rate when a hospital has higher number of cases (estimated with more reliability) Less weight on observed mortality rate for lower number of cases (lower reliability) Development of Measure

8 Validation of Composite Measure Determine value of measure: Step 1: Explaining variation in risk-adjusted mortality Technical implications: “Is it a better measure?” Step 2: Prediction of future performance Practical implications: “How much can I improve my chances by choosing this hospital?”

9 Table 1. Hospital caseload and the weight applied to each of the two inputs in the composite measure. The weight applied to the mortality rate is the reliability and the weight applied to hospital volume is 1-reliability. Hospital caseloads, mean (SD) Weight applied to observed mortality, mean (SD) Weight applied to hospital volume, mean (SD) Coronary artery bypass grafting 321 (319).46 (.21).54 (.21) Aortic valve replacement 59 (72).26 (.17).73 (.17) Percutaneous coronary interventions 442 (499).48 (.27).52 (.27) Elective abdominal aortic aneurysm repair 31 (46).21 (.18).79 (.18) Pancreatic cancer resection 4 (10).23 (.17).77 (.17) Esophageal cancer resection 4 (8).14 (.13).86 (.13)

10 Table 2. Relative ability of each measure to explain hospital-level differences in risk-adjusted mortality rates Proportion of hospital-level variation in mortality rates explained by each performance measure (%) Hospital volumeObserved mortalitySimple composite measure Coronary artery bypass grafting Aortic valve replacement Percutaneous coronary interventions Elective abdominal aortic aneurysm repair Pancreatic cancer resection Esophageal cancer resection

11 Table 3. Relative ability of historical measures ( ) to predict subsequent risk-adjusted mortality ( ). Adjusted odds ratio for risk-adjusted mortality ( ), best vs. worst quartile (95% CI) Hospital volume alone ( ) Observed mortality alone ( ) Simple composite measure ( ) Coronary artery bypass grafting 1.15( )1.71 ( )1.72 ( ) Aortic valve replacement 1.48 ( )1.73 ( )1.97 ( ) Percutaneous coronary interventions 1.49 ( )1.84 ( )1.95 ( ) Abdominal aortic aneurysm repair 1.48 ( )1.20 ( )1.69 ( ) Pancreatic cancer resection 3.93 ( )1.36 ( )4.23 ( ) Esophageal cancer resection 2.89 (2.15,3.89).90 (.72,1.12)2.71 ( )

12 Future risk-adjusted mortality rates ( ) for quartiles of hospital rankings based on historical ( ) hospital volume, risk-adjusted mortality rates, and composite measures. Future risk-adjusted mortality is shown for the composite measure created using both risk-adjusted and unadjusted mortality rates.

13 Risk-Adjustment Evaluated the extent to which risk-adjustment is important in improving the predictive ability of the composite measure Risk-adjustment performed using a logistic regression to estimate expected mortality rates (age, gender, race, urgency of operation, income, & coexisting diseases) Compared the ability of risk-adjusted and unadjusted composite measures to predict subsequent performance

14 Conclusions: Simple composite measures are better at explaining variations in mortality and predicting future performance Empirical Bayes approach is a valid and useful method for combining multiple domains of quality Composite measures based on unadjusted and risk- adjusted mortality rates were equally good at predicting future risk-adjusted mortality rates

15 Future Steps Incorporate other inputs into composite measure –e.g., outcomes with other related procedures Perform validation study for bariatric surgery

16 Reporting to Leapfrog For each of the six EBHR surgeries, hospitals asked to report on hospital volume and number of patient deaths following procedure No additional burden as compared to 2007 Leapfrog Hospital Survey

17 Scoring Details A hospital’s result on the composite measure for each procedure is publicly released and displayed on Leapfrog’s website in one of five categories: Best Chance of Survival (full circle) means the hospital is in the best quartile for the composite measure for that procedure. Good Chance of Survival (3/4-circle) means the hospital is above the midpoint (median), but not in the best quartile for the composite measure for that procedure. Fair Chance of Survival (1/2-circle) means the hospital is below the midpoint (median), but not in the worst quartile for the composite measure for that procedure. Worst Chance of Survival (1/4-circle) means the hospital is in the worst quartile for the composite measure for that procedure. Did not disclose this information (empty circle) means the hospital did not respond to this section of the survey, or the hospital was asked to complete the survey but has not submitted one. N/A -- Standard does not apply means the hospital does not perform the procedure electively. Scoring based on rank relative to all U.S. hospitals Results will be posted on Leapfrog’s website in early Fall

18 Resources White paper available on Leapfrog’s website (under “News & Events”) Recording of today’s townhall call is available for one month after today’s call

19 Questions?