Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores Jesse M. Pines, MD,

Slides:

Advertisements

Similar presentations

Young Woman With Abdominal Pain David Jones, MD Annals of Emergency Medicine Volume 64, Issue 4, (October 2014) DOI: /j.annemergmed

Advertisements

An Early Look at Performance on the Emergency Care Measures Included in Medicare's Hospital Inpatient Value-Based Purchasing Program Megan McHugh, PhD,

Increased Emergency Department Use in Illinois After Implementation of the Patient Protection and Affordable Care Act Scott M. Dresden, MD, MS, Emilie.

Colorado Cannabis Legalization and Its Effect on Emergency Care

The Impact of Screening, Brief Intervention, and Referral for Treatment on Emergency Department Patients' Alcohol Use Annals of Emergency Medicine

Matters of Concern: A Qualitative Study of Emergency Care From the Perspective of Patients Gert Olthuis, PhD, Carolien Prins, MSc, Marie-Josée Smits,

Can Vasopressors Safely Be Administered Through Peripheral Intravenous Catheters Compared With Central Venous Catheters? J. Michael Brewer, MD Annals.

Emergency Physician Ultrasonography for Evaluating Patients at Risk for Ectopic Pregnancy: A Meta-Analysis John C. Stein, MD, Ralph Wang, MD, Naomi Adler,

Annals of Emergency Medicine

The Presence of Outcome Bias in Emergency Physician Retrospective Judgments of the Quality of Care Malkeet Gupta, MD, MS, David L. Schriger, MD, MPH,

Many Emergency Department Patients With Severe Sepsis and Septic Shock Do Not Meet Diagnostic Criteria Within 3 Hours of Arrival Julian Villar, MD, MPH,

Vulvar Lesions Annals of Emergency Medicine

Pacemaker Extrusion Annals of Emergency Medicine

No SIRS; Quick SOFA Instead

Alan E. Jones, MD, Michael A. Puskarich, MD

Emergency Department and Inpatient Hospital Use by Medicare Beneficiaries in Patient- Centered Medical Homes Jesse M. Pines, MD, MBA, Vincent Keyes, MA,

Isopropyl Alcohol Nasal Inhalation for Nausea in the Emergency Department: A Randomized Controlled Trial Kenneth Lee Beadle, DSc, MPAS, Antonia R. Helbling,

Alternatives to the Conference Status Quo: Addressing the Learning Needs of Emergency Medicine Residents Michael Gottlieb, MD, Jeff Riddell, MD, Sara.

Ketamine as Rescue Treatment for Difficult-to-Sedate Severe Acute Behavioral Disturbance in the Emergency Department Geoffrey Kennedy Isbister, MD, FACEM,

Discharge Glucose Is Not Associated With Short-Term Adverse Outcomes in Emergency Department Patients With Moderate to Severe Hyperglycemia Brian E.

Annals of Emergency Medicine

Clinical Policy: Use of Intravenous tPA for the Management of Acute Ischemic Stroke in the Emergency Department Annals of Emergency Medicine Volume.

Evidence Behind the 4-Hour Rule for Initiation of Antibiotic Therapy in Community- Acquired Pneumonia Kenneth T. Yu, MD, MBA, Peter C. Wyer, MD Annals.

Procedural Sedation: Is Two Better Than One?

Maria C. Raven, MD, MPH, Margot Kushel, MD, Michelle J

Managing Nontraumatic Acute Back Pain

Rachel A. Lindor, JD, Ronna L. Campbell, MD, PhD, Jesse M

A Population-Based Analysis of Outcomes in Patients With a Primary Diagnosis of Hypertension in the Emergency Department Sameer Masood, MD, Peter C.

A Woman With a Click in Her Throat

Changing Dispensers May Prevent Intoxication From Isopropanol and Ethyl Alcohol- Based Hand Sanitizers Scott G. Weiner, MD, MPH Annals of Emergency Medicine

Observers in the Medical Setting

Steven M. Green, MD, Mark G. Roback, MD, Robert M

Michael A. LaMantia, MD, MPH, Frank C. Messina, MD, Cherri D

Middle-Aged Man With Rash

The Role of Antibiotic Prophylaxis for Prevention of Infection in Patients With Simple Hand Lacerations Shahriar Zehtabchi, MD Annals of Emergency Medicine

An Infant With Fever and Rash

Sangeeta Lamba, MD, Tammie E. Quest, MD Annals of Emergency Medicine

An Elderly Woman with Arm Twitching

Levocetirizine and Prednisone Are Not Superior to Levocetirizine Alone for the Treatment of Acute Urticaria: A Randomized Double-Blind Clinical Trial

Clinical Features From the History and Physical Examination That Predict the Presence or Absence of Pulmonary Embolism in Symptomatic Emergency Department.

Nupur Garg, MD, Michael Gottlieb, MD Annals of Emergency Medicine

Teenager With Scalp Mass

Measuring and Explaining Computed Tomography Use in the United States and Canada: A Consideration of Health Economics and Use Versus Appropriateness,

Longitudinal Trends in the Performance of Scientific Peer Reviewers

Emergency Department Planning and Resource Guidelines

The Effect of Pay for Performance in the Emergency Department on Patient Waiting Times and Quality of Care in Ontario, Canada: A Difference-in-Differences.

Eric Berger Annals of Emergency Medicine

A Refractory Wart? Annals of Emergency Medicine

Woman With Hip Pain Annals of Emergency Medicine

Steven M. Green, MD, Michael L. Callaham, MD

Analysis of Emergency Department Length of Stay for Mental Health Patients at Ten Massachusetts Emergency Departments Mark D. Pearlmutter, MD, Kristin.

Ultrasound-Guided Peripheral Intravenous Access Program Is Associated With a Marked Reduction in Central Venous Catheter Use in Noncritically Ill Emergency.

CHA2DS2-VASc Scores and Major Bleeding in Patients With Nonvalvular Atrial Fibrillation Who Are Receiving Rivaroxaban W. Frank Peacock, MD, CAPT Sally.

Lack of Association Between Press Ganey Emergency Department Patient Satisfaction Scores and Emergency Department Administration of Analgesic Medications

Incidence and Duration of Continuously Measured Oxygen Desaturation During Emergency Department Intubation Jerry B. Bodily, MD, Heather R. Webb, MD,

Chad Kessler, MD, MHPE, Nathaniel L

Kuo-Chen Cheng, MD Annals of Emergency Medicine

Images in Emergency Medicine

Images in Emergency Medicine

Penetrating Neck Injury: What’s In and What’s Out?

Images in Emergency Medicine

Continuing Care for Patients Choosing the Emergency Department as a Site for Primary Care: Feasibility, Benefits, and EMTALA Considerations David L.

Lean Thinking in Emergency Departments: A Critical Review

Vulvar Lesions Annals of Emergency Medicine

Young Boy With Shedding Nails

Global Emergency Medicine Journal Club: A Social Media Discussion About the Lack of Association Between Press Ganey Scores and Emergency Department Analgesia

Bridging Orders and a Dedicated Admission Nurse Decreases Emergency Department Turnaround Times While Increasing Patient Satisfaction John Patterson,

Woman With Facial Trauma

D. Stevenson, T. Long, J.P. Green, J. Rose

Young Man with Foreign-Body Sensation in the Right Eye

Presentation transcript:

Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores Jesse M. Pines, MD, MBA, Pooja Penninti, Sukayna Alfaraj, MBBS, Jestin N. Carlson, MD, MS, Orion Colfer, MD, Christopher K. Corbit, MD, Arvind Venkat, MD Annals of Emergency Medicine Volume 71, Issue 5, Pages 545-554.e6 (May 2018) DOI: 10.1016/j.annemergmed.2017.11.011 Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure 1 Changes in facility and physician Press Ganey percentile rank during the study period. A, Variability in facility Press Ganey percentile rank. B, Variability in physician Press Ganey percentile rank. Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure 2 Effect of significant facility independent variables on monthly facility Press Ganey percentile rank. The figure shows the predicted effect of each independent variable on monthly facility Press Ganey percentile rank, along with 95% confidence intervals, holding all other independent variables in the multivariable regression model constant. ∗A one hour increase in mean patient length of stay in the ED of patients who are discharged from the facility, holding all other factors constant, would be associated with the average monthly Press Ganey percentile rank for the facility decreasing between 3.1 to 8.9 points (95% confidence interval). ∗∗A one point increase in the percentage of patients discharged in a month at an ED, holding all other factors constant, would be associated with the average monthly facility Press Ganey percentile rank increasing between 0.5 to 1.0 points (95% confidence interval). Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure 3 Effect of significant physician independent variables on monthly physician Press Ganey percentile rank. The figure shows the predicted effect of each independent variable on monthly physician Press Ganey percentile rank, along with 95% confidence intervals, holding all other independent variables in the multivariable regression model constant. ∗A one hour increase in the mean patient length of stay of patients assigned to a physician who are discharged from the ED, holding all other factors constant, would be associated with the average monthly Press Ganey percentile rank for the physician decreasing between 4.4 to 7.4 points (95% confidence interval). ∗∗A one point increase in the mean RVUs/hour generated by the physician, holding all other factors constant, would be associated with the average monthly physician Press Ganey percentile rank increasing between 3.4 and 10.3 points (95% confidence interval). Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure 4 A, Plot of mean monthly facility Press Ganey percentile rank versus SD of mean monthly facility Press Ganey percentile rank. The 42 facilities (each represented by a single data point) in the chart show that those with low mean score (near 0%) or high mean score (near 100%) have lower Press Ganey variability. Facilities with percentile rank near 50% have higher volatility in their ranking, in which small changes in mean score could make a large difference in the percentile rank. B, Plot of mean monthly physician Press Ganey percentile rank versus SD of mean monthly physician Press Ganey percentile rank. The nature and structure of the chart for 242 physicians (each represented by a single data point) are similar to those of the facility chart, showing physicians with low mean score (near 0%) or high mean score (near 100%) having lower Press Ganey variability. Physicians with percentile rank near 50% have much higher volatility in their ranking, in which small changes in mean ratings could make a large difference in the percentile rank. Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure E1 Facility selection flow chart. There was an average of 45 months of data per facility out of 48 months in the data set. The order of applied data filters will not change the final total count because these conditions are applied as an “and” condition. Actual influence of each variable on “exclusion” count is provided in Table E1. Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

Figure E2 Physician selection flow chart. There was an average of 43 months of data per physician out of 48 months in the data set. The order of applied data filters will not change the final total count because these conditions are applied as an “and” condition. Actual influence of each variable on “exclusion” count is provided in Table E1. Annals of Emergency Medicine 2018 71, 545-554.e6DOI: (10.1016/j.annemergmed.2017.11.011) Copyright © 2017 American College of Emergency Physicians Terms and Conditions