Presentation is loading. Please wait.

Presentation is loading. Please wait.

Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores  Jesse M. Pines, MD,

Similar presentations


Presentation on theme: "Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores  Jesse M. Pines, MD,"— Presentation transcript:

1 Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores  Jesse M. Pines, MD, MBA, Pooja Penninti, Sukayna Alfaraj, MBBS, Jestin N. Carlson, MD, MS, Orion Colfer, MD, Christopher K. Corbit, MD, Arvind Venkat, MD  Annals of Emergency Medicine  Volume 71, Issue 5, Pages e6 (May 2018) DOI: /j.annemergmed Copyright © 2017 American College of Emergency Physicians Terms and Conditions

2 Figure 1 Changes in facility and physician Press Ganey percentile rank during the study period. A, Variability in facility Press Ganey percentile rank. B, Variability in physician Press Ganey percentile rank. Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

3 Figure 2 Effect of significant facility independent variables on monthly facility Press Ganey percentile rank. The figure shows the predicted effect of each independent variable on monthly facility Press Ganey percentile rank, along with 95% confidence intervals, holding all other independent variables in the multivariable regression model constant. ∗A one hour increase in mean patient length of stay in the ED of patients who are discharged from the facility, holding all other factors constant, would be associated with the average monthly Press Ganey percentile rank for the facility decreasing between 3.1 to 8.9 points (95% confidence interval). ∗∗A one point increase in the percentage of patients discharged in a month at an ED, holding all other factors constant, would be associated with the average monthly facility Press Ganey percentile rank increasing between 0.5 to 1.0 points (95% confidence interval). Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

4 Figure 3 Effect of significant physician independent variables on monthly physician Press Ganey percentile rank. The figure shows the predicted effect of each independent variable on monthly physician Press Ganey percentile rank, along with 95% confidence intervals, holding all other independent variables in the multivariable regression model constant. ∗A one hour increase in the mean patient length of stay of patients assigned to a physician who are discharged from the ED, holding all other factors constant, would be associated with the average monthly Press Ganey percentile rank for the physician decreasing between 4.4 to 7.4 points (95% confidence interval). ∗∗A one point increase in the mean RVUs/hour generated by the physician, holding all other factors constant, would be associated with the average monthly physician Press Ganey percentile rank increasing between 3.4 and 10.3 points (95% confidence interval). Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

5 Figure 4 A, Plot of mean monthly facility Press Ganey percentile rank versus SD of mean monthly facility Press Ganey percentile rank. The 42 facilities (each represented by a single data point) in the chart show that those with low mean score (near 0%) or high mean score (near 100%) have lower Press Ganey variability. Facilities with percentile rank near 50% have higher volatility in their ranking, in which small changes in mean score could make a large difference in the percentile rank. B, Plot of mean monthly physician Press Ganey percentile rank versus SD of mean monthly physician Press Ganey percentile rank. The nature and structure of the chart for 242 physicians (each represented by a single data point) are similar to those of the facility chart, showing physicians with low mean score (near 0%) or high mean score (near 100%) having lower Press Ganey variability. Physicians with percentile rank near 50% have much higher volatility in their ranking, in which small changes in mean ratings could make a large difference in the percentile rank. Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

6 Figure E1 Facility selection flow chart. There was an average of 45 months of data per facility out of 48 months in the data set. The order of applied data filters will not change the final total count because these conditions are applied as an “and” condition. Actual influence of each variable on “exclusion” count is provided in Table E1. Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions

7 Figure E2 Physician selection flow chart. There was an average of 43 months of data per physician out of 48 months in the data set. The order of applied data filters will not change the final total count because these conditions are applied as an “and” condition. Actual influence of each variable on “exclusion” count is provided in Table E1. Annals of Emergency Medicine  , e6DOI: ( /j.annemergmed ) Copyright © 2017 American College of Emergency Physicians Terms and Conditions


Download ppt "Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores  Jesse M. Pines, MD,"

Similar presentations


Ads by Google