2. Risk adjustment using administrative and clinical data: model comparison
Paola Colais
Workshop
“Challenges for epidemiology in the context of the National Health Service”
Rome, October 15th – 16th, 2012

3. Background
The assessment of hospital care quality has become increasingly important in many European countries, and worldwide, in response to requests for greater transparency and accountability and for quality improvement.
However, observational studies comparing groups or populations to evaluate services or interventions strongly require severity and comorbidity adjustment to account for differences between the groups being compared.
Little is known about the relative performance of available information systems in predicting outcomes and control for confounding.

4. Objective
To compare the performance of diagnosis, drug prescription and “RAD-esito”-based models in predicting outcomes and control for confounding in hospital quality of care comparative analysis.

5. Data sources Data sources are: Hospital Information System
pharmacy dispensing database
RAD-esito (AMI, CABG, Hip Fracture)

6. Study population
Hospital discharges in Lazio region between January 2010 and November 2010 with a diagnosis of
Acute Myocardial Infarction (AMI)
Hip Fracture

7. Outcomes
Thirty-days mortality from hospital admission was evaluated for AMI patients
Proportion of interventions performed within 48 hours from admission was evaluated for hip fracture patients

8. Methods (1)
A multivariate regression analysis was used to calculate adjusted hospital-specific risks using diagnosis, drug prescription and RAD-esito-based predictive models, both separately and jointly.
Factors associated with the outcomes were selected by a bootstrap stepwise procedure to assign an importance rank for predictors in logistic regression

9. Methods (2)
Performances were measured by using the ‘c’ statistic, ranging from 0.5 for chance prediction of outcome to 1.0 for perfect prediction.
change-in-estimate methods were applied to improve parsimony and gain estimates’ precision, by eliminating variables that are not actual confounders.

10. Results Hip Fracture (1)
INR International Normalized Ratio valuta la capacità del sangue di coagulare
c statistic: 0.555
c statistic: 0.573
c statistic: 0.576

11. Results Hip Fracture (2)
Adjusted proportion of interventions performed within 48 hours by hospitals

12. Results Hip Fracture (3)
Adjusted proportion of interventions performed within 48 hours by hospitals

13. Results Hip Fracture (4)
Adjusted proportion of interventions performed within 48 hours by hospitals

14. Results AMI (1) c statistic: 0.761 c statistic: 0.793

15. Results AMI (2)
Adjusted 30-days mortality after AMI by hospitals

16. Results AMI (3)
Adjusted 30-days mortality after AMI by hospitals

17. Results AMI (4)
Adjusted 30-days mortality after AMI by hospitals

18. Results Change-in-estimate methods Hospital Information System
Hip Fracture
No actual confounders
AMI
Age
Blood pressure
Diuretics
Hospital Information System
“RAD-esito” system
drug prescription

19. Critical aspects
The use of routinely collected administrative data in comparative outcome evaluations presents some limitations:
different coding practices across hospitals and misclassification of comorbidity;
absence of some clinical information needed to adjust for patients’ conditions;
some chronic comorbidities, such as hypertension and diabetes, are known to be currently under-reported at admission, mainly in more severely affected patients.
Hypertension, anemia, not controlled diabetes

20. Conclusions
Hip fracture no differences were found using risk adjustment models with different information systems because there are no factors that are actual confounders.
AMI small differences. Some actual confounders from the three different information systems were found.