Risk adjustment using administrative and clinical data: model comparison Paola Colais Workshop “Challenges for epidemiology in the context of the National Health Service” Rome, October 15th – 16th, 2012

Background The assessment of hospital care quality has become increasingly important in many European countries, and worldwide, in response to requests for greater transparency and accountability and for quality improvement. However, observational studies comparing groups or populations to evaluate services or interventions strongly require severity and comorbidity adjustment to account for differences between the groups being compared. Little is known about the relative performance of available information systems in predicting outcomes and control for confounding.

Objective To compare the performance of diagnosis, drug prescription and “RAD-esito”-based models in predicting outcomes and control for confounding in hospital quality of care comparative analysis.

Data sources Data sources are: Hospital Information System pharmacy dispensing database RAD-esito (AMI, CABG, Hip Fracture)

Study population Hospital discharges in Lazio region between January 2010 and November 2010 with a diagnosis of Acute Myocardial Infarction (AMI) Hip Fracture

Outcomes Thirty-days mortality from hospital admission was evaluated for AMI patients Proportion of interventions performed within 48 hours from admission was evaluated for hip fracture patients

Methods (1) A multivariate regression analysis was used to calculate adjusted hospital-specific risks using diagnosis, drug prescription and RAD-esito-based predictive models, both separately and jointly. Factors associated with the outcomes were selected by a bootstrap stepwise procedure to assign an importance rank for predictors in logistic regression

Methods (2) Performances were measured by using the ‘c’ statistic, ranging from 0.5 for chance prediction of outcome to 1.0 for perfect prediction. change-in-estimate methods were applied to improve parsimony and gain estimates’ precision, by eliminating variables that are not actual confounders.

Results Hip Fracture (1) INR International Normalized Ratio valuta la capacità del sangue di coagulare c statistic: 0.555 c statistic: 0.573 c statistic: 0.576

Results Hip Fracture (2) Adjusted proportion of interventions performed within 48 hours by hospitals

Results Hip Fracture (3) Adjusted proportion of interventions performed within 48 hours by hospitals

Results Hip Fracture (4) Adjusted proportion of interventions performed within 48 hours by hospitals

Results AMI (1) c statistic: 0.761 c statistic: 0.793

Results AMI (2) Adjusted 30-days mortality after AMI by hospitals

Results AMI (3) Adjusted 30-days mortality after AMI by hospitals

Results AMI (4) Adjusted 30-days mortality after AMI by hospitals

Results Change-in-estimate methods Hospital Information System Hip Fracture No actual confounders AMI Age Blood pressure Diuretics Hospital Information System “RAD-esito” system drug prescription

Critical aspects The use of routinely collected administrative data in comparative outcome evaluations presents some limitations: different coding practices across hospitals and misclassification of comorbidity; absence of some clinical information needed to adjust for patients’ conditions; some chronic comorbidities, such as hypertension and diabetes, are known to be currently under-reported at admission, mainly in more severely affected patients. Hypertension, anemia, not controlled diabetes

Conclusions Hip fracture no differences were found using risk adjustment models with different information systems because there are no factors that are actual confounders. AMI small differences. Some actual confounders from the three different information systems were found.