Risk adjustment and other reporting issues Shalini Santhakumaran NDAU Statistician Imperial College London

Why report outcomes ? Improvements in neonatal care and outcomes Audit Comparison CommissioningAccountability Transparency

Data Issues  Completeness  Accuracy

Data Issues  Completeness  Accuracy  Agreed case definitions

Data Issues  Completeness  Accuracy  Agreed case definitions  Neonatal transfers

Analysis issues  Unadjusted data is useful for some purposes  For others a more detailed analysis is required

Analysis issues  Unadjusted data is useful for some purposes  For others a more detailed analysis is required  3 reasons for variation in outcome:

Analysis issues  Unadjusted data is useful for some purposes  For others a more detailed analysis is required  3 reasons for variation in outcome: 1.Differences in case-mix 2.Random variation 3.Differences in care provided

Analysis issues 1.Differences in case-mix 2.Random variation 3.Differences in care provided  Selection of variables  Use of appropriate statistical models  Cannot completely control for case-mix

Survival Probability Calculator

Analysis issues 1.Differences in case-mix 2.Random variation 3.Differences in care provided  Occurs by chance even if the underlying mortality rate is the same for all providers  Illustrate significance using funnel plots

Funnel plot for adjusted SMR 95% confidence interval limits = “warning” 99.8% confidence interval limits = “alarm” ● = complete networks ○ = incomplete networks

Analysis issues 1.Differences in case-mix 2.Random variation 3.Differences in care provided  Not necessarily due to good/poor performance  ‘Constant risk fallacy’ and other local effects  Cannot tell us whether deaths were preventable  Needs to be linked to process measures

The National Neonatal Database  Individual, not aggregated  Population-based  Detailed clinical record  Electronic  Collaborative access to denominator data

The National Neonatal Database  Individual, not aggregated  Population-based  Detailed clinical record  Electronic  Collaborative access to denominator data …ideal for reporting outcomes

Acknowledgements All neonatal units contributing to the NDAU NDAU Team NDAU Steering Board Jane Abbott (BLISS)Jacquie Kemp Prof. Peter BrocklehurstProf. Azeem Majeed Prof. Kate CosteloeProf. Neena Modi Prof. Liz DraperProf. Andrew Wilkinson Imperial College London Academic Neonatal Medicine Unit