1. Database Use & Abuse Shoo K. Lee, MBBS, FRCPC, PhD Director, Canadian Neonatal Network Director, Centre for Healthcare Innovation & Improvement University of British Columbia

2. Centre for Healthcare Innovation and Improvement Growth of Vermont-Oxford Network

3. Centre for Healthcare Innovation and Improvement Neonatal Networks Australia-New Zealand Neonatal Network Canadian Neonatal Network European Neonatal Network International Neonatal Network Israel Neonatal Network South American Neonatal Network Vermont-Oxford Neonatal Network

4. Centre for Healthcare Innovation and Improvement Victoria Vancouver New Westminster Edmonton Calgary Saskatoon Regina Winnipeg Montreal Ottawa Kingston Toronto London Hamilton Halifax St John’s Canadian Neonatal Network Quebec City Sherbrooke Moncton Frederickton St John

5. Centre for Healthcare Innovation and Improvement Canadian Perinatal-Neonatal Research Networks

6. Centre for Healthcare Innovation and Improvement

7. Relationship between Networks Canadian Neonatal Network (CNN) Canadian Perinatal Network (CPN) NICE-Team Integrated Database System Data Project NICE-Team is CIHR-funded to provide: - Database support and management - Network coordination - Data analysis services - Training awards - Resource of experienced multi- disciplinary researchers who can assist investigators

8. Centre for Healthcare Innovation and Improvement Why join Databases and Networks? Audit – outcomes and resource use Research – clinical trials, health services, population health, translational research Quality improvement Professional guidelines Education and Training Policy and resource allocation decisions Advocacy International collaborations

9. Centre for Healthcare Innovation and Improvement Database Use – A Looking Glass

10. Centre for Healthcare Innovation and Improvement Database Abuse – Distortions and Illusions

11. Centre for Healthcare Innovation and Improvement Descriptive Data - Uses Tells the Simple Facts Reader does all the interpretation

12. Centre for Healthcare Innovation and Improvement Gestational age distribution

13. Centre for Healthcare Innovation and Improvement Birth weight distribution

14. Centre for Healthcare Innovation and Improvement Descriptive Data - Abuses Are the Data complete, accurate and unbiased? Reader may interpret incorrectly Often does not provide answers to address specific policy questions

15. Centre for Healthcare Innovation and Improvement Garbage In - Garbage Out

16. Centre for Healthcare Innovation and Improvement Crude Outcome Incidences - Uses Answers specific questions Permit longitudinal and trend analysis Surveillance and monitoring tool Early warning - emerging events and trends

17. Centre for Healthcare Innovation and Improvement Survival to discharge home

18. Centre for Healthcare Innovation and Improvement IVH incidence by GA

19. Centre for Healthcare Innovation and Improvement Crude Outcome Incidences - Abuses Similar to descriptive data

20. Centre for Healthcare Innovation and Improvement Outcome Comparisons - Uses Increased level of complexity Benchmark against industry standards Sentinel for monitoring patient safety Potential for improving quality and efficiency Provides management tool – measure performance, use carrot and stick Competitive advantage

21. Centre for Healthcare Innovation and Improvement Mortality rates among Canadian NICUs

22. Centre for Healthcare Innovation and Improvement Comparison of Outcomes - Abuses Lack of appropriate risk-adjustment Why do comparisons at all? - wrong interpretation - inappropriate change in practice patterns - competitive advantage/disadvantage - “gaming” system Fear-mongering Potential for inappropriate shifts in patterns of patient use

23. Centre for Healthcare Innovation and Improvement Benchmarking & Risk Adjustment Promise of overcoming problems associated with comparison of crude outcomes Permit population based data analysis and interpretation, and design of system to provide optimal quality and efficiency of care

24. Centre for Healthcare Innovation and Improvement Risk Adjustment Epidemiologic baseline population risks Diagnostic groups Therapeutic intensity (NTISS) Physiologic illness severity (CRIB, SNAP) True measure of illness severity at admission (TRIPS?) Where is the state of the art/science?

25. Centre for Healthcare Innovation and Improvement SNAP-II and Mortality

26. Centre for Healthcare Innovation and Improvement TRIPS (higher score is worse) TRIPS VariableTRIPS Score Points Temperature ( o C) 37.6 36.1 – 36.5 or 37.2-37.6 36.6 – 37.1 810810 Respiratory status Severe (apnoea, gasping, intubated) Moderate (RR >60/min or SpO2 <85) None (RR 85) 14 5 0 Systolic BP (mm Hg) <20 20-40 >40 26 16 0 Response to noxious stimuli None, seizure, muscle relaxant Lethargic response, no cry Withdraws vigorously, cries 17 6 0

27. Centre for Healthcare Innovation and Improvement Mortality Change associated with TRIPS Change

28. Centre for Healthcare Innovation and Improvement Which score to use? ROCGACRIBSNAPTRIPS Mortality0.650.90*0.910.85/0.91 IVH0.72na0.80 BPDna0.770.85na Limits<1500g* 12 hour One use All babies 12 hour One use Out-born 1 minute Repeat

29. Centre for Healthcare Innovation and Improvement Risk-adjusted Outcome Comparisons - Uses “True” comparison of outcomes Separate outcome differences due to patient differences from those due to practice differences Permits study and design of practice change to improve outcomes New tool that may overcome some of the disadvantages of randomized clinical trials

30. Centre for Healthcare Innovation and Improvement NICU Mortality Comparisons

31. Centre for Healthcare Innovation and Improvement Risk adjusted outcome comparisons - abuses Definitions - do we mean the same thing? Measurement criteria – were they the same? Consistency - were criteria applied consistently? Was the treatment the same? Was risk-adjustment appropriate?

32. Centre for Healthcare Innovation and Improvement Incidence of BPD by GA

33. Centre for Healthcare Innovation and Improvement ROP Diagnosis and Treatment

34. Centre for Healthcare Innovation and Improvement

35. How do we use data for quality improvement?

36. Centre for Healthcare Innovation and Improvement Quality Improvement – Current paradigm

37. Centre for Healthcare Innovation and Improvement Is this good enough? Why do some units have better results? Are all their practices “best”? Are their practices applicable to you? Might you copy something that is harmful? Is this shot-gun approach efficient? At best, this is a subjective and unscientific approach Is there another way?

38. Centre for Healthcare Innovation and Improvement Factors affecting IVH variations Synnes et al - 4 NICU practices accounted for IVH variation: (a) Antenatal steroids (b) C-section vs Vaginal delivery (c) Treatment for hypotension (d) Treatment for acidosis Implication – change in clinical practice may reduce IVH rates in some hospitals MacNab et al – developed Bayesian statistical methods using Markov Chain Monte Carlo hierarchical modeling to identify risk factors specific to each hospital

39. Centre for Healthcare Innovation and Improvement Log odds plot for risk factor small for gestational age, two-level hierarchical model C, Canadian NICU data 1996-97.

40. Centre for Healthcare Innovation and Improvement Residual NICU effects, means and 95% CI limits, two-level hierarchical models, Canadian NICU data 1996-97.

41. Centre for Healthcare Innovation and Improvement Probability of being NI free

42. Centre for Healthcare Innovation and Improvement Anderson-Gill Model for NI recurrence

43. Centre for Healthcare Innovation and Improvement Adjusted probability for developing nosocomial infection for PICC lines

44. Centre for Healthcare Innovation and Improvement The EPIC paradigm

45. Centre for Healthcare Innovation and Improvement New Method for Quality Improvement Clinical TrialsCQI EQI

46. Centre for Healthcare Innovation and Improvement EPIC – Phase 1 Baseline data collection Training of Infection Teams – MD, RN, QI Review of published literature Meeting to share findings Identify Critical Pathways & Incidents Qualitative research – identify Failure Modes Data analysis – identify practice differences associated with outcome variation Develop Change Strategy

47. Centre for Healthcare Innovation and Improvement EPIC – Phase 2 Staff communication and training – group sessions, information packages Prepare supporting materials, e.g. prompts printed on order sheets Publicize information, posters, newsletters Implement EPIC 3-monthly feedback – Control Charts Revise strategies, reinforce change

48. Centre for Healthcare Innovation and Improvement The EPIC Network & Database Approach Health Authority

49. Centre for Healthcare Innovation and Improvement Integrative Processes

50. Centre for Healthcare Innovation and Improvement Comparing Outcomes

51. Population and Policy Implications

52. Centre for Healthcare Innovation and Improvement

56. Daily Level 3 Bed Utilization in B.C.

57. Centre for Healthcare Innovation and Improvement Mortality by hospitals

58. Centre for Healthcare Innovation and Improvement Mean SNAP-II

59. Manpower

60. Centre for Healthcare Innovation and Improvement Neonatologist/1000 births

61. Centre for Healthcare Innovation and Improvement Normative values

62. Centre for Healthcare Innovation and Improvement Watkins Criteria for Hypotension

63. Centre for Healthcare Innovation and Improvement Hypotension Criteria and Incidence

64. Evaluate Expert Guidelines

65. Centre for Healthcare Innovation and Improvement Cost-effectiveness ROP screening

66. Develop new clinical guidelines

67. Centre for Healthcare Innovation and Improvement Cost-effective IVH Screening

68. Assess guideline use

69. Centre for Healthcare Innovation and Improvement Antenatal Steroid Use 1996-97: incidence of use = 59% among infants 24-34 weeks gestation Variation in use: Inborn 25% – 96% Outborn8% - 94% Potential to decrease neonatal deaths by 10%

70. Planning and Policy

71. Centre for Healthcare Innovation and Improvement Choice of transport system

72. Regional variations in outcomes and resource allocation planning

73. Planning for future

74. Centre for Healthcare Innovation and Improvement 10 year Projections

75. Centre for Healthcare Innovation and Improvement Summary Databases can be useful But use data and interpretation with CARE Clinical input is vital Database should meet your needs and goals Societal versus institutional goals

76. Centre for Healthcare Innovation and Improvement Concerns Public impact of information Privacy and confidentiality Conformity versus innovation Barrier to clinical trials “Big brother” control Not “real” hypothesis-driven research Not high quality evidence (e.g. clinical trials) Research versus Quality Improvement Clinician versus “real” researchers

77. www.canadianneonatalnetwork.org