1. NAPHSIS Annual Meeting 2014Slide 1 NAPHSIS ANNUAL MEETING | Seattle | June 8-11, 2014 VITAL RECORDS: A CULTURE OF QUALITY Ohio’s hospital birth certificate data quality dashboard John Paulson, Judy Nagy, Ike Mgbatogu, Jingyu Hu, Srivalsan Karuvada Ohio Department of Health, Office of Vital Statistics

2. NAPHSIS Annual Meeting 2014Slide 2 Why we created the dashboard NCHS documents many Ohio problems Utility of some variables compromised by inadequate quality VS desired an automated tool to – report data quality issues to hospitals – point VS staff to most important areas for improvement

3. NAPHSIS Annual Meeting 2014Slide 3NAPHSIS Annual Meeting 2014Slide 3 Main contents of the Dashboard --tabular

4. NAPHSIS Annual Meeting 2014Slide 4NAPHSIS Annual Meeting 2014Slide 4 Dashboard data sourcing IPHIS (BC application) Analysis data (nightly copy) Dashboard Hospital corrections occur in the source

5. NAPHSIS Annual Meeting 2014Slide 5 Data quality indicator sources NCHS validation and verification errors NCHS unknowns Other data quality items of special interest Included a few easy ones with few errors No independent data were used to evaluate BC data (e.g., Medicaid, hospital discharge data)

6. NAPHSIS Annual Meeting 2014Slide 6 Statistics shown in reports Birth count Error count Error rate (%) Error share (% errors within Ohio, within peer group) Font color denotes rates above a threshhold

7. NAPHSIS Annual Meeting 2014Slide 7 Thresholds Indicator thresholds define problem cases and font color One cut-point per indicator (not peer group specific) In general we used the 75 th percentile in error rates among hospitals as the cut-point

8. NAPHSIS Annual Meeting 2014Slide 8 Hospital and VS user types Hospital users see ( for top 12 indicators ) – own hospital’s statistics – peer group statistics – statewide statistics – listing of certificates with errors VS users see – All of above (for all hospitals and peer groups) – 12 more indicators

9. NAPHSIS Annual Meeting 2014Slide 9NAPHSIS Annual Meeting 2014Slide 9 Ohio statewide report for April/May 2014 births

10. NAPHSIS Annual Meeting 2014Slide 10NAPHSIS Annual Meeting 2014Slide 10 Ohio statewide graph for April/May 2014 births

11. NAPHSIS Annual Meeting 2014Slide 11NAPHSIS Annual Meeting 2014Slide 11 When user clicks on the indictor row, the peer group statistics are shown Large Statewide Medium Small Peer groups based on birth count in 2013: 10 large hospitals, 22 medium, 85 small

12. NAPHSIS Annual Meeting 2014Slide 12NAPHSIS Annual Meeting 2014Slide 12 Click on peer group to get hospital-level statistics

13. NAPHSIS Annual Meeting 2014Slide 13NAPHSIS Annual Meeting 2014Slide 13 Click on the hospital to see the line listing of certificates with problems

14. NAPHSIS Annual Meeting 2014Slide 14NAPHSIS Annual Meeting 2014Slide 14 Report details customized to indicator

15. NAPHSIS Annual Meeting 2014Slide 15NAPHSIS Annual Meeting 2014Slide 15 User can sort the rows by a columns’ values In this case bringing cases with most errors to top

16. NAPHSIS Annual Meeting 2014Slide 16NAPHSIS Annual Meeting 2014Slide 16 Sorting by error share to direct intervention

17. NAPHSIS Annual Meeting 2014Slide 17 Hospital user display features Show error rate and how it compares to other hospitals Show proportion of errors the hospital is responsible for statewide and within a peer group Use color to direct attention to problems Provide a line listing of certificates with problems

18. NAPHSIS Annual Meeting 2014Slide 18NAPHSIS Annual Meeting 2014Slide 18 Hospital is known from the local login process Date range defaults to the current year Hospital can choose a specific date range for the report as needed

19. NAPHSIS Annual Meeting 2014Slide 19NAPHSIS Annual Meeting 2014Slide 19 Tabular report shown in bar chart, hospital with a handful of problems

20. NAPHSIS Annual Meeting 2014Slide 20NAPHSIS Annual Meeting 2014Slide 20 Hospital with two handfuls of problems l

21. NAPHSIS Annual Meeting 2014Slide 21 Observations Good – Development was intensive iterative process – We are happy with the product and its functionality – We built a hospital usage report to track activity Not so good – We did not build a summary report that shows us the biggest problem facilities across all indicators – Adoption by hospitals has been slow Requires individual log-in assignment outside the electronic birth system We have not identified the correct staff within hospitals and hospital systems who are most likely to use the dashboard

22. NAPHSIS Annual Meeting 2014Slide 22 NAPHSIS ANNUAL MEETING| Seattle | June 8-11, 2014 VITAL RECORDS: A CULTURE OF QUALITY Thank You John Paulson john.paulson@odh.ohio.gov 614-644-8507john.paulson@odh.ohio.gov