1. Session 3: Assessment & Evaluation Lisa Jacques-Carroll, MSW NCIRD, CDC

2. Learning Objectives  Describe key components of evaluation of a perinatal hepatitis B prevention program describe different methods for assessing and evaluating a perinatal hepatitis B prevention program describe different methods for assessing and evaluating a perinatal hepatitis B prevention program identify several tools that are available identify several tools that are available

3. Assessment & Evaluation  Program evaluation vs. program monitoring or assessment  The focus of this session is on assessing performance measures, which is a component of program evaluation  For more information on program evaluation see:  For more information on program evaluation see: www.cdc.gov/vaccines/programs/progeval/

4. Activities Required by CDC  Requirements for assessment of the perinatal hepatitis B include: frequency of assessment frequency of assessment performance measures performance measures target levels target levels  For more information: http://www.cdc.gov/vaccines/vac-gen/policies/ ipom/downloads/chp-05-perinatal-hepb.pdf

5. Overview  Laboratory reporting audits  Hospital record reviews  Hospital policy surveys  Expected births to HBsAg+ women  Assessing case management  Monitoring birth dose

6. Laboratory Reporting Audits  Use to assess: completeness of reporting of HBsAg+ results completeness of reporting of HBsAg+ results timeliness of reporting timeliness of reporting  Include delivery hospital labs  Collaborate with communicable disease and state laboratory staff to help plan and execute reviews

7. Prioritize Labs to Evaluate  Priority labs: labs serving high-morbidity areas or populations labs serving high-morbidity areas or populations labs reporting large volume of hepatitis serology labs reporting large volume of hepatitis serology labs serving prenatal clinics labs serving prenatal clinics delivery hospital labs delivery hospital labs  How often? ideally, once/year for priority labs ideally, once/year for priority labs every 2–3 years for others every 2–3 years for others minimally every 5 years minimally every 5 years

8. Laboratory Audit Tool  A CDC tool available at: http://www.cdc.gov/ncidod/diseases/hepatitis /resource/LabReportingAssessment.doc

9. Completeness of Lab Reporting  Pick a time period (at least 3 months)  Calculate: Total # HBsAg+ test results reported Total # specimens that were HBsAg+ = Proportion of HBsAg+ tests reported, or completeness of reporting

10. Timeliness of Lab Reporting Date of HBsAg serology Date HBsAg+ report rec’d by health dept # Days between serology and report 8-18-20078-23-20075 8-23-20078-29-20076 9-7-20079-12-20075 11-18-200711-27-20079 SUM25

11. Timeliness of Lab Reporting - cont’d = Mean reporting time for the laboratory, or timeliness of reporting Time period for reporting is typically set by state statute Sum of column “# days between serology and report” Total # of serologies evaluated 25 4 = 6.25 days e.g. from the previous slide

12. Tips for Working with Laboratories  Contact other areas of health department to see if activity is already being done  See if an existing lab report can be used for the audit—ensure it is source data  When making contact with labs, include: lab director lab director technician technician IT staff IT staff

13. Hospital Record Reviews  Use to assess: maternal HBsAg screening maternal HBsAg screening birth dose administration birth dose administration post-exposure prophylaxis (PEP) for infants born to HBsAg+ and unknown-status women post-exposure prophylaxis (PEP) for infants born to HBsAg+ and unknown-status women  Audit paired maternal and infant records  Partner with health dept programs to collect other perinatal data rubella antibody HIV rubella antibody HIV group B strep early hearing group B strep early hearing syphilis syphilis

14. Determining Sample Size  Tool available in Guide to Life (Table 2.1) http://www.cdc.gov/ncidod/diseases/ hepatitis/resource/perinatalhepB.htm

15. Using the Guide to Life table  Determine hospital’s annual number of deliveries  Decide where it falls in the “Birth Cohort Size” category  Estimate HBsAg screening and hepatitis B birth dose coverage for the hospital  Use the lower coverage to determine the sample size

16. Sample Size Example Hospital X had 648 deliveries last year The expected HBsAg screening rate is 95% The state National Immunization Survey (NIS) birth dose rate is 65% Using the 65% coverage level, the sample size for Hospital X would be 111 records

17. Hospital Data to Collect  Delivery date/time  Mother’s HBsAg test date/result/type in: maternal record maternal record infant record infant record  Infant hepatitis B vaccination (yes/no)  Date/time of vaccination  Infant HBIG administration (yes/no)

18. Hospital Data to Collect - cont’d  Date/time of HBIG  Other variables as appropriate

19. Hospital Policy Surveys  Use to assess whether hospitals have written policies/standing orders in place: universal birth dose of hep B universal birth dose of hep B HBsAg screening of pregnant women HBsAg screening of pregnant women PEP to infants born to HBsAg+ and unknown- status women PEP to infants born to HBsAg+ and unknown- status women documentation of maternal HBsAg status, birth dose, and HBIG (if appropriate) in medical records documentation of maternal HBsAg status, birth dose, and HBIG (if appropriate) in medical records

20. Conducting Policy Surveys  Mail survey  Telephone survey  In-person survey during hospital during hospital medical record medical record reviews reviews Keep in mind, policies do not always match practice, so medical record reviews are essential

21. Expected Births to HBsAg+ Women  CDC expected birth estimates are a benchmark for programs  CDC provides estimates for states and some cities  Estimates are calculated using: birth data birth data prevalence of chronic prevalence of chronic HBV infection by HBV infection by race/ethnicity race/ethnicity

22. Expected Births - cont’d  Nationally, about 24,000 births expected annually, only 50% identified in 2005  CDC encourages programs to improve estimates of births to HBsAg+ women  More information in presentation at: http://www.cdc.gov/ncidod/diseases/hepatitis/resource/ presentations/2007-04PeriGrantees/PDFs/May1- p0130-1LJ_2007PeriCoordMtg-ExpectedBirths.pdf

23. Assessing Case Management  Number of infants born to HBsAg+ women  Proportion of infants receiving hep B vaccine & HBIG within 12 hours of birth hep B vaccine & HBIG within 12 hours of birth on-time completion of hep B vaccine series on-time completion of hep B vaccine series post-vaccination testing post-vaccination testing  Reasons for cases lost to follow-up  Proportion of screened and vaccinated household and sexual contacts

24. Assessing Case Mgmt - cont’d  Monitor case management outcomes: compare program-wide rates (entire state or city) to national rates (CDC peritable) compare program-wide rates (entire state or city) to national rates (CDC peritable) regionally regionally locally (county level) locally (county level) by case worker by case worker quarterly or monthly quarterly or monthly over time (trends) over time (trends)

25. Monitoring Birth Dose  National Immunization Survey (NIS) data measures hepatitis B birth dose rates measures hepatitis B birth dose rates  Hospital medical record reviews  Immunization Information Systems (IIS) hep B vaccine birth certificate data automatically uploaded into IIS hep B vaccine birth certificate data automatically uploaded into IIS calculate birth dose coverage for cities, regions, and individual hospitals calculate birth dose coverage for cities, regions, and individual hospitals

26. Continuous Monitoring  Some aspects of the program require continuous monitoring case management outcomes case management outcomes laboratory reporting laboratory reporting  Plan ahead for assessments your program will conduct over the next year/s

27. Using Your Results  Determine future direction and focus of program efforts  Provide feedback to: hospitals hospitals laboratories laboratories providers providers health departments health departments case workers case workers