Evaluation of EHDI Programs ________________________ Terry Foust, Au.D., CCC-A/SLP Karen Muñoz, Ph.D., CCC-A Kathleen Watts, M.S. National Center for Hearing Assessment & Management Utah State University

Why Evaluate? Acountability Acountability Demonstrate program improvement over time Demonstrate program improvement over time Demonstrate evidence based practice Demonstrate evidence based practice Demonstrate attainment of program goals (1.3.6) Demonstrate attainment of program goals (1.3.6) Required by grants, funding agencies Required by grants, funding agencies

Regular and continuous evaluation Regular and continuous evaluation Identifies areas to work on for improvement Identifies areas to work on for improvement

Identify Star Performers

Levels of Evaluation Screening Screening Diagnostic Diagnostic Early Intervention Early Intervention State Program State Program Financing the program Financing the program

Screening Level Statistics Number of live births Number of live births Number and percent screened Number and percent screened Number and percent missed Number and percent missed Number and percent referred Number and percent referred

Information needed to identify possible solutions Equipment at each hospital Equipment at each hospital Screening protocol Screening protocol Screeners Screeners Screener training Screener training Designated program coordinator Designated program coordinator Notification of parent/physician/state Notification of parent/physician/state Tracking protocol Tracking protocol

Diagnostic Level Statistics Number and percent obtaining outpatient testing Number and percent obtaining outpatient testing Number and percent identified with a hearing loss Number and percent identified with a hearing loss Average time lag between screening and diagnostic testing Average time lag between screening and diagnostic testing Average age of identification Average age of identification Number and percent lost to follow-up Number and percent lost to follow-up

Information needed to identify possible solutions Number of pediatric test sites with appropriate equipment Number of pediatric test sites with appropriate equipment Location of test sites Location of test sites Audiologist compliance with reporting procedures Audiologist compliance with reporting procedures Protocol for linkage with EI Protocol for linkage with EI

Early Intervention Level Statistics Number and % enrolled in EI Number and % enrolled in EI Number and percent lost to follow-up Number and percent lost to follow-up Average age enrolled in EI Average age enrolled in EI Average age fit with amplification Average age fit with amplification Average time lag between diagnosis and intervention Average time lag between diagnosis and intervention

Information needed to identify possible solutions Timeliness of referrals to EI following diagnosis Timeliness of referrals to EI following diagnosis Timeliness of intervention following enrollment Timeliness of intervention following enrollment Training/knowledge of EI case managers on issues related to hearing loss Training/knowledge of EI case managers on issues related to hearing loss Reporting protocol from EI to State EHDI coordinator Reporting protocol from EI to State EHDI coordinator

State Program Level Support Parent-to-parent family support Parent-to-parent family support Dissemination of resources for families Dissemination of resources for families Availability of medical home for all newborns Availability of medical home for all newborns Availability of cultural and linguistic family support Availability of cultural and linguistic family support Performance feedback from state to hospitals Performance feedback from state to hospitals

Tracking and Quality Assurance Program improvement over time Program improvement over time Identification of expected number of infants with hearing loss Identification of expected number of infants with hearing loss Attainment of program goals (1.3.6) Attainment of program goals (1.3.6) Existence of periodic and ongoing evaluation, action plan, and monitoring Existence of periodic and ongoing evaluation, action plan, and monitoring Identification & implementation of educational needs Identification & implementation of educational needs

Procedural Issues – keeping your head above water!

Hospital Site Visits

Information needed to identify possible solutions Existence of financial support for diagnostic testing, and amplification Existence of financial support for diagnostic testing, and amplification Existence of a hearing aid loaner program Existence of a hearing aid loaner program Mechanism for: Mechanism for: Informing families Informing families Informing practitioners Informing practitioners Dissemination of materials Dissemination of materials

How to Evaluate Hospital surveys Hospital surveys Physician surveys Physician surveys Parent surveys Parent surveys Patient focus groups Patient focus groups Getting patients back for follow up Getting patients back for follow up Data Analysis Data Analysis Hospital site visits Hospital site visits

Hospital Survey

Physician Survey

Parent Surveys

Getting Patients Back for Follow Up How are programs getting patients back for follow up? How are programs getting patients back for follow up? Audiologists are key Audiologists are key

Data Analysis Importance of tracking software Importance of tracking software Is it flexible enough to generate a wide variety of program statistics? Is it flexible enough to generate a wide variety of program statistics? Is it compatible with a variety of programs and equipment? Is it compatible with a variety of programs and equipment? Examples Examples

Lessons Learned From Site Visits Level of commitment to program is evident Level of commitment to program is evident Challenges are brought to life Challenges are brought to life Importance of onsite problem solving Importance of onsite problem solving Equipment and data management processes are clear Equipment and data management processes are clear Staff input reflects program health Staff input reflects program health

Summary Evaluation takes time and effort Evaluation takes time and effort Must be well planned Must be well planned