Using the Web to Support Data-based Decision Making Jay Buzhardt, Dale Walker, Charles Greenwood Juniper Gardens Children’s Project Division of Early Childhood Conference October 15, 2010

IGDI Website: Publicly accessible material IGDI descriptions IGDI descriptions Materials needed for IGDI implementation Materials needed for IGDI implementation Scoring definitions Scoring definitions Scoring sheets Scoring sheets Administration checklists Administration checklists Toys for administration Toys for administration Psychometric properties Psychometric properties Training and certification resources Training and certification resources Intervention materials Intervention materials Contact information Contact information Online data system (password-protected) Data management Web-based certification Assessor management Child rosters and demographics Program-wide aggregated reports Data-based intervention decision making tools Making Online Decisions (MOD)

Current Data System and MOD Use IGDI Online data system use IGDI Online data system use Over 32,000 assessments entered into the data system Over 32,000 assessments entered into the data system Over 11,000 infants and toddlers assessed Over 11,000 infants and toddlers assessed Approximately 1,100 user accounts (certified assessors, program directors, data entry personnel, trainees, etc.) Approximately 1,100 user accounts (certified assessors, program directors, data entry personnel, trainees, etc.) Usage covers 22 states and three countries Usage covers 22 states and three countries MOD currently being tested in a limited number of midwestern Early Head Start and Part C programs MOD currently being tested in a limited number of midwestern Early Head Start and Part C programs Five Kansas Early Head Start programs currently using the MOD Five Kansas Early Head Start programs currently using the MOD Iowa Heartland Area Agency (Part C) Iowa Heartland Area Agency (Part C)

Graphs and Reports that Support Data-based Decision Making Base level support… Base level support… Progress Monitoring Graphs Progress Monitoring Graphs Weighted Total Communication Weighted Total Communication Key Skill Elements Key Skill Elements Program Reports Program Reports Identify children falling below benchmark Identify children falling below benchmark Identify children due for an assessment Identify children due for an assessment Individual Child Reports Individual Child Reports

Child’s Data Summary Child’s Data Table Individual Child Report

Progress Monitoring Graph: Weighted Total Early Communication Slightly Below Benchmark Below Benchmark

Key Skill Elements: Gestures Graph

Key Skill Elements: Vocalizations Graph

Key Skill Elements: Single Words Graph

Key Skill Elements: Multiple Words Graph

Additional Support: Making Online Decisions (MOD) System Automate identification of low-performing children Automate identification of low-performing children Document decision making progress within the data system Document decision making progress within the data system Intervention(s) used Intervention(s) used Work with caregivers on implementing intervention strategies Work with caregivers on implementing intervention strategies Caregiver intervention implementation Caregiver intervention implementation Intervention changes Intervention changes Analysis of child’s response to the intervention Analysis of child’s response to the intervention

Yes MOD Overview No Quarterly ECI Assessments How is the intervention being used? What intervention should be used Yes What is causing the problem? Is there a problem? No Is the intervention working?

Randomized Trial of the MOD’s Efficacy Primary Research Questions Primary Research Questions Does the MOD impact child outcomes? Does the MOD impact child outcomes? Does the MOD affect service providers’ decision-making behavior? Does the MOD affect service providers’ decision-making behavior? Participants: 5 Kansas Early Head Start programs Participants: 5 Kansas Early Head Start programs Home visitors in each program randomly assigned to either use the MOD (n=28) or continue standard services (n=24) Home visitors in each program randomly assigned to either use the MOD (n=28) or continue standard services (n=24) NonMOD group (standard services): used the ECI, entered data into data system, provided and/or referred child for intervention services as needed NonMOD group (standard services): used the ECI, entered data into data system, provided and/or referred child for intervention services as needed MOD group: Standard services + the MOD MOD group: Standard services + the MOD Children whose ECI score fell at least 1 sd below benchmark either receive MOD services (n=45) or standard services (n=53) Children whose ECI score fell at least 1 sd below benchmark either receive MOD services (n=45) or standard services (n=53)

Effect of the MOD on ECI Outcomes Level-1 Analysis (Children who score at least 1 sd below benchmark on the ECI)

Effect of the MOD on ECI Outcomes Level-2 Analysis (Children who score at least 1 sd below benchmark on the ECI)

Effect of the MOD on Home Visitors’ Intervention Services

Home Visitor Perception of the MOD