1. Using ECI Data to Make Individual and Program Level Decisions Judy Carta, Ph.D & Dale Walker Ph.D. Part 2: ECI Workshop Juniper Gardens Children’s Project September 23, 2005

2. Current IGDI and ECI Development Staff Charles Greenwood, Judith Carta, Dale Walker, Jay Buzhardt and Barbara Terry Original development team: Judith Carta, Charles Greenwood, Dale Walker Jane Atwater, Gayle Luze, Deborah Linebarger, Carol Leitschuh, Ken Parsley, Annessa Staab, Gabe Cline, and Susan Higgins

3. Goals of this session Learn how to use the ECI to make individual intervention decisions Use the Decision-Making Model for children who may need intervention Use Exploring Solutions Model for individual children Discuss potential interventions for promoting communication Learn how to use ECI to make program decisions

4. Focus on Outcomes-Based Program Evaluation Increased expectations for accountability Programs and individual staff members need to know when they are making a difference in moving children toward outcomes EC practitioners need to know when interventions are effective

5. Most available approaches to program evaluation: Conventional forms of early childhood assessment not linked to individual child progress on important outcomes Child data not used in making ongoing program decisions Not repeatable enough Not accessible enough for program staff Not sensitive to intervention or program effects or growth over time Difficult for parents and practitioners to understand

6. The ECI can be used for Identifying children who are not progressing at expected benchmark levels Monitoring growth over time of individual or groups of children Assisting in intervention planning process Monitoring progress during and after intervention

7. Focus so far has been… 1. Describing outcomes for children in the program 2. Identifying children who are below expected benchmarks 3. Monitoring children’s growth over time

8. Child slipping below benchmark.

9. 36 Mos Expectation Child’s Observed Trajectory Normative Trajectory Monitoring Growth Over Time

10. We want you…. To get the most from your ECI data.

11. Using ECI Data for Decision- Making about Individual Children

12. Just like Pediatric Height & Weight Charts The ECI is useful both for monitoring AND for examining effects of intervention

13. IGDIs ESAs IGDI Decision Making Model Monitor Identify/ Validate Need For Intervention Generate Intervention Strategies Implement Intervention Evaluate Intervention Effectiveness Monitor ©2003 Juniper Gardens Children’s Project

14. What is Individual Monitoring? Doing quarterly assessments on each child Seeing if any individual falls below benchmark Taking action if child falls below benchmark

15. During quarterly monitoring If child scores…And….Then… Slightly below averageNo other concernsContinue quarterly monitoring Slightly below averageOther concernsBegin monthly monitoring Below averageNo other concernsBegin monthly monitoring Below averageOther concernsGo to Exploring Solutions On TargetNo other concernsContinue quarterly monitoring

16. During monthly monitoring If child…And….Then… Continues to score slightly Below Average No other concernsContinue monthly assessments Continues to score Slightly Below Average Other concernsGo to exploring solutions Continues to score Below Average No other concernsGo to Exploring Solutions Returns to On TargetNo other concernsReturn to quarterly

17. Using the ECI to monitor before and after intervention

18. Assessment Guidelines: What is needed in decision-making? Make decisions on more than one assessment point Use other data and information from parents and other caregivers Check that assessment is representative of the child’s functioning Consider that child may already be receiving Early Intervention (Part C)

19. Assessment Guidelines: Determining if ECI representative Does low ECI score fit with what you already know about child’s communication? Questions to ask: Was the child feeling well and alert? Was the assessor familiar to the child? Was the assessment conducted in a manner that was conducive to promoting communication? If not, do another ECI to obtain a more accurate picture.

20. One data point. Child care provider says child doesn’t join in to conversations very much What’s the next step?

21. What do the data show? What do you do next?

22. What do the data show? What do you do?

23. What do the data show?

24. What do you do next?

25. What do the data show? What do you do?

26. What do the data show? What do you do?

27. Implementing and Evaluating Interventions

28. Implementing and Evaluating Interventions to Promote Communication Goals: Review General Language Intervention Strategies and Introducing and Monitoring Use of Strategies and Activities from Tool Kit Discuss Promoting Communication Strategies Discuss successes and challenges in implementing intervention Ways to monitor if intervention is happening..

29. Exploring Solutions: Using program and child Information to plan Intervention (Called Family Needs Assessment: Child Data in Tool Kit pages 3 - 4) How child is growing in communication? (ECI) Family/caregiver concerns? Medical issues? (hearing, vision, premature, illness) What child knows and does? (other measures of development) Settings, interactions, activities, routines, curriculum What is possible in terms of intervention?

30. Communication Intervention Strategies: How to use in practice Introducing strategies and activities to parent(s), home visitors, and other staff Discuss rationale for strategy May provide modeling, coaching and practice Be positive Use self-evaluation form Using ECI to monitor intervention outcome

31. Selecting Intervention from ECI Dropdown Menu When ready to begin an intervention Select intervention description from menu on Child Data Entry Form Select menu under Condition Change Select intervention that best describes the intervention Use Other and describe in child’s file if none fit What are others that would be helpful?

34. Using effective interventions to promote communication Importance of using effective, evidence- based interventions to promote communication Importance of deciding what the possible resources are to effectively deliver intervention Importance of monitoring the intervention delivery to make sure that it is effective in promoting communication

35. General Language Intervention Strategies: Tool Kit Establishing joint attention Balancing interactions (communication turns) Matching utterances Being responsive Following child’s lead Being positive

36. Strategies from Promoting Communication Project, JGCP Environmental arrangement (activities to promote attention, book-reading etc.) Following infant/toddler’s lead and Joint Attention Commenting and labeling Expanding on child communication Increasing responsiveness (talking more) Using questions (open-ended and yes/no) Prompts for initial vocal responses if needed Praise and positive attention Time delay

37. Promoting Communication: Arranging the Environment Designing the environment to promote communication Having books to read in quiet, comfortable area Regular routines Displaying pictures at child’s eye level Provide opportunities for exploring environment

38. Following Child’s Lead Noticing and talking about the child’s interest, activity, or materials Using child’s interest to provide opportunities for communication Describe child’s actions Allow child to direct activities Give choices Adult: “ I’ll dance with you.”

39. Commenting and Labeling Describing actions, toys, activities Children hear talk about actions and hear labels Comment on what the infant is playing with Talk about the pictures in a book Teacher: Mmm. Milk Teacher: Keysha is swinging.

40. Imitating and Expanding Imitating is repeating a child’s vocalizations or words Repeat sounds child makes Expanding is adding something new to what the child said Add to the child’s statement by describing in slightly more complex way Dog ! Adult: That’s a big brown dog. Eat! – Eat! Adult: Eat. We’re eating carrots.

41. Asking open-ended questions Asking who, what, when, where, why questions that children can answer in multiple ways Ask questions related to child’s play or interest Ask questions related to routines Pause after asking question, if no answer, fill in answer yourself Ball! – Ball ! Adult: Where are you throwing the ball? Adult: What are we doing? We’re changing your diaper!

42. Time delay/ Fill in the blank Planning a delay during a predictable routine is promotes communication Giving child the chance to fill in the blank in a song or common phrase they are familiar with Children practice talking Emphasizes child’s success with words Ashes, ashes They all fall ___. Down!

43. Positive attention and praise Attending to and being positive about behavior and communication Positive attention for communication and prosocial behaviors creates opportunities for children to practice skills Playing with children gives positive attention Adult: You’re such a great helper. Paying attention to good behavior

44. Providing Choices Providing choices means the child will need to communicate their preferences Encourages communication Provide choices by rotating materials At snack time present two options Give child options in play and book reading Adult: Would you like blue or green? Adult: Which book do you want to read?

45. Evaluating use of language promoting interventions Complete self evaluation form to document use of intervention Make self evaluation practical for parents and home visitors What to do with information from self evaluation? Determine if intervention is being implemented as intended If not, make modifications in intervention

47. Evaluating and monitoring intervention effectiveness Continue administering ECI monthly or quarterly Look at Key Skill Elements and Total Communication rates to check that they are moving toward benchmark Look at the slope of communication progress to determine if the slope is increasing Is there evidence that intervention has increased communication? If progress is not at expected rate, examine intervention and make necessary changes

48. Child below benchmark after intervention

49. Key Skill Element: Single Words

50. Key Skill Elements: Multiple Words

51. Using ECI Data for Decision- Making about Programs

52. For groups of children, ECI data can tell us… How children in our program are performing relative to benchmark on communication outcome. Whether our program is moving children toward important outcomes. When to consider modifications in program. When we institute a change in the program, whether it makes a difference in percentage of children who are on- target.

53. Which children are currently off- target? Child NameRisk Status Randall, DonnieSlightly Below Tarantino, QuentinSlightly Below Pitt, BradBelow Average Leary, TimothyBelow Average

54. Identify children who are not on- target Are there more/less than the previous month? What might account for this? What strategies are in place for off-target? More frequent monitoring? Language promoting strategies?

55. Histograms provide a group snapshot of Total Communication

56. Examine the percentage of children who are on-target Possible questions... How does this vary by age group? What might account for differences by age? What types of changes might you make as a result of this information?

57. Looking at program data by age group: Age Group On Target Slightly Below Average Below AverageTotal 1-11 Months 11 100%0% 11 100.0% 12-23 Months 16 94.1% 1 5.9%0% 17 100.0% 24-35 Months 19 90.5%0% 2 9.5% 21 100.0% 36-47 Months 7 77.8% 1 11.1% 9 100.0% Overall 53 91.4% 2 3.4% 3 5.2% 58 100%

58. How is the entire group performing in key skill elements?

59. Group Graph for Single Words

60. Group Graph for Multiple Words

61. How does data change from one month to the next?

62. Looking at program data month- by month Age Group Percentage of ON-TARGET MarchJuneSeptDecember 1-11 Months 11 100% 11 100.0% 12-23 Months 16 94.1% 17 95.9% 17 95.9% 17 100.0% 24-35 Months 19 90.5% 20 95.5% 20 95.5% 21 100.0% 36-47 Months 7 77.8% 7 77.8% 7 77.8% 9 100.0% Overall 53 91.4% 57 94.5% 57 94.5% 67 100%

63. Questions to ask if program is not meeting your expectations: Are high quality language-promoting strategies being implemented? Is the program being delivered at the expected levels of intensity, duration? Are children/families participating at appropriate levels? Are additional levels of supports, resources, training necessary?

64. How Approach is Used to Inform Decision-Making at the Program Level Administrative and program staff may use ECI for : Program evaluation Program planning Quality improvement Inform public and funding sources about program progress

65. Importance of Sensitive Measurement When program staff have tools that more adequately reflect change….

66. They will be more effective in guiding positive change in programs.