Early Childhood Outcomes: Implications for Personnel Preparation and Professional Development Session Leader: Kathy Hebbeler ECO at SRI International OSEP Project Directors’ Conference Washington DC July 2007 Early Childhood Outcomes Center

2 Panel Robin Rooney, University of North Carolina at Chapel Hill, NECTAC Beth Tolley, Infant and Toddler Connection of Virginia Mary McLean, University of Wisconsin- Milwaukee

3 Objective Provide overview of OSEP’s early childhood outcomes work Provide state and higher education perspectives on training needs Facilitate a discussion of the implications of outcomes work on personnel prep and professional development needs in the field

OSEP’s Early Childhood Outcomes Work Beth Caron, OSEP

5 Why OSEP’s focus on early childhood outcomes? Improve results for young children with disabilities and their families Meeting a need in the field Development of outcomes for general early childhood programs Address GPRA, PART, and IDEA 2004

6 OSEP Reporting Requirements: Part C and Preschool Child Outcomes Percent of children who demonstrate improved: Positive social emotional skills (including positive social relationships) Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) Use of appropriate behaviors to meet their needs

7 Reporting Categories a. % of children who did not improve functioning b. % of children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers c. % of children who improved functioning to a level nearer to same-aged peers but did not reach it d. % of children who improved functioning to reach a level comparable to same-aged peers e. % of children who maintained functioning at a level comparable to same-aged peers

8 State Performance Plan (SPP) and Annual Performance Report(APR) Each state is required to submit a State Performance Plan in December 2005 and an SPP/APR beginning Feb Part C SPP has 14 indicators Part B SPP (includes preschool) has 20 indicators Early childhood outcomes are part of the SPP/APR

9 Critical Events / Timelines Spring 2005: Public input on child and family outcomes Summer 2005: OSEP released reporting requirements December 2005: States submitted SPP’s Fall 2006: OSEP expanded reporting categories February 2007: Status at entry data due February 2008: first progress data due

Preparing Professionals to Measure Outcomes: the National View Kathy Hebbeler OSEP Project Directors Meeting July, 2007 Early Childhood Outcomes Center

11 Trends in approaches to measurement for Part C child outcomes 40 states using (or will use) the ECO Child Outcome Summary Form 40 states using (or will use) the ECO Child Outcome Summary Form A 7 point rating scale based on multiple sources of data, often including assessment tools, observation, family report A 7 point rating scale based on multiple sources of data, often including assessment tools, observation, family report 8 states using 1 assessment tool statewide 8 states using 1 assessment tool statewide BDI-2: 3 states BDI-2: 3 states State developed tools: 3 states State developed tools: 3 states AEPS: 2 states AEPS: 2 states 3 states using on-line assessment systems with the capacity to report OSEP data reports 3 states using on-line assessment systems with the capacity to report OSEP data reports 5 states using other unique approaches 5 states using other unique approaches Early Childhood Outcomes Center

12

13 Trends in approaches to measurement for Preschool (Section 619) child outcomes 34 states using (or will use) the ECO Child Outcome Summary Form 34 states using (or will use) the ECO Child Outcome Summary Form A 7 point rating scale based on multiple sources of data, often including assessment tools, observation, family report A 7 point rating scale based on multiple sources of data, often including assessment tools, observation, family report 11 states using 1 assessment tool statewide 11 states using 1 assessment tool statewide BDI-2: 4 states BDI-2: 4 states State developed tools: 4 states State developed tools: 4 states AEPS, Brigance, Work Sampling: 1 state each AEPS, Brigance, Work Sampling: 1 state each 5 states using on-line assessment systems with the capacity to report OSEP data reports 5 states using on-line assessment systems with the capacity to report OSEP data reports 7 states using other unique approaches 7 states using other unique approaches Early Childhood Outcomes Center

15 Features of the Child Outcomes Summary Form It is not an assessment tool It uses information from assessment tools and observations to get a global sense of how the child is doing at one point in time 7-point rating scale Rating is based on the child’s functioning: What the child does across settings and situations Compared with what is expected given the child’s age Early Childhood Outcomes Center

16 The Two COSF Questions a.To what extent does this child show age- appropriate functioning, across a variety of settings and situations, on this outcome? (Rating: 1-7) b.Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No) Early Childhood Outcomes Center

18 Essential Knowledge for Completing the COSF Between them, team members must: 1.Know about the child’s functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

19 Implications for Personnel Preparation Understanding functional outcomes Moving beyond domains Moving beyond disciplines Understanding age expected behavior Early Childhood Outcomes Center

20 Implications for Personnel Preparation: Assessment What is good assessment? What is the role of assessment in EI and ECSE, especially ongoing assessment? Using assessment results to plan intervention Sharing assessment results with parents Early Childhood Outcomes Center

21 For More Information Early Childhood Outcomes Center

National Training on the Use of the COSF Robin Rooney University of North Carolina, Chapel Hill NECTAC

23 National Training on the Use of the COSF ECO/NECTAC staff have conducted on-site training for 22 states/jurisdictions on the COSF All ECO training materials are available on the ECO website Most, if not all, states using the COSF are using the ECO training materials

24 What We Cover in Training Why collect outcomes data? Understanding the three child outcomes Assessing the accomplishment of the three child outcomes Using the Child Outcomes Summary Form – 7- point rating scale Practice with the Child Outcomes Summary Form – 7-point rating scale

25 Essential Knowledge for Completing the COSF Between them, team members must: 1.Know about the child’s functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the child’s culture

26 Professional development challenges: COSF users struggle with The team decision-making process Age expectations for child development in the context of the three functional child outcomes Talking with families about their child’s progress compared to age expectations Functional assessment

27 COSF training challenges: Ongoing training needs for New staff -- given provider turnover Continuing staff -- follow up with those who receive COSF training to see if they are keeping with initial training messages Trainers/TA providers who will continue to support COSF users

28 The COSF Training Consortium An ECO-state collaborative Established to develop COSF training materials in tandem In order to: Ensure that providers understand and can use the COSF Improve the quality of ratings Ensure consistency in the use of the COSF Ensure consistency in the training on the use of the COSF

29 Training consortium: Scope of work New learning objects to enhance existing COSF training materials COSF train-the-trainer curricula Alternatives to in-person training Learning objectives and a process for testing mastery Quality assurance systems

30 COSF learning objects under development Written child examples for practice rating Videos of the COSF team process Materials that highlight skills needed for effective use of COSF

31 COSF Train-the-Trainer Curriculum Guidance on how to use existing materials and new learning objects Learning objectives How to test training recipients for competency/mastery How to use materials for first-time learners How to use materials for re-training

32 Alternatives to In-person Training Learning objects and strategies for using technology for distance learning, such as through on line and video training/materials Supervisors’ guide for overseeing training for small groups, individuals How to test training recipients for competency/mastery How to use curricula for first-learners versus re- learners

33 Learning Objectives and a Process for Testing Mastery A stand-alone set of learning objectives for training in the effective use of the COSF Learning objectives adapted for new learning objects and curricula A process for testing for competence in and mastery of the learning objectives across COSF training materials and curricula and across learning environments Guidance for using testing materials

34 Quality Assurance Systems How to monitor the use of the COSF, including documentation of evidence and reliability and validity of ratings Strategies for data collection and analysis for procedural quality assurance Strategies for data collection and analysis for program quality assurance Implications for program improvement, training and TA

Integrating OSEP Outcomes Content and Competencies into Higher Education Curriculum Mary McLean University of Wisconsin-Milwaukee

49 Issues What content and competencies should be included in pre-service curriculum? What are effective strategies for facilitating their inclusion in higher education curriculum?

50 What Content/Competencies Should Be Included? Background information Reporting requirements Instruments/COSF Recommended practices in assessment Managing assessment Using assessment information to improve services for children and families

51 Background Information “Why do we have to do this?”

52 Federal Initiatives 1993 Government Performance and Results Act 2001 No Child Left Behind 2002 Good Start, Grow Smart 2002 OMB Budgeting Process: PART 2004 Individuals with Disabilities Education Improvement Act (IDEA-2004)

53 Federal programs are categorized as: Performing or Not Performing Effective Results not demonstrated Mod. Effective Ineffective Adequate

54 Reporting Requirements What are OSEP’s reporting requirements? What are the child and family outcomes? How is progress toward the outcomes measured? What are the requirements for my state?

55 Instruments/COSF Use of the Child Outcomes Summary Form Administration of state-identified instruments

56 Recommended Assessment Practices Assessment is authentic - based on the completion of real life tasks in typical environments Assessment information is gathered from multiple sources including families and others who provide services to the child Assessment is systematic and ongoing

57 Assessment Tasks Screening Eligibility for special education Planning the IEP/IFSP Progress monitoring monitoring progress toward EC standards monitoring progress toward IEP/IFSP outcomes monitoring effectiveness of intervention strategies Accountability for IDEA

58 Managing Assessment Creating an “assessment framework” (Grisham-Brown, Hemmeter & Pretti-Frontzcak, 2005) Identifying assessment tools and strategies Identifying overlap Embedding assessment into ongoing activities

59 Using Assessment Information to Improve Services Appreciating the importance of assessment for improving the quality of intervention Implementing the “teaching cycle”

60 Strategies for Facilitating the Inclusion of Content/Competencies into Curriculum in Higher Education: What Does It Take? Informing faculty Sharing information Providing training

61 Wisconsin OSEP Outcomes Training Includes 6 modules: Objectives Powerpoint slides Speaker Notes Hand outs

62 California CDE/ECE Faculty Initiative Project Caroline Pietrangelo Owens, Director

Mary McLean, Ph.D. University of Wisconsin-Milwaukee