1. Early Childhood Outcomes Center New Tools in the Tool Box: What We Need in the Next Generation of Early Childhood Assessments Kathy Hebbeler ECO at SRI International Presented at the OSEP EC Conference, December 2008

2. Early Childhood Outcomes Center Today’s Topics The new National Academy of Sciences Report: Early Childhood Assessment: Why, What, How What should the next generation of assessment tools look like?

3. Early Childhood Outcomes Center

4. Reason for the Committee Controversy over Head Start National Reporting System (NRS) Congressional mandate “review and provide guidance on appropriate outcomes and assessments for young children.” 2 key topics: Identification of key outcomes for children 0-5 Quality and purpose of different state-of-the art techniques

5. Early Childhood Outcomes Center Organization of the Report Overview of purposes What to assess: Outcomes And Quality of Environment How to assess: Technical discussion of validity Issues related to minority, ELL, children with disabilities Thinking systematically

6. Early Childhood Outcomes Center 2 Key Principles The purpose of the assessment should guide assessment decisions Assessment should be conducted within a coherent system of medical, educational, and family support services, that promote optimal development for all children

7. Early Childhood Outcomes Center Purposes Determining an individual child’s level of functioning Individual-focused screening Community-focused screening Diagnostic testing Establishing readiness

8. Early Childhood Outcomes Center Purposes (Cont’d) Guiding intervention and instruction Evaluating the performance of a program or society Program effectiveness Program impact Social benchmarking Advancing knowledge of child development

9. Early Childhood Outcomes Center Guidelines from Multiple Groups Assessment should benefit children Assessments should meet professional, legal, ethnical, standards Assessments should be age-appropriate or developmentally/individually appropriate Parents/family should be involved in the assessment whenever possible Assessments should be linguistically and culturally appropriate/responsive

10. Early Childhood Outcomes Center Guidelines from Multiple Groups (Cont’d) Assessments should assess developmentally/educationally significant content Assessment information should be gathered from familiar contexts (NEGP), realistic settings and situations (NAEYC), or be “authentic” (DEC). Information should be gathered from multiple sources. Screening should be linked to follow-up assessment.

11. Early Childhood Outcomes Center Recommendations Related to Purpose Make assessment purpose explicit and public Assessment strategy must match purpose

12. Early Childhood Outcomes Center [Which] Outcomes Why this domain? Evidence of consensus around value Evidence of continuity within domain over development or links to other domains Evidence that it is a target for intervention and affected by the environment

13. Early Childhood Outcomes Center Outcome: Domains (NEGP) 1.Physical well-being and motor development 2.Socioemotional development 3.Approaches to learning 4.Language (and emerging literacy) 5.Cognitive skills (including mathematics)

14. Early Childhood Outcomes Center Classification Challenges Boundaries are artificial Researchers (and test developers) use different classification schemes

15. Early Childhood Outcomes Center Recommendations related to Domains and Outcomes Go beyond the standard domains of language and mathematics Need support for better measures of socioemotional development D-4 “For children with disabilities and specials needs, domains-based assessments may need to be replaced or supplemented with more functional approaches”

16. Early Childhood Outcomes Center Validity and Reliability Weight of evidence Not whether an assessment is valid and reliable Use accumulation of evidence to judge whether the assessment is suitable for the task for which it is intended. Evidence pertains to specific type of uses.

17. Early Childhood Outcomes Center Assessing All Children Much more research needs to be conducted on the validity of tools with minority children, ELL, and children with disabilities. Children with disabilities Construct irrelevant skills as a threat to validity Interrelatedness of domains Functional outcomes Universal design Many tools used with children with disabilities have not been validated for the purposes for which they are being used.

18. Early Childhood Outcomes Center Instrument Selection Recommendations Select assessment with acceptable levels of validity Examine data produced with the assessment for validity for purposes for which the assessment is being used. Test developers need to collect and make available evidence related to validity for minority, ELL, children with disabilities. Be very cautious about reaching conclusions about a group not well represented in the validation sample.

19. Early Childhood Outcomes Center Situating Assessment in a System EC System has A goal for children Strategies to achieve this goal Infrastructure supporting the goal and the strategies Selects assessment to be compatible with other elements of the system.

20. Early Childhood Outcomes Center Systems Recommendations EC assessment system must be part of a larger system with strong infrastructure to support early care and education. Standards Assessment Reporting Professional development Opportunity to learn Inclusion Resources Monitoring and evaluation

21. Early Childhood Outcomes Center Systems Recommendations Coherence Horizontally coherent – curriculum, instruction, assessment aligned with early learning standards Vertically coherent – shared understanding of goals at all levels; consensus about purposes of assessment Developmentally coherent – taking into account what is known about young children’s skills and understanding develop

22. Early Childhood Outcomes Center Research Needed New conceptual frameworks Application of technical advances, e.g., IRT Span birth to 6 or 7 Social emotional measures Capture children’s growth toward being able to meaningfully participate in variety of everyday setting Use of technology

23. Early Childhood Outcomes Center Challenges with Current Assessments for All Children/Children with Disabilities Domains-based When should assessment look at only one domain? When should it look at the whole child? How functional are current tools? Varies Need for tools to assess the three outcomes How authentic? How real? How much is lost with structured requests? And for which children?

24. Early Childhood Outcomes Center Challenges with Current Assessments for All Children/Children with Disabilities Tool structure E.g., Work Sampling Need for specificity – sometimes.. E.g., AEPS; High Scope COR Do we want predictive validity? Universal design principles DRDP as the exception Adaptations and accommodations

25. Early Childhood Outcomes Center Challenges with Current Assessments for All Children/Children with Disabilities Diagnosis vs. intervention planning purpose Must these be separate tools? Are the things we want to know about children to prove eligibility so different from what we want to know to develop an intervention plan? AEPS in JEI, December 2008 Recognition and Response/RTI’s tools for young children IGDI’s How do these fit with curriculum-based assessment?