Community Input Discussions: Measuring the Progress of Young Children in Massachusetts August 2009

Statement of Intent Massachusetts is in the early stages of developing a statewide system to measure developmental progress of its young children EEC is engaging parents, providers, program administrators, teachers, higher education institutions, and policy makers to build a responsive approach This initiative is separate from (and would not replace) developmental information that programs gather about children to use for curriculum planning and to individualize instruction

Questions for Stakeholder Input What are your hopes for measuring the developmental progress of young children in state? How do you hope information will be used? What are the most important things to measure about school readiness? What are some of the challenges you foresee in moving forward with this effort?

Measuring School Readiness Across the Country Massachusetts is joining rapidly growing trend to understand school readiness 29 states currently collect statewide data on children’s progress These efforts are often directly connected to state-funded preschool efforts

Why Are Statewide Data Needed? To inform policy makers about the benefits of and impact of investments in early childhood education and care in Massachusetts To better understand school readiness gaps(s) for subgroups of children To inform statewide policy development Data will not be used for high stakes “testing” of young children or providers

Statewide Measures of School Readiness vs Statewide Measures of School Readiness vs. Child Assessments and Screenings Currently, many providers are already using a developmental assessment or screening tool to inform practice and individualize instruction Providers are currently using a variety of assessment measures UPK grantees are required to use one of four assessment systems Creative Curriculum Developmental Continuum Ages & Stages High Scope Child Observation Record (COR) Work Sampling System Statewide measurement of school readiness is not intended to be used to replace program-level assessment practices

Statewide Measures of School Readiness vs Statewide Measures of School Readiness vs. Child Assessments and Screenings Different purposes Statewide system: information about the success of all children in Massachusetts Program-level assessments: information for parents and caregivers about individual children Different level of information about child Statewide system: measure a small number of indicators of school readiness Program-level assessments: comprehensive look at child progress across all developmental domains

Different usefulness to providers Statewide Measures of School Readiness vs. Child Assessments and Screenings Different usefulness to providers Statewide system: for broader policy purposes Program-level assessments: to help provider support each child’s growth and development Other differences Statewide system: children will be anonymous when data are reported

Design Options for Statewide Measure: Key Issues to Consider WHAT to measure about child development WHO to measure WHEN to measure HOW will measures/information on children be collected WHAT ELSE to measure about home environments, program characteristics, and community context

Design Options for Statewide Measure: Key Issues to Consider WHAT: how broadly should we measure children’s development – which domains are most important? WHO: all children and programs, or just a sample? WHEN: longitudinal data in preschool only (ages 3 and 4) or linking with school data (K and/or grade 1)? HOW: providers trained to administer measures, outside evaluators, and/or other informants?

Other State Approaches All states look at development either at one point in time or over time, usually using different samples of children at each age All states interested in measuring school readiness skills, although the skills that they measure vary Types of measures used vary (standardized assessment vs. developmental observations, number of development areas measured) Where system is ongoing, states involve providers in data collection

Other States: Approaches Using Providers as Data Collectors MD: Ongoing assessment of all children at school entry by kindergarten teachers, using Work Sampling System PA: Ongoing measures of all children three time per year during preschool and kindergarten by providers, using Work Sampling System NC: Snapshot of school readiness information using variety of measures from a sample of principals, kindergarten teachers, parents, and children just entering kindergarten, coordinated by research firm

Key Issues Raised To Date (based on six stakeholder meetings) Engage parent and provider support for initiative to ensure maximum participation Make sure purpose of initiative is clear Ensure information collected about children does not reflect negatively on families Consider how to support providers and minimize burden if they will be on the front lines of the measurement effort Consider how to provide context for data on development of vulnerable subgroups of children Consider linking child assessment data to program quality data to better understand linkages between quality and child outcomes

Key Decisions Moving Forward: WHAT to measure? Narrow vs. broad measurement Tension between desire to measure “whole child” and what is feasible to collect  Which skills/outcomes to measure Focus on outcomes that research tells us are related to success in school such as: Academic skills in reading, writing, and/or math Social skills Cognitive and behavioral self-regulation

Key Decisions Moving Forward: WHO to measure? Could measure as broad a population as possible or target more narrowly Might oversample particular subgroups of interest Effort will likely be limited to preschool children in licensed and license-exempt settings Probably not feasible to access children who are in informal and/or unlicensed care settings or who are not in out-of-home care at all  

Key Decisions Moving Forward: HOW to measure? Providers as assessors Advantages: cost, may help with getting parent permission, providers learn about their children’s skills Disadvantages: concerns about bias if providers assess their own children, need to train large number of providers, difficult for providers to find time to conduct quiet standardized assessment  Outside assessors Advantages: can be trained to reliability, no public concerns about partiality, possibility of building on early childhood education infrastructure in state to develop group of assessors Disadvantages: cost of conducting assessments, cost of training Parents as assessors Advantages: builds parent buy-in, not expensive Disadvantages: public perception of bias, some concepts might be hard to explain to parents, may be difficult to get parents to return this information

UPK grantees using one of four specified child assessment tools Key Decisions Moving Forward: Consider Integration/Overlap with Other Assessment Efforts UPK grantees using one of four specified child assessment tools Many other programs are using either one of these four tools or another measure School district programs using Battelle in spring of 3-year-old year with children who will be evaluated for special needs Other efforts?

Key Decision Points Moving Forward: WHEN to measure? Options for data collection schedule before school entry Measure children once, at the end of preschool, to describe school readiness of preschoolers Measure children twice, at the beginning and end of preschool, to describe school readiness but also to see whether children who start out below average are closing the gap over time Measure children three times, once during the 3-year-old year and fall and spring of the 4-year-old year. This design provides more information on progress over time   Looking at same group of children over time vs. different groups of children each time  Although following the same children would allow us to look at individual growth over time, this is a very expensive undertaking

Questions for Stakeholder Input What are your hopes for measuring the developmental progress of young children in state? How do you hope information will be used? What are the most important things to measure about school readiness? What are some of the challenges you foresee in moving forward with this effort?