Positive Impacts and Return on Investment: What We Can Learn from Head Start Research Hilary Shager WI Department of Children and Families April 25, 2012.

Slides:



Advertisements
Similar presentations
Education and Policy aimed at Early Childhood. Education and Early Childhood Policy We have seen throughout this class that poverty and racial inequality.
Advertisements

What randomized trials have taught us about what works and doesn’t work in education Jon Baron Coalition for Evidence-Based Policy December 9, 2003.
Penny Wise and Effect Size Foolish Greg J. Duncan Northwestern University.
Sample size issues & Trial Quality David Torgerson.
1 Graduation Rates: Students Who Started 9 th Grade in 2005, 2006, 2007, 2008 and 2009.
School Report Cards 2004– The Bottom Line More schools are making Adequate Yearly Progress. Fewer students show serious academic problems (Level.
Early Childhood Education Cost Analysis & Recommendations New Mexico Early Childhood Development Partnership December 2013 New Mexico Early Childhood Development.
A Framework for Minnesota
What do we know about effective preschool programs? Steve Barnett, PhD.
School Report Cards For 2003–2004
Financing Early Childhood Care and Education: A Critical Investment Larry Schweinhart South Asian Regional Conference in ECCE New Delhi August 27, 2012.
HEAD START WORKS !!!. Head Start began in 1965 as part of the War on Poverty program launched by president Lyndon B. Johnson. Nearly half the nation's.
1 James P. Smith Childhood Health and the Effects on Adult SES Outcomes.
School dropouts: A framework for public policy analysis Joshua Hawley, The Ohio State University.
UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Keeping Kids in School:
Fertility (within and across cohorts)  What is available over time?  Longitudinal perspectives  Complimentary analytical inputs and outputs  How to.
Agenda: Block Watch: Random Assignment, Outcomes, and indicators Issues in Impact and Random Assignment: Youth Transition Demonstration –Who is randomized?
Grade 3-8 English. 2 The Bottom Line This is the first year in which students took State tests in Grades 3,4,5,6,7, and 8. With the new individual.
NEARLY 50% OF OUR KIDS ENTER KINDERGARTEN ALREADY BEHIND COST: $860 MILLION MINNESOTA’S PROBLEM:
Pascaline Dupas, UCLA Pupil-Teacher Ratios, Locally-Hired Contract Teachers, and School-Based Management: Evidence from Kenya Making Schools Accountable:
School Readiness Initiatives: Assessing Their Yield Craig T. Ramey, Ph.D. & Sharon L. Ramey, Ph.D. Georgetown University Center on Health and Education.
The Long Reach of Early Math Skills Greg J. Duncan University of California, Irvine Robert Siegler Carnegie Mellon University.
Community Input Discussions: Measuring the Progress of Young Children in Massachusetts August 2009.
The Foundation for a Thriving New Mexico Economy: Success Begins at Birth Early Childhood Action Network 2008.
March 2010 what the school readiness data mean for Harford County’s children ©
Longitudinal Studies The ultimate Goal for many extension faculty is to implement programs that result in permanent, positive change in human social behavior.
1 Economics of UPK Benefits and Costs of Quality Early Childhood Education for All Presentation at a Conference on The Law and Policy of Universal Preschool.
A Benefit Cost Analysis of Early Childhood Intervention By Jenny Yaillen And Nate Blair.
Ramey & Ramey (1998) Early Intervention: activities designed to enhance a young child’s development Initial evaluation of child’s abilities and needs (in.
Early Childhood Education The Research Evidence Deborah Lowe Vandell December 11, 2003.
Economics of Investments in Early Childhood Development Paris, France June 22, 2010 Steve Barnett, PhD.
Education and Policy aimed at Early Childhood. Education and Early Childhood Policy We have seen throughout this class that poverty and racial inequality.
Reclassification of English Learner Students in California
NHSA Update NHSA September 2006 Leadership Institute Arlington, Virginia Research And Assessment Update 2007 Ben Allen, Ph.D. NHSA Leadership Institute.
Children Entering School Ready to Learn The Maryland School Readiness Report what the school readiness data mean for Maryland’s children.
GEORGE L. ASKEW, MD, FAAP OFFICE OF THE ASSISTANT SECRETARY ADMINISTRATION FOR CHILDREN AND FAMILIES U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES AMERICAN.
Abecedarian Project. Problems With Prior Research few early childhood programs have been sufficiently well controlled to permit scientists to evaluate.
1 Economics of Early Education Benefits and Costs of Quality Early Education for All Presentation to the Pre-Kindergarten Education Study Committee Vermont.
Financing Early Education Why does early education need more public funding? K Early education is an essential investment K Too few children have access.
Benefit-Cost Analysis of Preschool Education W. Steven Barnett, Ph.D. Director, National Institute for Early Education Research Rutgers—The State University.
Universal Preschool The Promise and the Peril The 16 th Annual Early Childhood Iowa Congress Des Moines, Iowa February 10, 2009 Walter S. Gilliam, PhD.
NEARLY 50% OF OUR KIDS ENTER KINDERGARTEN ALREADY BEHIND COST: $860 MILLION MINNESOTA’S PROBLEM:
New Research and the Pay-Off to Quality Early Care and Education Atlanta, GA September 23, 2013 Steve Barnett, PhD.
What policy makers should know about Pre-K effectiveness Steve Barnett, PhD
Financing Early Education Presentation to Governors Forum on Quality Preschool December 15-16, 2003 W. Steven Barnett, Ph.D. National Institute for Early.
Evaluation of the DC Opportunity Scholarship Program: Final Report Conducted by Westat, University of Arkansas, Chesapeake Research Associates Presented.
College of Education Helping Schools Evaluate Needs and Select Best Practices in Childhood Mental Health.
Assessment at KS4 Bury C of E High School Engaging Parents Information.
Minnesota's Approach to Comprehensive Assessment Megan E. Cox, Ph.D. Principal Leadership Academy January 11, 2016 Minnesota’s Approach to Comprehensive.
Considerations Related to Setting Targets for Child Outcomes.
1 Developing State Specific Evidence on the Costs and Benefits of Early Childhood Education (ECE) Kenneth Troske, Director Center for Business and Economic.
1 July 24, 2009Latest Research on Head Start and Other Early Learning Programs Ben Allen, Ph.D. Washington State Assn. of Head Start & ECEAP/Washington.
Comments on: ”Educating Children of Immigrants: Closing the Gap in Norwegian Schools” The Nordic Economic Policy Review Conference 2011 Lena Nekby Department.
1 March 3, 2010HS and EHS Research Update Research Demonstrating Head Start Works Ben Allen, Ph.D. National Head Start Association Pennsylvania Head Start.
A review of academic research.  The ways parent education programs shapes their children’s well- being are multifaceted and complex  We summarize the.
Chronic Absence in the Early Grades Jane Quinn, Director Abe Fernández, Deputy Director November 8, 2010 | Portland, OR.
Children Entering School Ready to Learn The Maryland School Readiness Report what the school readiness data mean for Maryland’s children.
NUMBERS TO REMEMBER ABOUT EARLY CHILDHOOD DEVELOPMENT PER SECOND MONTHS 90– % DOLLARS This feature highlights numbers to.
Our State. Our Students. Our Success. DRAFT. Nevada Department of Education Goals Goal 1 All students are proficient in reading by the end of 3 rd grade.
IMPACT EVALUATION PBAF 526 Class 5, October 31, 2011.
EXPLORE NEW WORLDS The possibilities are infinite.
Performance and Progress 2012/2013. Why We Do an Annual Data Presentation To assess the Levy’s performance in various categories against goals. To highlight.
TCAI: Lessons from first Endline TCAI Development Partners Feb 27, 2013.
Introduction: “Please write down everything that comes into your head when you hear the words ‘DNA risk tests’” Colorectal cancer Type 1 diabetes Information.
Portland State University
Brad Neuenswander & Tony Moss September 2014
What are the SATS tests? The end of KS2 assessments are sometimes informally referred to as ‘SATS’. SATS week across the country begins on 14th May 2018.
Student Equity Planning August 28, rd Meeting
Inequality Starts Before Kindergarten
Early Childhood Education
Presentation transcript:

Positive Impacts and Return on Investment: What We Can Learn from Head Start Research Hilary Shager WI Department of Children and Families April 25, 2012

Overview Being a good consumer of Head Start research Early cohort research The National Head Start Impact Study Early Head Start Research What’s new in Head Start Research Remaining research questions

“...finally there is indisputable evidence about the program's effectiveness, provided by the Department of Health and Human Services: Head Start simply does not work.” -Joe Klein, Time Magazine, July 2011

“Head Start improves the odds and the options for at-risk kids for a lifetime. Kids that have been through Head Start and Early Head Start are healthier, more academically accomplished, more likely to be employed, commit fewer crimes, and contribute more to society. Simply put: Head Start works. It’s been proven.” -Letter to Congress, signed by over 270 top early childhood researchers March, 2011

Being a good consumer of Head Start research Lots of Head Start research Lots of rhetoric –Which research is good research? –What do the results tell us about Head Start’s effectiveness? –What are the caveats? Bottom line… –There is a lot to be excited about!

Challenges in Head Start Research Lots of things affect educational, health, family outcomes –Hard to isolate the “true” impact of Head Start programming Finding the appropriate control group –Head Start targeted at particularly disadvantaged children and families

Questions to ask 1)Is the evaluation design strong enough to produce trustworthy evidence? 2)What program services were actually received by participating children and families in the comparison group? 3)How big an impact did the program have? 4)Do the program’s benefits exceed its costs? 5)How similar are the programs, children, and families in the study to those in my community?

Research 101: Research Design Gold Standard=Experiment –Randomized controlled trial Quasi-experimental approaches –Does the control group really look like the Head Start population? –What services do children and families in the control group get? External validity –If I reproduced the study in my community, would I get the same results?

Research 101: Effect Size What is an effect size? –Measure of “magnitude” of impact The bigger, the better! More meaningful than “statistically significant” –Fraction of the difference between the treatment and control group over variation Allows comparison across different measures Need this to do a cost/benefit analysis

Early cohort research: Some history The Westinghouse study (1968) –Quasi-experimental approach –Focus on cognitive outcomes only –Almost defunds Head Start –Consortium of researchers rallies against the study We need better research on more outcomes! We waited a long time...

Janet Currie and colleagues: Sibling fixed effect studies (1990s) Use nationally representative data sets Include families in which one sibling went to Head Start and one didn’t Teases out all the “fixed” family- related factors that influence outcomes Still questions about why one sibling attended and not the other

Currie et al. findings Large gains in vocabulary test scores –Equal to about 1/3 of achievement gap –Effects persist into adulthood for whites, but fade quickly for African-Americans Reduced probability of retention (47%) –Again, only for whites More likely to be immunized (8-11%) –Both whites and African-Americans Cost-benefit implications –Increased test scores would increase expected future wage by 4% –Reduction in retention would lead to 5% decline in probability of high school drop-out –Preventative health savings –Definitely passes test for whites, question for African-Americans

More Currie et al. findings Educational attainment (whites only) –22 percentage points more likely to complete high school –19 percentage points more likely to attend some college Less likely to be arrested or charged for crime –12 percentage points, African-Americans only Positive effects for Hispanic children –Closes 1/4 gap in test scores –Closes 2/3 gap in probability of grade repetition Maybe effects less, fade for African-American children because they attend lower quality schools later

Ludwig & Miller (2007) 50 to 100% increase in Head Start funding in 1960s and 70s associated with –Increase of schooling attainment of about 1/2 year –15% increase in likelihood of attending some college –Found for both African-Americans and whites –Decrease in infant mortality Benefit/cost ratio as large as 7 to 1 –Decline in mortality $120 per 4-year-old –Extra half year of schooling increases earnings 2% or $15,000 –Even higher if consider increased schooling reduces crime

Deming, 2009 Uses same method as Currie et al., but more recent cohort Effects on math and reading that fade –0.15 at age 5-6 –0.13 at age 7-10 –0.06 at age –Larger for African-Americans at first, but fade more quickly Larger positive effects on long-term outcomes –18.6 percentage points more likely to complete high school (African-Americans) –“Index” of long-term outcomes, 0.23 Mechanism = less retention, special education

Hard to generalize these findings to today More working parents More early childhood programming options Decrease in achievement gaps Increase in Head Start quality

The National Head Start Impact Study (NHSIS) Randomized controlled trial required by Congress in 1990s Nationally representative sample –84 Head Start programs –Approximately 4,600 children –Begin attending in 2002 Follow kids through end of 1 st grade

First year findings: 4-year-olds 7 positive cognitive effect sizes –Effect sizes = 0.16 to 0.31 More likely to visit the dentist –(73.2% vs. 56.9%) Parents more likely to read to children –(3x/wk vs. 2.8x/wk) No significant findings for more advanced reading skills, math, or behavior problems

First year findings: 3-year-olds Same positive impacts as 4-year-olds, but bigger, plus... –Math –Fewer problem behaviors –Parents spanked children less –Overall health Still no effects on more advanced reading skills

Lots of Caveats “Intent to treat” analysis –17% of 3-year-olds and 14% of 4-year-olds never got Head Start services Crossovers –18% of 3-year-olds and 14% of 4-year-olds in control group attended Head Start 60% of control kids in other programs Evidence of circumventing randomization –Center directors enroll those with highest needs

Ludwig & Phillips re-analysis “Treatment on the treated” (TOT) –Adjust for crossovers and no-shows –Larger effect sizes (0.14 to 0.50) Pool together 3- and 4-year-olds –Larger sample, more “power” –Statistically significant impacts for nearly all main cognitive outcomes

First grade follow-up Significant effects from first year mostly disappear –For 4-year-olds, only impact on receptive vocabulary remains (0.09) –For 3-year-olds, only impact on oral comprehension remains (0.08)

Why we shouldn’t compare these results to pre-k study results Pre-k studies use different research design that is not as strong as NHSIS Fewer kids in pre-k control groups attend other programs Different populations Pre-k only looks at academic outcomes No long-term follow-up of pre-k (therefore, no info about “fade out”)

So, what does this study really tell us? Impacts represent short-term effects of offer of one year of Head Start Measures impact of Head Start versus everything else available Initial impacts found are about the same size as those found for children attending Head Start in the 1960s-1980s For those cohorts, Head Start seems to have produced long-term benefits large enough to outweigh program costs, despite fade-out of initial test score impacts (nothing like that measured here) But test score impacts do seem to fade more quickly in new cohort—not sure why or whether we should worry about this

Does Head Start pass the benefit/cost test today? Ludwig & Phillips (2008) –Cost = $9,000 per child –Head Start might pass a benefit-cost test if short-term effects on achievement test scores were as small as 0.1 to 0.2

Early Head Start research Randomized controlled trial Follow through 5 th grade Positive impacts at age 3 –Cognitive and language development –Socio-emotional development –Home environment and parenting behavior –Parents’ progress toward self-sufficiency –Positive impacts specifically for fathers “Sleeper” effects Differences across different service models

Early Head Start Home Visiting Evidence

What’s new in Head Start research? Focus on “quality” Curriculum enhancements Professional development Early Head Start and home visiting Head Start Family and Child Experiences Survey (FACES) –New! “Baby FACES”

Meta-analysis (Shager et al., 2012) Study of 33 Head Start evaluations Overall, positive effect of Head Start on cognitive and achievement outcomes (effect size = 0.27) What happens in the control group matters a lot! –Effect sizes for studies that have an “active” control group are 0.33 smaller –Type of skill, measures used, and where published also matter

Conclusions Some high quality Head Start research exists, but you need to be an alert consumer Head Start and Early Head Start have had positive impacts on children and families across a variety of domains Head Start appears to pass the benefit/cost test We still have a lot to learn about how and why early childhood programs improve long-term outcomes

Remaining questions Instead of asking whether Head Start is effective at all, we must ask how effective Head Start is compared to the range of other options available. What happens to kids after they attend Head Start? –Do effects really “fade out,” or do other kids “catch up”? Which behavioral and socio-emotional skills are important, and how can Head Start foster those? Other benefits? –Health, parenting, family self-sufficiency

What are your questions?