Using State Longitudinal Data Systems for Education Policy Research : The NC Experience Helen F. Ladd CALDER and Duke University Caldercenter.org

Slides:



Advertisements
Similar presentations
Value Added in CPS. What is value added? A measure of the contribution of schooling to student performance Uses statistical techniques to isolate the.
Advertisements

Understanding the North Carolina School Report Card.
Jamesville-DeWitt School Report Card Presented to the Board of Education May 10, 2010.
Why Take PLAN? PLAN shows your strengths and weaknesses in English, mathematics, reading, and science. PLAN lets you know if you’re on target for college.
Developmental Education in North Carolina Community Colleges Charles T. Clotfelter Helen F. Ladd* Clara Muschkin Jacob L. Vigdor Sanford School of Public.
Miami ISD Texas Academic Performance Report (TAPR)
Teacher Training, Teacher Quality and Student Achievement Douglas Harris Tim R. Sass Dept. of Educational Dept. of Economics Policy Studies Florida State.
Challenge to Lead Southern Regional Education Board Kentucky Challenge to Lead Goals for Education Kentucky is On the Move Progress Report 2008 Challenge.
Challenge to Lead Southern Regional Education Board Tennessee Challenge to Lead Goals for Education Tennessee is On the Move Progress Report 2008 Challenge.
SARAH FULLER HELEN LADD DUKE UNIVERSITY SANFORD SCHOOL OF PUBLIC POLICY School Based Accountability and the Distribution of Teacher Quality Across Grades.
Teacher Effectiveness in Urban Schools Richard Buddin & Gema Zamarro IES Research Conference, June 2010.
Continuum of Teacher Development and Shared Accountability Leading to Increased Student Performance Teaching Quality Policy Center Education Commission.
Senate Bill 1908 Beginning in the school year, 50% of the school’s grade will be based on the existing FCAT- related factors and the remaining.
School Report Cards 2004– The Bottom Line More schools are making Adequate Yearly Progress. Fewer students show serious academic problems (Level.
Teacher Credentials and Student Achievement in High School: A Cross Subject Analysis with Student Fixed Effects Charles T. Clotfelter Helen F. Ladd Jacob.
Explaining Race Differences in Student Behavior: The Relative Contribution of Student, Peer, and School Characteristics Clara G. Muschkin* and Audrey N.
KIPP: Effectiveness and Innovation in Publicly-Funded, Privately-Operated Schools October 4, 2012 Presentation to the APPAM/INVALSI Improving Education.
Informing Policy: State Longitudinal Data Systems Jane Hannaway, Director The Urban Institute CALDER
Data for Student Success Comprehensive Needs Assessment Report “It is about focusing on building a culture of quality data through professional development.
Staar Trek The Next Generation STAAR Trek: The Next Generation Performance Standards.
What Makes For a Good Teacher and Who Can Tell? Douglas N. Harris Tim R. Sass Dept. of Ed. Policy Studies Dept. of Economics Univ. of Wisconsin Florida.
Value-added Accountability for Achievement in Minneapolis Schools and Classrooms Minneapolis Public Schools December,
AN EVALUATION OF THE EIGHTH GRADE ALGEBRA PROGRAM IN GRAND BLANC COMMUNITY SCHOOLS 8 th Grade Algebra 1A.
Different Skills? Identifying Differentially Effective Teachers of English Language Learners Ben Master, Susanna Loeb, Camille Whitney, James Wyckoff 5.
-- Preliminary, Do Not Quote Without Permission -- VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY Douglas HarrisTim R. Sass Dept. of Ed. LeadershipDept.
The State of Texas Assessments of Academic Readiness (STAAR): What You Need to Know Feb. 16, 2012 HOUSTON INDEPENDENT SCHOOL DISTRICT.
Illinois State Assessment Sarah Jones April 16, 2002 Educational Psychology 311.
9/6/2015 Choices for Studying Choice Dev Davis Macke Raymond.
Including a detailed description of the Colorado Growth Model 1.
The Narrowing Gap in NYC Teacher Qualifications and its Implications for Student Achievement Don Boyd, Hamp Lankford, Susanna Loeb, Jonah Rockoff, & Jim.
Measuring the Achievement Effects of Charter Schools in North Carolina Helen F. Ladd (Duke) Presentation for SREE conference Based on joint research with.
Human Capital Policies in Education: Further Research on Teachers and Principals 5 rd Annual CALDER Conference January 27 th, 2012.
Arizona’s Federal Accountability System 2011 David McNeil Director of Assessment, Accountability and Research.
Student Engagement Survey Results and Analysis June 2011.
Challenge to Lead Southern Regional Education Board Georgia Goals for Education Challenge to Lead: Georgia 2006 Challenge to Lead Southern Regional Education.
Kentucky’s New Assessment and Accountability Model June 2011.
Equity Scorecard Update A presentation to the Madison College Board of Trustees February 09, 2011.
SEDL’s Resource Allocation Studies Examination of Resource Allocation in Education: Connecting Spending to Student Performance States in study:
1 Why should MSRI be involved in education, and how?
Mathematics Indicators and Goals. Math Tier II Indicator Indicator 1.8: All junior high students will meet or exceed standards and be identified as proficient.
Evaluating the Vermont Mathematics Initiative (VMI) in a Value Added Context H. ‘Bud’ Meyers, Ph.D. College of Education and Social Services University.
Special Education Teacher Quality and Student Achievement Li Feng Tim R. Sass Dept. of Finance & Econ.Dept. of Economics Texas State UniversityFlorida.
Developing an Effective Teacher Education System.
RESEARCH ON MEASURING TEACHING EFFECTIVENESS Roxanne Stansbury EDU 250.
State Charter Schools Commission of Georgia SCSC Academic Accountability Update State Charter School Performance
ACCOUNTABILITY UPDATE Accountability Services.
Drake 274.  Interests and Preferences for Postsecondary Expectations  Definition: Interests are those things/activities that evoke curiosity and preferences.
Release of PARCC Student Results. By the end of this presentation, parents will be able to: Identify components of the PARCC English.
1 Superintendents’ Quarterly Meeting A Next Generation Accountability Model March 25, 2010.
© CCSR ccsr.uchicago.edu. © CCSR Early Warning Indicators of High School Graduation and Dropout Elaine Allensworth.
Impediments to the estimation of teacher value added Steven Rivkin Jun Ishii April 2008.
Strategies for estimating the effects of teacher credentials Helen F. Ladd Based on joint work with Charles Clotfelter and Jacob Vigdor CALDER Conference,
DVAS Training Find out how Battelle for Kids can help Presentation Outcomes Learn rationale for value-added progress measures Receive conceptual.
1 Monroe County School District Spending vs. Student Achievement John R. Dick School Board District 4.
PED School Grade Reports (with thanks to Valley High School) ACE August 3, 2012 Dr. Russ Romans District Accountability Manager.
2015 State PARCC Results Presented to the Board of Elementary and Secondary Education Robert Lee MCAS Chief Analyst and Acting PARCC Coordinator October.
Kingsville ISD Annual Report Public Hearing.
Challenge to Lead Southern Regional Education Board Arkansas Goals for Education Challenge to Lead: Arkansas 2006 Challenge to Lead Southern Regional Education.
MSRI CONFERENCE: THE MATHEMATICAL EDUCATION OF TEACHERS MAY 12, 2011 ZALMAN USISKIN THE UNIVRSITY OF CHICAGO Curriculum and Teacher.
Title I Annual Meeting What Every Family Needs to Know!
School Accountability and Grades Division of Teaching and Learning January 20, 2016.
© 2014, Florida Department of Education. All Rights Reserved. Accountability Update School Grades Technical Assistance Meeting.
Annual Report Georgetown ISD 2016 Accountability Rating:
Release of PARCC Student Results
Texas Academic Performance Report TAPR)
Understanding Your PSAT/NMSQT® Results
Dan Goldhaber1,2, Vanessa Quince2, and Roddy Theobald1
Understanding Your PSAT/NMSQT® Results
Challenge to Lead: Texas
Annual Report Public Hearing
Presentation transcript:

Using State Longitudinal Data Systems for Education Policy Research : The NC Experience Helen F. Ladd CALDER and Duke University Caldercenter.org

Examples of research in NC by CALDER researchers Charter schools (Bifulco and Ladd) –How charter schools affect student achievement, achievement gaps and racial sorting. Achievement gaps (Clotfelter, Ladd and Vigdor) –By student cohort grades 3-8; by race. Distribution and movement of teachers ( Clotfelter, Ladd Vigdor and Wheeler ) –Descriptive with focus on high poverty schools Public policies and the distribution of teachers ( Clotfelter, Ladd, Vigdor and Glennie ) –Salaries, alternative salaries and working conditions –$1800 bonus program program for teachers of math, and special education in low performing middle and high schools –The state’s accountability system

NC research (cont) Teacher credentials and student achievement (Clotfelter, Ladd, and Vigdor) –Cross sectional analysis – 5 th graders (JHR, 2006) –Longitudinal analysis – grades 3-5. (CALDER web page and shorter version forthcoming in EER) –Cross subject analysis – high school. Work in progress

Strengths of NC Data Student test scores –Based on tests that are linked to the state’s standard course of study Grades 3-8 End-of-Grade (EOG) tests in math and reading High School End-of-Course (EOC) tests in multiple subjects – e.g. algebra I, English I, biology, –Available since the mid 1990s. –Test scores can be linked by student over time

Strengths (cont.) Other student data –Standard demographic data Race, gender, free lunch status, LEP –Education level of the parents (Some concerns) –Survey responses (connected to the test) E.g. homework, use of computer, TV watching – Student addresses in some districts and in some years

Strengths (cont.) Teachers –Wide array of credentials –Teacher licensure test scores –Licensure – regular, lateral entry, other –National board certification (NC has high percentage) –Graduate degrees and when they got them –Undergraduate college –Certification by subject –Teacher salary data –Can follow teachers over time

Limitations of NC data Linking of student to their specific teacher is possible but imperfect. Grades 3-5. About 85 percent match Grades 6-8. Poor match High school- courses with EOC tests Good match – 75 percent Incomplete home address data – relevant for choice and charter school studies No link yet with higher education data

Basic “value-added” model of student achievement ( grades 3-5) A student’s achievement in year t is a function of: her achievement in the previous year (accounts for the cumulative nature of achievement) teacher characteristics and credentials (e.g. gender, years of experience, teacher test score) classroom characteristics (e.g. class size, profile of students in the class) student characteristics (e.g. race, gender, poverty status)

Achievement models 1-3 Achievement levels (A it ) or achievement gains (A it - A i,t-1 ) No fixed effects 1.Levels. Upward biased coefficients because of teacher- student sorting; potential bias from lagged achievement on RHS. With school fixed effects 2. Levels. Better, but problem of sorting within schools remains and potential bias from lagged achievement; direction of bias unclear (see our JHR paper) 3. Gains. Downward bias from misspecified persistence effect.

Models 4 and 5 (preferred) Full use of the longitudinal aspect of the data With student fixed effects 4. Levels (but no lagged achievement). Lower bound estimates of effects of teacher credentials 5. Gains. Upward bound estimates of effects of teacher credentials

Interpretation of results All test scores are normalized to have a mean of 0 and a standard deviation of 1. => coefficients of interest will be small. E.g Point of reference. Compare test score of a typical student whose parent has a high school degree but no college degree to the test score of a student whose parents are college educated. Estimated effect size = Question : are the effects of teacher credentials on student achievement large enough to counter this negative effect of relatively low parental education?

Effects of credentials: Teacher experience Coefficients from preferred model (model 5) All are statistically significant Base= no experience MathReading 1-2 years years years years years

Other teacher characteristics and credentials ( math only, preferred model) Teacher test score – linear form Quality of undergraduate institution Competitive (Base = noncompetitive) License Other license (Base = regular license) (Note: all coefficients are statistically significant)

Teacher credentials Master’s degree (math: preferred model) Master’s degree * unexpected Disaggregated specification MA before teaching MA 1-5 years into teaching MA 5+ years into teaching * * indicates that the coefficient is statistically significant at the 0.05 level..

Teacher credentials National Board Certification (math: preferred model NBCT NBCT NBCTcurr0.046 NBCTpost 0.041

Are these effects large or small? Teachers have bundles of characteristics Comparison of teacher with average/strong credentials compared to one with weak credentials math to 0.20 reading to 0.12 Magnitudes are large relative to the estimated effects of reducing class size by 5 students. Sufficient to offset much of the effect of weak parental education – particularly for math.

Teacher credentials at the high school level – work in progress Use NC EOC test scores – with students linked to their teachers by subject Same problem of teacher and student sorting as at the elementary level Additional challenge – students self select into courses

Our approach – cross-subject model with student fixed effects Start with all students in grade 10 in four different cohorts ( ) Focus on the subjects typically taken in 9 th or 10 th grade (algebra 1, English 1, biology, geometry, and ELP)– but include all test scores in whatever grade the course (and test) was taken Define unit of observation as the student by subject Include student fixed effects – to control for unmeasurable characteristics of students such as their ability

Preliminary illustrative results: Teacher credentials matter Teacher experience -- mainly first 2 years Teacher licensure -- negative effect for lateral entry teachers Certification by subject -- Teachers certified in subject > those in related subject> than other certified teachers NBCT – more effective pre-NBCT; even more effective after certification. Teacher test scores. Similar to elementary school. And more – to come. Stay tuned and check the CALDER web page later this summer.