quality George N. Noell, Ph.D. Louisiana State University and A&M College & Louisiana Department of Education Jeanne M. Burns, Ph.D. Louisiana Board of Regents Office of Special Education Programs Annual Directors’ Confere nce July 18, 2011 USING THE LOUISIANA DATA LONGITUDINAL SYSTEM TO IMPROVE TEACHER PREPARATION

quality What has Louisiana learned about teacher effectiveness and data systems (2000 – 2011)? Teacher preparation IS important

quality... especially when examining the achievement of exceptional students.

quality How does Louisiana know this is true? Louisiana can now link teacher preparation programs to the new special education and regular teachers they prepared to the students that the new teachers taught Louisiana can provide teacher preparation programs with relevant data pertaining to their effectiveness in preparing new special education and regular teachers  (October 2006) By content areas (Reading, Mathematics, Science, Social Studies, English/language arts)  (March 2011) By grade spans within content areas (Grades 1-5, Grades 4-8, Grades 6-12, & Special Education)  (Fall 2011) By student performance subsets within content areas (Low, Middle, and High)  (Fall 2011) By individual teacher actual student achievement, predicted student achievement, and content standards breakdowns for achievement tests

quality Why Louisiana? Louisiana has had the political will (1999 – Present)  Blue Ribbon Commission for Educational Excellence (1999 – Present)  Commissioners of Higher Education (2000-Present) (Current: Dr. Jim Purcell; Past: Dr. Sally Clausen & Dr. E. Joseph Savoie)  State Superintendents (2000-Present) (Current Acting: Ollie Tyler; Past: Paul Pastorek & Cecil Picard)  Governors (2000 – 2011) (Current: Bobby Jindal; Past: Kathleen Blanco & Mike Foster) Louisiana has had researchers who possess the knowledge and expertise to create a Value-Added Teacher Preparation Assessment and change agents at the State level (Louisiana Board of Regents and Louisiana Department of Education) to implement and support the use of the assessment

quality Four Levels of Effectiveness for Teacher Preparation Programs in Louisiana Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs) Level 2: Effectiveness of Implementation (NCATE/TEAC – Comprehensive Assessment System) Level 3: Effectiveness of Impact (Teacher Preparation Accountability System) Level 4: Effectiveness of Growth in Student Learning (Value Added Teacher Preparation Program Assessment)

quality Redesign of Special Education Programs Guidelines and Evaluation – Redesign of Teacher Preparation Programs  October 1, 2001 – June 30, 2003 (New Programs: July 1, 2003)  Baccalaureate and Alternate: Grades PK-3, 1-5, 4-8, 6-12, and Mild/Moderate Special Education (Grades 1-12)  Evaluation by National Experts Guidelines and Evaluation - Redesign of Special Education Programs  April 20, 2005 – December 30, 2006 (New Programs: Jan. 1, 2007)  Baccalaureate: Early Interventionist (Birth – Five Years) (Note: Special Education Mild/Moderate Blended Program Withdrawn due to problems with structure)  Alternate : Early Interventionist (Birth – Five Years); Deaf/Hard of Hearing (Grades K-12); Visual Impairments/Blind (Grades K-12); and Significant Disabilities (Grades 1-12);  Advanced Masters – Add on Certification: Deaf/Hard of Hearing; Educational Diagnostician; Gifted; Inclusive Practices; Instructional and Assistive Technology; Learning Disabilities; Mild/Moderate Special Education; Significant Disabilities; and Visual Impairments  Evaluation by National Experts

quality Redesign of Integrated to Merged Special Education Programs Guidelines & Evaluation - Redesign of General-Special Education Mild/Moderate: An Integrated to Merged Approach for Grades 1-5, Grades 4-8, & Grades 6-12  August 23, 2009 – June 30, 2010 (New Programs: July 1, 2010)  Baccalaureate: General-Special Education Mild/Moderate: An Integrated to Merged Approach (Grades 1-5; Grades 4-8; and Grades 6-12)  Alternate: General-Special Education Mild/Moderate: An Integrated to Merged Approach (Grades 1-5; Grades 4-8; and Grades 6-12)  All teacher preparation programs were asked to use the document Collaborative Programs in General and Special Teacher Education: An Action Guide for Higher Education and State Policymakers (2007) that was developed by Linda Blanton and Marleen C. Pugach to guide them as they developed their programs  Teams from universities met with national experts at statewide meetings sponsored by the LA Department of Education to develop a better understanding of integrated to merged approaches  Evaluation by National Experts

quality Four Levels of Effectiveness for Teacher Preparation Programs in Louisiana Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs) Level 2: Effectiveness of Implementation (NCATE/TEAC – Comprehensive Assessment System) Level 3: Effectiveness of Impact (Teacher Preparation Accountability System) Level 4: Effectiveness of Growth in Student Learning (Value Added Teacher Preparation Program Assessment)

quality Challenges in Assessing Teacher Preparation The challenge of measures –Achievement versus opinions Geography Heterogeneous schools & classes Data management Technical issues The plausible counter factual (Rubin)

quality Value-Added Teacher Preparation Assessment Model Predict achievement of individual students based on prior achievement, demographics, and attendance Assess actual student achievement Link growth of student achievement to new teachers and teacher preparation programs that taught the new teachers Calculate degree to which students taught by new teachers met achievement of similar students taught by experienced teachers Act on results

quality Nesting Structure of Students with Teachers Within Schools

quality Prior Achievement Demographics Expected Achievement Strong Result Average outcome 300 Weak Result 290 Illustration of the Impact of Teacher Preparation on Average Student Test Performance Actual Achievement

quality Tests and Teachers Tests - State Achievement Tests (Math, Science, Social Studies, Reading, & Language Arts - Grades 3-9) New Teachers: – 1 st and 2 nd year teachers with teaching certificates – Teaching within area of certification Experienced Teachers –3 rd or subsequent year teachers with teaching certificates – Teaching within area of certification

quality Criteria for Inclusion of Programs in the Assessment Inclusion for each content area –Redesigned programs only –25 or more new teachers in grades 4-9 –Teaching within certification –Remained with student full academic year 10 universities and 2 private providers are included in the results 9 universities lacked a sufficient number of new teachers in the content areas to be included – they will be included in the future once they meet the criteria for inclusion

quality Value-Added Teacher Preparation Assessment Results EXAMPLE: NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM ( Data) Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers Science Language Arts ReadingMath Social Studies Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Example: +3.3 Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers. Level 3: Growth in achievement COMPARABLE to students taught by NEW teachers. Level 4: Growth in achievement BELOW students taught by other NEW teachers. Level 5: Growth in student achievement SIGNIFICANTLY BELOW students taught by other NEW teachers.

quality State Research Teams and Universities Using Data to Create Effective New Teachers Dr. Vickie Gentry, Dean College of Education and Human Development Northwestern State University Dr. Gerald Carlson, Dean College of Education University of Louisiana at Lafayette

quality Longitudinal Assessment Results NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM SCIENCE Amount of Growth in Achievement Results Results Results Results Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers.

quality Longitudinal Assessment Results NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM LANGUAGE ARTS READING Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers Results Results Results Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers. Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers Results Results Results Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers.

quality Longitudinal Assessment Results NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM SOCIAL STUDIES Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers Results Results Results Results Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers. Level 3: Growth in achievement COMPARABLE to students taught by NEW teachers. Level 4: Growth in achievement BELOW students taught by other NEW teachers. Level 5: Growth in student achievement SIGNIFICANTLY BELOW students taught by other NEW teachers.

quality Example of Longitudinal Assessment Results NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM MATHEMATICS Amount of Growth in Achievement of Grades 4- 9 Students Taught by New Teachers Results Results Results Results Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers. Level 3: Growth in achievement COMPARABLE to students taught by NEW teachers. Level 4: Growth in achievement BELOW students taught by other NEW teachers. Level 5: Growth in student achievement SIGNIFICANTLY BELOW students taught by other NEW teachers.

quality Example of Grade Span Results ( Report) NORTHWESTERN STATE UNIVERSITY ALTERNATE CERTIFICATION PROGRAM MATHEMATICS Mean Effect Estimate for All New Teachers for Mathematics = -3.1 Grade SpansNEffect Estimates Elementary (Grades 1-5) Middle School (Grades 4-8)7 0.4 Secondary (Grades 6-12) Special Education Mean Effect Estimate for Northwestern State University for Mathematics (Level 3 Performance Level) -1.5 CURRENT QUESTION: Are students of new teachers performing equally as well in all mathematical content standard areas on the achievement test (e.g., Number and number relations; algebra; measurement; geometry; data analysis, probability, & discrete math; and patterns, relations, & functions)? Strength

quality Value-Added Teacher Preparation Assessment Results EXAMPLE: UNIVERSITY OF LOUISIANA AT LAFAYETTE ALTERNATE CERTIFICATION PROGRAM ( Data) Amount of Growth in Achievement of Grades 4-9 Students Taught by New Teachers Science Language Arts ReadingMath Social Studies Level 1: Growth in achievement GREATER than students taught by EXPERIENCED teachers. Level 2: Growth in achievement COMPARABLE to students taught by EXPERIENCED teachers. Level 3: Growth in achievement COMPARABLE to students taught by NEW teachers. Level 4: Growth in achievement BELOW students taught by other NEW teachers. Level 5: Growth in student achievement SIGNIFICANTLY BELOW students taught by other NEW teachers.

quality Example of Grade Span Results ( Report) UNIVERSITY OF LOUISIANA AT LAFAYETTE ALTERNATE CERTIFICATION PROGRAM LANGUAGE ARTS Mean Effect Estimate for All New Teachers for Language Arts = -2.7 Grade SpansNEffect Estimates Elementary (Grades 1-5) Middle School (Grades 4-8) Secondary (Grades 6-12) Special Education9 7.4 Mean Effect Estimate for the University of Louisiana at Lafayette for Language Arts (Level 4 Performance Level) -5.1 Strength

quality Example of Content & Achievement Test Breakdown UNIVERSITY OF LOUISIANA AT LAFAYETTE ALTERNATE CERTIFICATION PROGRAM LANGUAGE ARTS (Grade 4 Achievement Test) Levels for State Achievement Tests Average Actual Achievement Levels of Students in Teachers’ Classrooms Number of New Teachers Whose Actual Student Achievement Scores were at or above the Predicted Student Achievement Scores Unsatisfactory 3%0 out of 1 new teachers Approaching Basic 26% 2 out of 8 new teachers Basic 68%12 out of 21 new teachers Mastery (Proficient) 3%0 out of 1 new teachers Advanced out of 31 new teachers (45%) met their predicted student growth targets for the Grade 4 achievement test.

quality Example of Content, Achievement Test, & Content Standard Breakdown UNIVERSITY OF LOUISIANA AT LAFAYETTE ALTERNATE CERTIFICATION PROGRAM LANGUAGE ARTS (GRADE 4) Description Write competently Locate, select, and synthesize information Percentage of new teachers whose students scored at or above the state mean on items addressing specific state content standards for language arts on the Grade 4 achievement test 10% 61% DATA DRIVEN FINDING BY UL-L: Writing competently is a need of students taught by new teachers from UL-L.

quality Considerations in Making Assessment of Teacher Preparation Programs Work Data quality issues & what do the data mean Who taught vs. who tested the student Modeling issues: who, how, and where? Is it complicated enough yet or too complicated? The time to action in the information feedback loop

quality George N. Noell Jeanne M. Burns What do you need to do to make this happen in your states?