The University of Texas System Assessment of Teacher Preparation Programs Funded by the Houston Endowment September, 2004 9/18/2018

Overview of Presentation National Center for Educational Accountability Introduction to the study Research questions Indicators Data sources Data security and confidentiality Teacher Survey Data collection (Mary Lummus-Robinson) School district partnership (Holly Williams) Questions & discussion 9/18/2018

National Center for Educational Accountability Created in 2001 Sponsoring organizations Education Commission of the United States Just for the Kids The University of Texas at Austin www.nc4ea.org 9/18/2018

Introduction to the study Purpose: To identify relatively high performing teacher certification programs in the UT System and to determine what makes these programs so successful and what staff from other programs can learn from them. 9/18/2018

Introduction to the study UT System requested development of a system for evaluating teacher preparation programs 4-year study of graduates from 9 UT System components 4 cohorts of graduates beginning in 2002-2003 teaching in Texas public schools grades 4-8 grades 9-11 on an exploratory basis 9/18/2018

Introduction to the study Unique contributions of the study Value-added assessment of student growth Colleges of Education evaluated on the basis of the performance of the graduates’ students Translation of evaluation results into improvement of teacher preparation programs 9/18/2018

Research Questions Is teacher preparation program related to students’ growth on the Texas Assessment of Knowledge and Skills (TAKS) when controlling for teachers’ prior academic achievement and for school characteristics? 9/18/2018

Research Questions Is teacher preparation program related to graduates’ performance on teacher certification exams when controlling for graduates’ prior academic achievement? 9/18/2018

Research Questions Is teacher preparation program related to graduates’ self-reported teaching ability when controlling for graduates’ prior academic achievement and for school characteristics? 9/18/2018

Research Questions What are the characteristics and content of the most highly effective teacher preparation programs? 9/18/2018

Indicators Academic growth of graduates’ students as measured by the Texas Assessment of Knowledge and Skills (TAKS) Graduates’ teacher certification exam scores (TExES/ExCET) Graduates’ self-reports of their teaching ability as indicated on the Teacher Survey Qualitative data on the characteristics and content of each program 9/18/2018

Data Sources School districts State Board for Educator Certification (SBEC) Graduates UT System components Texas Education Agency (TEA) 9/18/2018

Data Security and Confidentiality Protect confidentiality of the data Merge datasets Encryption algorithm developed at TEA Shared with UT System and SBEC School district data to TEA for encryption Files merged at NCEA after encryption 9/18/2018

Teacher Survey Interstate New Teacher Assessment and Support Consortium Core Standards How well do you do the following? How well did your teacher preparation program prepare you to do the following? How well have your in-school training experiences prepared you to do the following? 9/18/2018

Teacher Survey Interstate New Teacher Assessment and Support Consortium Core Standards (item examples) Adjust plans to meet students’ needs Foster relationships with colleagues Communicate student progress to parents 9/18/2018

Teacher Survey Rating of overall teaching ability Attributions of teaching ability Teacher preparation program In-school training experiences Natural ability Other 9/18/2018

Data Collection Overview Data sources 5, 297 Teacher Preparation Program Graduates 1,675 Texas High School Principals 9 UT System academic components Texas Education Agency (TEA) State Board for Educator Certification (SBEC) 102 school districts 9/18/2018

Teacher Survey Administered online March-August 2004 5,297 Graduates sampled for online survey 15% Response Rate 6 attempts made to reach target graduates Introductory letter Follow up letter Two email reminders Two mailings including a hard copy of the survey 9/18/2018

Data Collection UT Components Graduates’ prior academic achievement Transcript data Aggregates for college Specific data collected at each institution varies State Board of Educator Certification Certification type and level Exam scores Years of experience Employment information 9/18/2018

Data Collection Texas Education Agency Teacher and student demographic data Holding all district data submitted Teacher Preparation Program Staff Survey Specify unique attributes of each program 100% response rate 9/18/2018

Data Collection School Districts Districts procedures for submitting research request vary 102 districts contacted 8 had detailed procedures Followed set procedures 94 had no procedures NCEA developed generic request 9/18/2018

Data Collection School Districts Between 4-12 contacts made with each district Introduction Specific Data Request Follow up via Mail, Phone Calls, and Email Follow up Data Request Repeat 9/18/2018

Data Collection School Districts 102 School Districts contacted 51 Districts participated in study 51 Districts declined 35 Districts had no response 6 Districts had problems extracting data 5 Districts declined citing strain on resources 4 Districts’ superintendents declined participation 1 District refuses all research unless state mandated 9/18/2018

District Participation Austin Independent School District Participation in the NCEA Project will help lead to Ongoing improvement in UT System teacher preparation programs A more Highly Qualified applicant pool for our District Academic growth for AISD students 9/18/2018

New Teachers New teachers in AISD, 2004-05 800 new teachers hired 300 of these hold an alternative certification Special interest in findings regarding student outcomes for teachers who are alternatively certified Results and recommendations in this area will likely impact our future hiring practices 9/18/2018

Research in AISD Due to an increase in requests by external groups to conduct research in AISD, in 1996, our Dept. formalized a Research Proposal Application and Review Process (similar to an IRB process) Researcher submits application describing methodology of the study and the benefit to district (2003-04 N=70) Committee of staff with expertise in the appropriate program area(s) and staff with expertise in research methodology review application Project is approved, approved with modifications, or declined 9/18/2018

NCEA Project NCEA submitted application Project was approved because of the clear benefit to AISD Many of our teachers come through the UT system; The content area of Teacher Preparation is of interest, particularly in light of NCLB requirements; The design of the study and the evaluation system was well thought out and clearly articulated; and The project did not infringe upon student or campus staff time. 9/18/2018

District Expectations We look forward to reaping the benefits of this project Expectations include Best Practice findings will be used to make modifications, where recommended, to teacher pre-certification training in the UT system Outcomes regarding alternatively certified teachers will help district staff make more knowledgeable hiring decisions The evaluation system will be used for continuous improvement in UT system teacher preparation programs 9/18/2018

Wrap-up Questions Available information Business cards Higher Achievement for All Students (NCEA) 9 Essential Elements of Statewide Data Collection Systems 9/18/2018