1. Bonnie S. Billingsley, Professor Virginia Tech College of Liberal Arts & Human Sciences School of Education, Department of Teaching & Learning billingsley@vt.edu OSEP Project Director’s Conference July 20, 2010 1 Evaluation for Special Education Teacher Effectiveness

2. It’s Complicated… “…it’s hard, you know there are days I think coming up with new vaccines for things like malaria are in some ways more straightforward than a personnel system that everyone is going to feel great about.” Bill Gates to Jon Stewart in a discussion about teacher evaluation (The Daily Show, January 25, 2010)

3. Changing Context for Teacher Evaluation  Controversial: varied stakeholders & interests  Interest in teacher evaluation related to relationship between teacher quality and student achievement  Teacher evaluation as “policy lever” to improve teacher performance & reduce achievement gaps

4. Challenges  Conceptual Effective teaching Purposes of evaluation  Measurement Psychometric properties of assessments In specific contexts & with special populations  Contextual Teachers’ opportunities for effectiveness Teacher support & learning

5. Selected Teacher Evaluation Approaches/Measures  Examples Value-added Structured Observation Systems Other observations Teacher checklists/rating scales Portfolios/classroom data/artifacts Individual educational plans Student performance on learning goals

6. Structured Observations  Global observation measures Framework for Teaching Observation Survey (Danielson, 1996, 2007)  Used in special education research (Nougaret et al., 2005; Sindelar et al., 2004) Classroom Assessment Scoring System [CLASS] (Pianta, La Paro & Hamre, 2008)  Specific observation measures Mathematical Quality of Instruction (MQI) (Hill et al., 2008) English Language Learner Classroom Observation Instrument (ELLCOI) (Baker et al., 2006)

7. Chicago Public Schools: Pilot of Excellence in Teaching Project  Modified Danielson framework  Purposes: improve teaching, basis for coaching, reflection & professional development (PD)  Pre-observation/observation/post-observations  Four levels: unsatisfactory, basic, proficient & distinguished  Studying effects: (e.g., reliability, relationship between observation scores & student achievement, user perspectives)  “Clarifying” use with special education

8. Contextual Challenges: Opportunity for effectiveness?  Consistent SET shortages, especially high poverty districts (Boe, 2006; Fall & Billingsley, 2008; U.S. Department of Education)  Formidable content & pedagogical demands  Diverse & increasing caseloads (Carlson et al., 2002; McLeskey et al., 2004)  Lack of clarity about roles (Gehrke & Murri, 2006)  Insufficient time on instruction: 40% (Vannest & Hagan-Burke, 2010)

9. Contextual Challenges: Opportunities for SET Learning?  Evaluator expertise Principals Teacher leaders & instructional coaches  Opportunities for professional development  Professional learning communities

10. Conclusions

11. References  Baker, S., Gersten, R., Haager, D., & Dingle, M. (2006). Teaching practice and the reading growth of first-grade English learners: Validation of an observation instrument. Elementary School Journal, 107(2), 199–221.  Boe, E. E., & Cook, L. H. (2006). The chronic and increasing shortage of fully- certified teachers in special and general education. Exceptional Children, 72(4), 443-460.  Carlson, E., Brauen, M., Klein, S., Schroll, K. & Willig, S. (2002). Study of Personnel Needs in Special Education: Key Findings. Retrieved from http://fer.dig.coe.ful.edu/spense/KeyFindings.pdf. http://fer.dig.coe.ful.edu/spense/KeyFindings.pdf  Danielson, C. (1996; 2007). Enhancing professional practice: A framework for Teaching. Alexandria, VA: Association of Supervision and Curriculum Development.  Fall, A. M., & Billingsley, B. (2008). Disparities in teacher quality among early career special educators in high and low poverty districts. In T. E. Scruggs & M. A. Mastropieri (Eds.), Advances in learning and behavioral disabilities: Vol. 21. Personnel preparation (pp. 181-206). Stanford, CT: JAI.

12. References (continued)  Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26(4), 430–511.  Gehrke, R. S., & Murri, N. (2006). Beginning special educators' intent to stay in special education: Why they like it here. Teacher Education and Special Education, 29(3), 179-190.  McLeskey, J., Tyler, N., & Flippin, S. S. (2004). The supply of and demand for special education teachers: A review of research regarding the nature of the chronic shortage of special education teachers. The Journal of Special Education, 38(1), 5-21.  Nougaret, A. A., Scruggs, T. E., & Mastropieri, M. A. (2005). Does teacher education produce better special education teachers? Exceptional Children, 71(3), 217-229.

13. References (cont.)  Pianta, R. C., La Paro, K., & Hamre, B. K. (2008). Classroom Assessment Scoring System (CLASS). Baltimore: Paul H. Brookes.  Sindelar, P. T., Daunic, A., & Rennells, M. S. (2004). Comparisons of traditionally and alternatively trained teachers. Exceptionality, 12(4), 209- 223.  United States Department of Education. Annual Reports to Congress on the Implementation of the Individuals with Disabilities Education Act, Washington, DC: Author.