Assessing Learning-Centered Leadership Andrew C. Porter University of Pennsylvania Joseph Murphy, Ellen Goldring, & Stephen N. Elliott Vanderbilt University October 2010

The Goal – Produce a Psychometrically Sound Assessment of School Leadership The team – Porter, Murphy, Goldring, Elliott, O’Toole, Cravens The Stimulus and Support – The Wallace Foundation identified the need, provided $1.5 million of support for 2005 to The Institute for Education Sciences is currently funding psychometric studies for the period of 2008 to

Purposes of school leadership assessment To guide self improvement and professional development For high stakes decisions; pay for performance, tenure Higher student achievement Better teacher retention Benefits of better school leaders 2010

The need for multiple measures when making high stakes decisions “As the stakes of testing increase for individual students, the importance of considering additional evidence to document the validity of score interpretations and the fairness in testing increases accordingly” (AERA, APA, NCME, 1999, p.141). Value added to student achievement is one measure. A direct assessment of learning centered leadership such as the VAL-ED is another The relative weight given to each measure is an important consideration. Probably greater weight, say 0.75, to behavior in a principal’s first year in a building and less as the principal continues in the same building, say

Learning-Centered Leadership Leaders should be assessed on leadership behaviors associated with student learning. Learning-centered leadership is leadership for student performance. Learning-centered leadership is the framework for our leadership assessment system. 2010

Leadership Behavior Framework 2010

Purpose & Uses The VAL-ED reports principal performance through  Norm-referenced scores and  Criterion-reference scores. VAL-ED can be used annually or more frequently to:  Facilitate a data-based performance evaluation,  Measure performance growth, and  Guide professional development. 2010

An Example Set of Responses 2010

Assessing Learning-Centered Leadership: The VAL-ED vision… A leadership assessment system that has the following properties: Works well in a variety of settings and circumstances, Is construct valid, Is reliable, If feasible for widespread use, Provides accurate and useful reporting of results. Is unbiased, Yields a diagnostic profile for summative and formative purposes. Can be used to measure progress over time in the development of leadership, and Predicts important outcomes. 2010

Some Results Respondent Groups – Supervisors (3.68) were significantly more positive than were teachers (3.59) or principals (3.53). There were no significant differences among levels of schooling (with controls high school principals were slightly less effective). There was no difference between Forms A (3.61) and C (3.62). The greater a principal’s years of experience in their school, the more effective the principal. Internal Consistency – Internal consistency reliability was high for both total score and each of the core components and each of the key processes with all reliabilities.87 or higher. For total score, the correlation between principal and supervisor was.13, principal and teacher.27, supervisor and teacher

Aggregated Effectiveness Ratings 2010

Comparisons Across Respondent Groups 2010

Cell-by-Cell Feedback 2010

Supporting Research & Publications Murphy, J., Elliott, S.N., Goldring, E., & Porter, A.C. (In Press). Leaders for productive schools. In M. Brundrett & M. Crawford (Eds.), Developing School Leaders: An International Perspective.London: Routledge. Porter, A.C., Polikoff, M.S., Goldring, E., Murphy, J., Elliott, S.N., & May, H. (2010). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. The Elementary School Journal, 111(2). Porter, A.C., Polikoff, M.S., Goldring, E., Murphy, J., Elliott, S.N., & May, H. (April 2010). Developing a psychometrically sound assessment of school leadership: The VAL-ED as a case study. Educational Administration Quarterly, 46(2), Murphy, J., Elliott, S. N., Goldring, E., & Porter, A. C. (2010) Leaders for productive schools. In B. McGaw, P. Peterson, & E. Baker (Eds), International Encyclopedia of Education (3 rd ed.). Elsevier. Polikoff, M.S., May, H., Porter, A.C., Elliott, S.N., Goldring, E., & Murphy, J. (November 2009). An examination of differential item functioning in the Vanderbilt Assessment of Leadership in Education. Journal of School Leadership, 19(6), Goldring, E., Porter, A.C., Murphy, J., Elliott, S.N., & Cravens, X. (2009). Assessing learning-centered leadership: Connections to research, professional standards, and current practices. Leadership and Policy in Schools, 8(1), Goldring, E., Cravens, X. C., Murphy, J., Elliott, S. N., Porter, A. C. & Carson, B., (2009). The evaluation of principals: What and how do states and urban districts assess leadership? The Elementary School Journal, 110(1), Goldring, E., Porter, A., Murphy, J., Elliot, S., & Cravens, X. (2009). Assessing Learning-Centered Leadership: Connections to Research, Standards and Practice. Leadership and Policy in Schools, 8(1), Goldring, E. Porter, A. Pollikoff, M (2008). Report on the Study to Evaluate the Performance Level Descriptors for the VAL-ED. New York, NY: Wallace Foundation. Murphy, J., Elliott, S.N., Goldring, E., & Porter, A.C. (2007). Leadership for learning: A research-based model and taxonomy of behaviors. School Leadership & Management, 27 (2), Murphy, J., Elliott, S.N., Goldring, E.B., & Porter, A.C. (2006). Learning-centered leadership: A conceptual foundation. New York, NY: Wallace Foundation. Murphy, J.F., Goldring, E.B., Cravens, X.C., Elliott, S.N., Porter, A.C. (2007, August). The Vanderbilt Assessment of Leadership in Education: Measuring Learning-Centered Leadership. Journal of East China Normal University. Porter, A.C., Goldring, E.B., Murphy, J., Elliott, S.N., & Cravens, X. (2006). A framework for the assessment of learning-centered leadership. New York, NY: Wallace Foundation. Porter, A.C., Goldring, E.B., Elliott, S.N., Murphy, J., Polikoff, M., and Cravens, X. (2008). Setting Performance Standards for the VAL- ED Assessment of Principal Leadership, New York: NY: Wallace Foundation. Porter, A.C., Goldring, E.B., Murphy, J., Elliott, S.N., Polikoff, M., and May, H. (2008). VAL-ED Assessment of Principal Leadership Technical Manual, New York, NY: Wallace Foundation. Elliott, S.N., Porter, A.C., Goldring, E.B., Murphy, J., Polikoff, M., and May, H. (2008). VAL-ED Users’ Guide, New York, NY: Wallace Foundation. These and other publications are all available for download at

Reliability and Validity Studies in Progress – IES-funded IES-funded Studies  Known Group Study – Does the VAL-ED reliably distinguish principals who are identified by others as more or less effective?  Test/Retest Reliability  Consequences Study – Study of How the VAL-ED Was Used and to What Effect  Longitudinal Study – Does effectiveness as measured by the VAL-ED predict future gains in student achievement? 2010

To Learn More … Google: VAL-ED We invite conversation with you on the timely and important topic of school leadership assessment 2010