1. Learning Point Associates
Measuring Principal Performance: How Rigorous Are Publicly Available Assessment Instruments?
Matthew Clifford
Learning Point Associates
Presentation for PEAC
July 2010

2. Learning Point Associates Overview

3. What is Your Perspective on Principal Performance Assessment?
What instruments are you or others currently using?
How systematically are the instruments being administered?
How are results being used?
How confident are you in the assessment instruments and process?

4. What Research Tells Us about Principal Performance Assessment
School districts’ principal performance assessments appear to be inconsistently administered and measured (Goldring et al, 2009; Thomas et al, 2000; Murphy et al, 2007).
Principal performance assessments are not always aligned with existing state or national professional standards for practice, and lack justification or documentation of psychometric research (Heck & Marcoulides, 1996).

5. Learning Point Associates: Managing Educator Talent
Performance Management
Coherent
Systematic

6. Principal Performance Assessment: Recommended Attributes
Transparent
Trusted
Consequential
Systematic and consistent
Involving multiple measures
Validate and reliable
(Brown-Sims, 2010)

7. Our Study: Question & Method
Question: How valid and reliable are publicly available principal performance assessment instruments?

8. Our Study: Review Criteria and Results
Criteria for inclusion:
Claim of use as a principal performance assessment
Recent (within past 15 years or so)
Publicly available technical information
Results of scan:
2000 articles identified by keyword search & snowball sampling
Approx examined
20 articles reviewed on 8 assessment instruments

9. Our Study: Instruments Reviewed
Approach
Validity
Reliability
Change Facilitator Style Questionnaire
(1988)
77-item survey focusing on 6 domains
360-degree of principal as change facilitator
Content: Lit review
Construct: Factor analysis
Alpha Range:
.64 to .95
Diagnostic Assessment of School and Principal Effectiveness
(1992)
213-item survey
360-degree
Content: Expert review
Construct: Factor analysis and inter-item correlation
.8 to .97
Instructional activity questionnaire
(1987)
34-item assessment
Focus on instructional leadership
Alpha Range: .7 to .9

10. Our Study: Instruments Reviewed
Approach
Validity
Reliability
Instructional activity questionnaire
(1987)
34-item assessment
Focus on instructional leadership
Content: Lit review
Construct: Factor analysis
Alpha Range: .7 to .9
Leadership Practices Inventory
(2002)
30-items
Principal and supervisor completes
Leadership effectiveness
Content: Interviews and surveys of leaders
Concurrent: Other management measures
Test/re-test: .79
Performance Review Analysis and Improvement System for Education (1985)
81-item with 9 sub-scales
Two-dimensional profile with +/-
Content: Lit review with Delphi panel
Construct: Unknown
Alpha Range: .88 to .98
Test/re-test: .59 to .8

11. Our Study: Instruments Reviewed
Approach
Validity
Reliability
Principal Instructional Management Rating Scale
(1985)
71-item questionnaire with 11 subscales
Content: Lit review and Delphi panel
Construct: Correlation within subscale and school document review
Alpha: .75
Principal Profile
(1986, 1987)
Interview-based protocol characterizes principal effectiveness and leadership style
Content: Lit review
Construct: Factor analysis
Inter-rater agreement range from .5 to 1
VAL-ED
(2006)
72-items
360-degree, produces profile
Alpha= .98 for 12 scales on different forms

12. Our Study: What did we learn?
If there are many principal performance assessment being used in the field, then evidence of their reliability and validity is not publicly available.
Of those reviewed, 5 of 8 were developed in tested in the mid-to-late1980s.
Survey-based assessment gather self, and others (3), perceptions of principal performance. One used a trained rater.
No information on consequential validity, and two examples of concurrent validity.

13. Matt Clifford
Phone:
1120 East Diehl Road, Suite 200
Naperville, IL
General Information: