1. Alabama Professional Education Personnel Evaluation Program Administrator Systems August 2002

2. Administrator Evaluation Systems Superintendent Central Office Administrator Principal Assistant Principal Alabama Department of EducationAdministrator Evaluation

3. 13 Knowledge/Skill Areas 1. Communication 2. Collaboration 3. Assessment/Evaluation 4. Organization 6. Laws/Policies 7. Problem Solving 8. Innovation 10. Operations/Management (school system, program, or school) 11. Fiscal Management/Leadership 9. Technology Management 13. Leadership of Human Resources 12. Professional Responsibilities 5. Planning Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

4. Scoring 4-point rating scale with rationale statements (rubrics) Holistic scoring cross-cutting 13 knowledge/skill areas Evaluators Trained in conducting evaluations Must meet reliability standards Administrators Evaluation content varies with years of experience in position Evaluation process is same for all Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

5. Instrumentation/Data Sources 1. Evaluation Summary Report (shows scores and score rationales by knowledge/skill area) 2. Professional Development Plan (becomes a data source in next cycle) 1. Portfolio developed by evaluatee 2. Surveys of various stakeholder groups 3. Interviews (written or oral) 4. Supervisor’s Review Form (except for superintendent system) 6. School observation (principal system only) 5. Board questionnaire (superintendent system only) Summary Report Sources Optional Instrumentation 1. Self-Assessment Form (completed by evaluatee, but not considered in evaluation scoring) Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

6. Administrator Data: evidence of competence produced, assembled and submitted by the administrator  Portfolio  Written interview  Interpretation of survey responses  Line of Evidence: all data/information supporting the score awarded in a knowledge/skill area, whether produced by the administrator or the evaluator  Evaluator Data: evidence of administrator competence developed by the evaluator  Oral Interview  School management observation (principal system only)  Supervisor's Review Form (except for superintendent system)  Clarifying interview   Board questionnaire (superintendent system only) Administrator Evaluation Systems Alabama Department of EducationAdministrator Evaluation

7. Instrument SuperintendentCentral OfficePrincipalAsst. Principal Self Assessment Portfolio Surveys Interviews Supr. Review Board Quest. Observation Eval. Summary Prof. Devel. XXXX XXXX XXXX XXXX XXXX X XXX XXXX X Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

8. Instrument SuperintendentCentral OfficePrincipalAsst. Principal Self Assessment Portfolio Surveys Interviews Supr. Review Board Quest Observation Eval. Summary Prof. Devel. 12 know/skills 11 know/skills 12 know/skills All know/skills Cntrl Off/Prin Cntrl Off/Prin/ Faculty 7 know/skills 8 know/skills 7 know/skills Faculty/Parents Students/Comm Faculty/Parents Students All know/skills 6 know/skills 4 know/skills 6 know/skills 5 know/skills All know/skills 2-3 know/skills Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

9. Alabama Department of EducationAdministrator Evaluation How the Administrator Evaluation System Works Self Assmnt Interview SRF Portfolio Mgt Obs (Principal) Evaluation Summary Report Surveys Prof Devlpmnt Plan Knowledge/ Skills Lines of Evidence Scores

10. Steps in Evaluation Process 1. Orientation (LEA responsibility) 2. Self-assessment (evaluatee) 3. Prepare portfolio (evaluatee) 4. Distribute and analyze surveys (evaluatee) 5. Supervisor Review Form - all but superintendent system (evaluator/supervisor) 7. Conduct observation - principal system (evaluator) 6. Board questionnaire - superintendent only (board members) Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

11. Steps in Evaluation Process (cont.) 9. Conduct interviews A and B (evaluator/evaluatee ) 10. Prepare Evaluation Summary Report (evaluator) 11. Summary conference and PDP (evaluator/evaluatee) 8. Review portfolio (evaluator) Alabama Department of EducationAdministrator Evaluation Administrator Evaluation Systems

12.  Generates data for all areas of knowledge and skills  Optionally used by the administrator and shared with evaluator only if administrator agrees  Used by administrator to: 1)Identify areas that need improvement 2)Compare the administrator’s perceptions of performance with the results of the evaluation 3)Determine areas for professional growth activities to be included in a professional development plan. Alabama Department of EducationAdministrator Evaluation Self-Assessment

13.  Two types of interviews:  A Structured Interview with questions commonly asked of administrators (can be oral or written response format)  A Clarifying Interview with probing questions focused on clarifying or bringing understanding to the portfolio and/or written interview (questions differ from administrator to administrator)  Conducted after the evaluator analyzes administrator’s portfolio  Modified scripting is used to record responses Alabama Department of EducationAdministrator Evaluation Interviews

14.  A discussion between the administrator and the evaluator (or his/her designee) about particular areas of the administrator’s knowledge, skills, and performance.  Generates data for seven areas of knowledge/skills:  Communication (oral)  Collaboration processes and skills  Assessment/measurement/evaluation  Organizing for results  Planning  Problem solving  Technology management Alabama Department of EducationAdministrator Evaluation Structured Interviews

15.  Each of the structured interview question sets is intended to cover one knowledge and skill area (although responses may cut across more than one area). Communication skills are considered as part of the oral presentation.  The structured interview itself is not scored, but responses are added to other evidence provided through other sources. Alabama Department of EducationAdministrator Evaluation Structured Interviews (cont.)

16.  Conducted after the structured interview is completed with a focus on the administrator’s portfolio and/or written interview  No set number of questions, but key questions specific to the portfolio are developed by evaluator to bring clarity and understanding of its contents  Responses are added to other evidence provided through other sources Alabama Department of EducationAdministrator Evaluation The Clarifying Interview

17.  Provides data where a supervisor should have objective, experiential information  Generates data for six (4 for principal) areas of knowledge and skills:  Professional Responsibilities  Fiscal Leadership/Management  Laws/Policies  Innovation (except principal system)  Operations/Management (except principal system)  Leadership of Human Resources  Not used in superintendent system Alabama Department of EducationAdministrator Evaluation The Supervisor’s Review Form

18.  Generates data for all areas of knowledge and skills  Questionnaire returned to state pool evaluator of superintendent  Used in superintendent system as a replacement for Supervisor’s Review Form  Questions relate to areas with which board members should be knowledgeable, although a “don’t know” response is allowable Alabama Department of EducationAdministrator Evaluation The Board Questionnaire

19.  Administrator is responsible for providing some evidence of his/her knowledge and skills  The totality of the evidential material (lines of evidence for all knowledge/skill areas) will constitute evaluatee’s evaluation portfolio  Generates data for 12 areas of knowledge and skills: (11 for principal)  Communication  Collaboration  Assessment/Evaluation  Organizing for Results  Planning  Laws and Policies  Problem Solving  Innovation  Technology Management  Operations/Management  Fiscal Leadership  Human Resources Leadership Alabama Department of EducationAdministrator Evaluation The Administrator Portfolio

20.  There is no one specific form of packaging required for the portfolio, but there are organizational requirements  Each entry (piece of evidence) is accompanied by a Portfolio Entry Form  Total portfolio requires approximately 28 pieces of evidence (varies by system)  One piece of evidence might serve 2-3 purposes (e.g., Program Improvement Plan)  With multiple purposes, a portfolio may have 10-12 entries (pieces of evidence) Alabama Department of EducationAdministrator Evaluation The Administrator Portfolio

21.  Explicit guidelines for developing a line of evidence in each area are provided in the evaluation manual "Extra" entries will not be considered by the evaluator   Evaluated on the quality of materials rather than the quantity of materials in the portfolio Alabama Department of EducationAdministrator Evaluation The Administrator Portfolio

22.  Respondents are to remain anonymous and represent a cross-section within the group  Generates data for 12 areas of knowledge and skills:  Communication  Collaboration  Assessment/Evaluation  Organizing for Results  Laws and Policies (staff only)  Problem Solving  Innovation  Operations/Management  Fiscal Leadership  Technology Mgt.  Professional Responsibilities  Human Resources Leadership  One survey sent to the following groups:  School system administrators/principals (supt. & central office)  Faculty/Staff (principal, asst. principal & central office, if appropriate)  Students (principal and asst. principal systems)  Parents (principal and asst. principal systems)  Community leaders (principal system) Alabama Department of EducationAdministrator Evaluation Surveys

23.  Analyses and reporting are responsibilities of administrator:  Information/report will be used with other evidence from other sources  Procedures and methodologies used by the administrator will in itself be used as part of evidence for assessing the knowledge/skill area of assessment/evaluation  Analyses and reporting:  Number of respondents by subgroup surveyed  Any problems with data  Conclusions by knowledge/skill area  Patterns and interpretation  Numbers for administration:  Total for all groups: 20 or all, if fewer than 20 in school system  Dependent on system, not all subgroups may need to be surveyed during one year Alabama Department of EducationAdministrator Evaluation Surveys (cont.)

24.  Generates data for five areas of knowledge and skills:  Communication  Organizing for Results  Innovation  Technology Management  School Operations/Management  At least 1 Management Observation required  Observer completes 4 major activities:  Observes "routines" of the school at active time of day  Walks through the school plant and facilities  Observes the pattern of the principal's activities  Observes the utilization and management of technology in the schools  Uses Modified Narrative Scripting  A Post-Observation conference will be held Alabama Department of EducationAdministrator Evaluation The Management Observation (Principal)

25.  Scores are not generated by instrument  There will be a total of 13 scores, one for each knowledge/skill area  At end of process, evaluator(s) will: 1.Organize evidence by knowledge/skill area 2.Read through all evidence for a knowledge/skill area 3.Award a score (1, 2, 3, 4) based on the patterns in the evidence in relation to the score level rationales for that particular knowledge/skill area 4.Place score with comments/justification on the Evaluation Summary Report 5.Use the scores and evidence for determining what should be addressed in the Professional Development Plan Alabama Department of EducationAdministrator Evaluation Scoring

26.  Used to present the scores developed from the lines of evidence for each of the 13 knowledge and skill areas  To complete the Evaluation Summary Report, the evaluator: 1.Completes demographic information requested 2.Records the scores developed from the lines of evidence for each knowledge and skill area 3.Writes any comments, explanations, or justifications in the space provided that support the score for the specific area 4.Identifies three areas for focus in planning professional development for the coming year(s) Alabama Department of EducationAdministrator Evaluation The Evaluation Summary Report

27. The Professional Development Plan Alabama Department of EducationAdministrator Evaluation  Activities focus on areas of improvement as identified in the Evaluation Summary Report  Activities focus on the improvement of student achievement and school improvement  Used to generate portfolio entry for Human Resources Leadership after first year of evaluation Professional Development Plan