Field Test of Psychologist/ Psychometrist Evaluation System July 2000 Alabama Professional Education Personnel Evaluation Program

Provides feedback from those who are being evaluated and those who are evaluators  Provides data for setting standards  Alabama Psychologist/Psychometrist Evaluation System Field Test Provides a sense of how well the instruments and procedures are working  Provides information on what areas of the evaluation system need changing  Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

Eight Competencies: Structure parallels other evaluation systems 1.0Preparation for Delivery of Services 2.0Delivery of Psychological/Psychometric Services 3.0Evaluation of Psychological/Psychometric Services 4.0Management of the Psychological/Psychometric Program and Student Behavior 5.0Positive Professional Climate 6.0Communication 7.0Professional Development and Leadership 8.0Performance of Professional Responsibility Competencies Alabama Department of Education Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

Data Collection Instruments/Forms Alabama Department of Education Self-Assessment Interview (oral or written) Supervisor Review Form Professional Development Plan (generates Professional Development/Leadership Competency score during next cycle) Evaluation Summary Report Portfolio Surveys Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

 Generates data for all competency areas  Optionally used by the psychologist/psychometrist and shared with evaluator only if psychologist/psychometrist agrees  Used by the psychologist/psychometrist to: 1)Identify areas that need improvement 2)Compare the psychologist/psychometrist ’s perceptions of performance with the results of the evaluation 3)Determine areas for professional growth activities to be included in a professional development plan Self Assessment Alabama Department of Education Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

 Two types of interviews:  A Structured Interview with 6 questions commonly asked of all psychologists/psychometrists (oral and written options)  A Clarifying Interview with probing questions focused on clarifying or bringing understanding to the written interview and portfolio (questions differ from one psychologist/psychometrist to another)  Clarifying Interview conducted after the evaluator analyzes the written interview and portfolio  Modified scripting is used to record responses in oral option The Interview Alabama Department of Education Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

 Generates data for competencies in Delivery of Psychological/Psychometric Services, Management, Communication, and Professional Responsibilities  Provides data where a supervisor should have objective, experiential information Supervisor’s Review Form Alabama Department of Education Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

Portfolio  There is no one specific form of packaging required for the portfolio, but there are organizational requirements  Each entry (piece of evidence) is accompanied by a Portfolio Entry Form  Total portfolio requires approximately 16 pieces of evidence  One piece of evidence might serve 2-3 purposes  With multiple purposes, a portfolio may have 9-11 entries (pieces of evidence)  Psychologist/Psychometrist is responsible for providing some evidence of his/her knowledge and skills  Generates data for 4 Competencies and 6 Indicators Alabama Department of Education Alabama Psychologist/Psychometrist Evaluation System Psychologist/Psychometrist Field Test Evaluation

Portfolio  Explicit guidelines for developing evidence in each area are provided in the evaluation/orientation manual "Extra" entries will not be considered by the evaluator   Evaluated on the quality of materials rather than the quantity of materials in the portfolio Portfolio Alabama Psychologist/Psychometrist Evaluation System Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

IndicatorEvidence  2.1 Assessment and Evaluation  Survey results 2 artifacts  2.2 Intervention, Prevention Consultation  Survey results 3 artifacts (1 in each area)  2.3 Education/Research  Survey results 2 artifact  3.1 Evaluates Program  1 artifact around evaluation 1 artifact around use of results  6.2 Written Communication  2 artifacts around written work  8.4 Cooperation with School/Comm  Survey results 1 artifact Portfolio Alabama Psychologist/Psychometrist Evaluation System Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

 Surveys are being field tested to determine whether they can be used in the Portfolio and Supervisor’s Review Form as part of the holistic scoring process  Surveys are provided for teachers, students in grades 4-6 and 7-12, and parents  Survey questions address the Competencies and Indicators The Survey  The psychologist/psychometrist will distribute, collect, and analyze the surveys  Respondents are to remain anonymous and represent a cross-section within the group Alabama Psychologist/Psychometrist Evaluation System Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

 Used to present the scores developed from each of the instruments  To complete the ESR, the evaluator: 1.Completes demographic information requested 2.Records the scores from each instrument 3.Writes comments, explanations, or justifications in the space provided that support the score for the specific area 4.Identifies areas for focus in planning professional development for the coming year(s) Evaluation Summary Report Alabama Psychologist/Psychometrist Evaluation System Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

 Activities focus on areas of improvement as identified in the Evaluation Summary Report  Activities focus on the improvement of student achievement Professional Development Plan  Used to evaluate Competency 7: Professional Development and Leadership after first year of evaluation Professional Development Plan Alabama Psychologist/Psychometrist Evaluation System Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

Questions? Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

Teacher SystemPsychologist/Psychometrist System  Eight Competencies  COMPETENCIES PSYCHOLOGIST/PSYCHOMETRIST  Preparation for Instruction  Presentation of Instruction  Assessment of Student Performance  Classroom Management  Positive Climate  Communication  Prof. Development/Leadership  Professional Responsibilities  Preparation for Delivery of Services  Delivery of Psychological/Psychometric Services  Evaluation of Psychological/Psychometric Services  Management of Psychological/Psychometric Program and Student Behavior  Positive Professional Climate  Communication  Prof. Development/Leadership  Professional Responsibilities

Teacher System  Self Assessment  (Same format)  Interview   Observation (3 questions, 5 indicators. Focus around instructional preparation, student assessment) (6 questions, 7 indicators. Focus around preparation of services, delivery of services and evaluation of those services (17 indicators)  No Equivalent  Pre/Post Observation  No Equivalent INSTRUMENTS/DATA SOURCES PSYCHOLOGIST/PSYCHOMETRIST Psychologist/Psychometrist System

Teacher System (Adds program parts of services, management, and climate. Rest same as teacher system)  Supervisor’s Review Form   Evaluation Summary Report  ESR is same except for adding portfolio scoring and removing observation  PDP  PDP has no changes (Not scored in field test since no past PDP has been developed)  Competency 7 Scoring  Competency 7 Scoring is same INSTRUMENTS/DATA SOURCES PSYCHOLOGIST/PSYCHOMETRIST  Portfolio and surveys  (No Equivalent) Psychologist/Psychometrist System

PORTFOLIO PSYCHOLOGIST/PSYCHOMETRIST IndicatorEvidence  2.1 Assessment and Evaluation  1 survey results 2 artifacts  2.2 Intervention, Prevention, and Consultation  1 survey results 3 artifacts around each of the three areas  2.3 Education and Research  1 survey results 2 artifacts  3.1 Evaluates Program  1 artifact around evaluation 1 artifact around use of results  6.2 Written Communication  2 artifacts around written work  8.4 Cooperation with School/Comm  1 survey results 1 artifacts