Susan Malone Mercer University

 “The unit has taken effective steps to eliminate bias in assessments and is working to establish the fairness, accuracy, and consistency of its assessment procedures and unit operations.”

 Address contextual distractions (inappropriate noise, poor lighting, discomfort, lack of proper equipment)  Address problems with assessment instruments (missing or vague instructions, poorly worded questions, poorly produced copies that make reading difficult)

 [Review candidate performance to determine if candidates perform differentially with respect to specific demographic characteristics]  [ETS: Guidelines for Fairness Review of Assessments; Pearson: Fairness and Diversity in Tests]

 Assure candidates are exposed to the K, S, & D that are being evaluated  Assure candidates know what is expected of them  Instructions and timing of assessments are clearly stated and shared with candidates  Candidates are given information on how the assessments are scored and how they count toward completion of programs

 Assessments are of appropriate type and content to measure what they purport to measure  Aligned with the standards and/or learning proficiencies they are designed to measure [content validity]

 Produce dependable results or results that would remain constant on repeated trials ◦ Provide training for raters that promotes similar scoring patterns ◦ Use multiple raters ◦ Conduct simple studies of inter-rater reliability ◦ Compare results to other internal or external assessments that measure comparable K, S, & D [concurrent validity]

 Dispositions Assessment  Portfolios  Analysis of Student Learning  Summative Evaluation

 Alignment with INTASC standards, program standards, and the Conceptual Framework (accuracy; content validity) ◦ Matrices Matrices ◦ LiveText standards mapping LiveText standards mapping ◦ PRS Relationship to Standards section PRS Relationship to Standards section ◦ PRS alignment with program standards requirement in Evidence for Meeting Standards section  Alignment with other assessments (accuracy; concurrent validity) ◦ Matrices Matrices ◦ Potential documentation within LiveText Standards Correlation Builder Potential documentation within LiveText Standards Correlation Builder

 Rubrics/assessment expectations shared with candidates in courses; field experience orientations; and LiveText (fairness)  Rubrics/assessment expectations shared with cooperating teachers by university supervisors (consistency)  Statistical study (in process) examining correlations among candidate performances on multiple assessments (where those assessments address comparable K, S, & D) (consistency; concurrent validity)

 Multiple assessors (consistency)  Exploration of faculty’s assumptions re: purpose of the assessment, expectations of behaviors, and meaning of rating scale (consistency; reliability)  Revision of rating scale, addition of more specific indicators, development of two versions (courses/field experiences) (accuracy; content validity)

 Norming session with supervisors (consistency; reliability)  Revision of instructions to align more closely with rubric expectations and expected process (fairness; avoidance of bias)  Review of coursework and fieldwork to ensure candidates are prepared for assignment (fairness)

 Seeking feedback from experts (P12 partners) on whether assignment reqs and assessment criteria are authentic (accuracy; content validity)  Annual review of data disaggregated by demographic factors (gender, race/ethnicity, site, degree program) (fairness, avoidance of bias)

 Recent revision of portfolios and rubrics to align with new INTASC (accuracy; content validity)  Revision of rubrics to include more specific indicators related to the standards (change from generic rating descriptors) (accuracy; content validity)  Cross-college workshop on artifact selections (accuracy; content validity)

 Annual review of artifact selections (accuracy; content validity)  Inter-rater reliability study (consistency; reliability)

 Review of rubric expectations and all other PR and ST assignments to ensure opportunities to demonstrate all standards and indicators during experience (fairness)  Workshops for supervisors on the rubric expectations (consistency)  Feedback from cooperating teachers on relevance of the assessment (accuracy; content validity)  Annual review of data disaggregated by demographic variables (fairness; avoidance of bias)

 Statistical study to identify correlations among entry reqs and successful program completion  Statistical study to determine if key assessments and entry criteria are predictive of program success (as defined by success in student teaching)