Fitz-Albert R. Russell, PhD JP Establishing Standards for Students’ Assessment

INTRODUCTION Psychometric assessments used for decisions about work, jobs, and careers. In education, students assessment used for four types of decisions. Administrative & policy formulation, instructional, certification, selection decisions. Testing & measurement applied to other persons & for other reasons-not dealt with here

BACKGROUND > 1,000 incidences of between Grs Each is important to the child life chances Some are high stakes & summative in nature Examples of high stakes students’ assessment Grade Six Achievement Test (GSAT) CXC Secondary Education Certificate (CSEC)

VALIDITY AS QUALITY INDICATOR Are decisions made about: possibilities, abilities, achievements, prospects, & predisposition based on assessments appropriate for purpose? Three considerations: Are testing conditions are fair to all test takers Are scores reliable: consistent across time, space, thought, culture variations, instrument form and test takers’ ability Is test valid: uses made of results are appropriate & consistent with intended use.

MESSICK’S VIEW Validity: the pre-eminent consideration in students’ assessment Incorporates concerns of fairness and reliability Drives need for the establishment of standards. Messick (1989a, 1989b) viewed validity as “evaluative judgment of a unitary concept that is based on evidence and consequences of appropriate test use and score interpretation” Professionals must recognize issues of bias & drift in items, be able to obviate them

STANDARDS IN ASSESSMENT Classical concepts of validity do not represent all issues of quality. All elements in arrangements & administrative conditions impact validity judgment, & require quality considerations. Quality is not necessarily JIT delivery of material Rather, a good regime and protocol consistently applied (Meeter 1997) is required Standards established & rigorously applied serve to improve the validation judgment.

TEST DEVELOPMENT PROCESS Curriculum Analysis Assessment planning Item development, improvement, and validation Compilation of assessment instruments Preparation of first copies Enroll test takers & prepare registers Materials Management & Presentation Response digitization Mark, score, scale, equate, bias assessment Score utilization & reporting Process documentation.

NCME STANDARDS BY APA The “standards” organized into three parts Part 1: Test Construction, Evaluation, and Documentation Part 2: Fairness in Testing Part 3: Testing Applications.

JAMAICAN STANDARDS: SET 1 Relates to instrument construction & validation Explicit curriculum referencing the assessment Learning outcomes with assessment specification A set of exclusive sample items; Preliminary testing of item in final form of a test ; Items reliability (pbis) >=.75 At least one parallel form of a test available No item re-used within a three year period.

JAMAICAN STANDARDS: SET 2 Relates to materials management and student processing First copy of instrument lodged & used as authentic paper. Security features built into the printing, processes and packaging; Packages opened in assessment room, with certification by principal; Documentation of the process, at each step;

JAMAICAN STANDARDS: SET 3 Relates to scoring, scaling equating & analysis Turnaround on all scores within 30 days Appropriate scoring procedures & scaling procedures pre-designated for open-ended/self response items Pre-designated procedures for dealing with missing responses Comparative scores reported as equated scores Item & test analysis results documentation; Procedures carried out by professionals

JAMAICAN STANDARDS: SET 4 Relates to appropriate documentation, reporting & use of data Interpretation: Appropriate pre-designated procedure for producing standardized scores; Reporting: documentation to inform interpretation & judgment of validity Documentation: assessment documented & presented to the NCE Utilization: Response available for individual test takers, facilitating “value added” analysis.

JAMAICAN STANDARDS: SET 5 Relates to issues of fairness Administration: conditions for test takers to write tests under fair and equal conditions Presentation of assessments: Tests may be presented as paper based or tablet form. Rate of work: empirically determined providing time allowing 90 % of test takers to complete Right of appeal: An appeal panel established

CONCLUSION I Standards for governance & management, accountability, quality assurance measures, & expected service delivery Standards proposed a) assessment instruments construction & validation b) materials management & student processing c) scoring, scaling equating & analysis d) appropriate documentation, reporting & use of data e) issues of fairness. Standards for high stakes assessments

CONCLUSION II Standards represent important advance governance in Jamaica. Process of consultation & agreement important as complex & technical issues are discussed, possibilities explored & expectations accepted Standards codified & compiled into a document along with standards for other technical and related areas of service delivery. Standards are about governance, management, and service delivery. Accountability of the Permanent Secretary.

Thank you!

Fitz - A. Russell, PhD JP for Edumetrics HPT Consulting 18