A GLIMPSE OF FAIR TESTING Intended to decode the meaning of “fair” in testing practice and jot down some of the factors and conditions affecting language test results by creating favorable test environment for some candidates while impairing the performance of others; Petya GEORGIEVA Skopje, 5 – 7 September 2017 STANAG 6001 Тesting Workshop

A GLIMPSE OF FAIR TESTING Fair = free from bias, dishonesty, or injustice (a fair decision; a fair judge) Synonyms - impartial, disinterested, unprejudiced  “Fair” implies the treating of all sides alike, justly and equitably All synonyms refer to lack of bias in opinions, judgments, etc.

A GLIMPSE OF FAIR TESTING “I will always remember the horror of receiving my chemistry result when I was thirteen years old. I knew it wasn't going to be high, but to come bottom of the class was very upsetting. It was all made worse by the fact that the chemistry teacher read the results to the whole class, from first to last place. My humiliation was complete.” Richard Frost, British Council When he said that, the man was over 70! Some food for thought which can lead us to the question “Who is interested in fair testing?” and, also of the relationship between fair testing and ethical considerations when it comes to reporting results.

A GLIMPSE OF FAIR TESTING WHO IS INTERESTED IN FAIR TESTING? STAKEHOLDERS TEST PARTICIPANTS TEST DEVELOPERS AND TEST USERS TEST ADMINISTRATORS A non-exclusive list;. If we try to define each of these groups we’ll see that there is some overlapping between them. It may be worth noting here that the Code of Fair Testing Practices in Education developed by the Joint Committee on Testing Practices in Washington , D,C. (1988) highlights key points of the quality of testing practices. This is also true of the Code of Ethics prepared by the International Language Testing Association (2000) and the Code of Ethics of the Association of Language Testers in Europe

A GLIMPSE OF FAIR TESTING The Codes of Ethics ( ILTA, ALTE) The Code of Fair Testing Practices in Education

A GLIMPSE OF FAIR TESTING Defining fair testing B.Spolsky(1997); Bachman & Palmer(1996;2010); Fulcher & Davidson (2007) A Handbook of Research in Second Language Teaching and Learning (2016) International Journal of Modern Language Teaching and Learning (2016, Vol. 1) Based on Fulcher, conducting a fair test involves: - replicating the meticulous conditions that the test designers set out in the delivery specifications and under which the test was originally piloted; - ensuring that these conditions are replicated for all test takers, so that no one experiences taking the test in a condition that provides either advantages or disadvantages.

A GLIMPSE OF FAIR TESTING Ensure efficient control of the THREE TEST STAGES to guarantee a fair testing process Before (A) /During (B) /After (C) test administration 1. test media (paper-based or CBT) - equivalent paper-based versions (length; difficulty; item specifics; tasks layout; quality of printing, font; quality of sound for assessing listening, etc.) - comparable CAT tasks for the respective skills tested; adequate properly functioning equipment, technically proficient proctors, etc.) 2. familiarization materials (instructions, guide books, demo versions or sample questions for the different skills tested) 3. test administrators should be informed on how to deal with latecomers and discourage/prevent any attempts to cheat, etc.

A GLIMPSE OF FAIR TESTING B. STRIVING FOR FAIRNESS DURING THE TEST ADMINISTRATION STAGE - Defining and adhering to unified test administration procedures Established examination bodies should enact procedures that help to ensure that differences in performance are related primarily to the skills under assessment rather than to irrelevant factors. Guarantee anonymity of test takers – use codes, not names; Testers’ behavior at interviews - do not test your students; be polite but keep the distance, etc. Uniform conditions for all test takers; Commitment to test security – all violations are reported; Observe carefully in giving instructions or clarifications; Avoid interruptions;

A GLIMPSE OF FAIR TESTING Walking around the room? – Does it help or distract the test participants? Keep testers informed of the progress of testing time? No eating/drinking in the test hall (both invigilators and testees); No use of phones; Handling emergencies; Etc.

A GLIMPSE OF FAIR TESTING C. AFTER TEST ADMINISTRATION Securing test material; preventing information leak Reporting and Interpreting scores Complying with deadlines for reporting scores In case scores are uploaded on a platform or made public in any way, always ask candidates’ consent Who is involved at this stage?

A GLIMPSE OF FAIR TESTING THANK YOU FOR YOUR ATTENTION !