November 4-5, 2002Test Coordinator's Training1 Testing 101 Dr. Nick Stayrook Fairbanks North Star Borough School District

November 4-5, 2002Test Coordinator's Training2 What is a test? A test is a collection of questions designed to evaluate a students knowledge Students responses to test questions are scored to determine level of knowledge Right or wrong (multiple choice) Quality of answer (rubric/scoring guide) Tests generate scores for students

November 4-5, 2002Test Coordinator's Training3 Tests Used in State Assessments Benchmark Exams in Grades 3, 6, and 8 TerraNova Exams in Grades 4, 5, 7, and 9 Alaska High School Qualifying Exam (beginning in grade 10) Tests measure subject area knowledge Alaska Performance Standards Reading, Writing, and Mathematics

November 4-5, 2002Test Coordinator's Training4 Test Maps Every test has a map of the content measured on the test Benchmark Exams use the performance standards TerraNova Exams use common objectives across curricula used in the United States Benchmark Test Maps on TerraNova Test Maps in Teachers Guide

November 4-5, 2002Test Coordinator's Training5 Determining Student Performance Students receive points for each test question and total test points are computed The total points are converted into a scale score which ranges from 100 to 600 for Benchmark Exams and HSGQE The total points (raw score) are converted into a percentile score for TerraNova Exams which ranges from 1 to 99

November 4-5, 2002Test Coordinator's Training6 Performance Levels & Quartiles For Benchmark Exams and the HSGQE the State has established performance levels Advanced, Proficient, Below Proficient, Not Proficient for Benchmark Exams Proficient and Not Proficient for the HSGQE For TerraNova Exams we report the percent of students in each quartile: Top Quartile is percentile, bottom quartile is 1-25.

November 4-5, 2002Test Coordinator's Training7 Test Reports Student Reports provide information to parents and students School Reports provide summary information to schools and teachers District Reports are summaries at the district level and for each school

November 4-5, 2002Test Coordinator's Training8 Uses of Tests To assess individual student performance In relation to a specific cut score – Criterion Referenced Tests (CRT) In relation to a norm group – Norm Referenced Tests (NRT) The difference in CRTs and NRTs is not the test questions, but how you use the results The Benchmark Exams could be NRTs if we compared student results to a norm group TerraNova could be a CRT if we established a cut score for proficiency

November 4-5, 2002Test Coordinator's Training9 Test Administration 101 Based on the work of Dean Arasmith Northwest Regional Educational Laboratory

November 4-5, 2002Test Coordinator's Training10 Test Scores from Assessments A students observed test score consists of the students true score plus error. Assumption: Error is random and not systematic Systematic error can be understood as assessment bias

November 4-5, 2002Test Coordinator's Training11 Characteristics of Assessments Reliability of Test – Accuracy Reliability is the concept that if a test is given repeated times the scores would be similar Types of Reliability Parallel Forms & Split Half Repeated measures (Test/Re-test) Internal Consistency

November 4-5, 2002Test Coordinator's Training12 Characteristics of Assessments Validity of Assessment Tool Validity is the concept that an assessment tool measures what it is intended to measure. An assessment is validated based on the use to which it is put. Types of Validity Content Validity Predictive Validity Concurrent Validity Construct Validity Consequential Validity

November 4-5, 2002Test Coordinator's Training13 Sources of Errors in Test Scores Errors due to the assessment tool Errors due to student factors Errors due to testing environment

November 4-5, 2002Test Coordinator's Training14 Errors Due to the Assessment Tool Lack of high reliability or validity Too much reading on the math test Unclear or novel instructions Unnecessarily difficult vocabulary used in directions or test items Irrelevant clues to right answers Not appropriate for the purpose intended

November 4-5, 2002Test Coordinator's Training15 Errors Due to Student Factors Student fatigue, illness, or hunger Lack of motivation Lack of test wise-ness Not understanding test directions Not understanding purpose and importance of the test

November 4-5, 2002Test Coordinator's Training16 Errors Due to Testing Environment Physical Room Conditions Too hot, dim lighting, no ventilation Noisy, distractions inside and outside room Test Proctor Behaviors Does not adhere to test directions Gives assistance to students Negative attitude toward test

November 4-5, 2002Test Coordinator's Training17 To Eliminate or Reduce Errors in Test Scores Strictly follow test administration procedures contained in: District Test Coordinator Manuals Teacher Administration Directions Review your testing tasks for: Test preparation Testing day After Testing

November 4-5, 2002Test Coordinator's Training18 Test Preparation Know who, when, where, and by whom. Communicate test information to parents, students and teachers. Provide training to test proctors and teachers. Prepare students for the test (use practice tests). Be familiar with procedures. Establish and use a Hotline number if people have questions.

November 4-5, 2002Test Coordinator's Training19 Testing Day Minimize disturbances. Check that proctors/teachers are familiar with materials, directions, and procedures. Put up Do Not Disturb signs. Verify identity of examinees. Check answer sheets/booklets for accuracy. Monitor breaks and going to the bathroom. Record all irregularities (test proctor logs) Account for all materials when finished.

November 4-5, 2002Test Coordinator's Training20 After Testing Examine all answer sheets/booklets for tampering, excessive erasures, or other signs of cheating. Investigate all reports of breaches of test security. Account for ALL test materials – answer sheets, booklets, scrap paper, rulers, etc. Follow return directions precisely and return by due date. Know the Hotline number or contact for questions. Mark Leal – EED

November 4-5, 2002Test Coordinator's Training21 Questions & Answers