Recovering from the Unexpected 22nd Annual CLEAR Conference Las Vegas, NVSeptember 12, 2002 Ron Rodgers, Ph.D. Director of Measurement Services, Continental Testing Services (CTS) & President, Employment Research Institute

2 What do the Joint Standards Say? Test Security Test items on secure tests are not published; unauthorized copying is forbidden by … anyone. A secure test is not for publication in any form, in any venue Standards, page 183

3 What do the Joint Standards Say? Test Administration, Scoring & Reporting 5.6Reasonable efforts should be made to assure the integrity of test scores by eliminating opportunities for test takers to attain scores by fraudulent means. 5.7Test users have the responsibility of protecting the security of test materials at all times Standards, page 64

4 What do the Joint Standards Say? Rights & Responsibilities of Test Takers 8.7Test takers should be made aware that having someone else take the test for them, disclosing confidential test material, or any other form of cheating is inappropriate and that such behavior may result in sanctions Standards, page 88

5 What do the Joint Standards Say? Rights & Responsibilities of Test Takers 8.10When an individual score report is expected to be delayed … because of possible irregularities such as suspected misconduct … efforts should be made to expedite the review and protect the interests of the test taker Standards, page 89

6 What do the Joint Standards Say? Rights & Responsibilities of Test Takers 8.11When scores are cancelled or withheld because of possible testing irregularities, including suspected misconduct, … test takers should be given a timely opportunity to provide evidence that the score should not be cancelled or withheld Standards, page 89

7 What do the Joint Standards Say? Rights & Responsibilities of Test Takers 8.12When testing irregularities are suspected, reasonably available information bearing directly on the assessment should be considered consistent with the need to protect the privacy of test takers Standards, page 89

8 What do the Joint Standards Say? Rights & Responsibilities of Test Takers 8.13Test takers are entitled to fair consideration and reasonable process … in resolving disputes about testing. Test takers are entitled to be informed of any available means of recourse Standards, page 89

9 What do the Joint Standards Say? Responsibilities of Test Users 11.7Test users have the responsibility to protect the security of tests to the extent that developers enjoin users to do so. 11.8Test users have the responsibility to respect test copyrights Standards, page 115

10 What do the Joint Standards Say? Responsibilities of Test Users 11.9Test users should remind test takers and others who have access to test materials that the legal rights of test publishers … may prohibit the disclosure of test items without specific authorization Standards, page 115

11 What do the Joint Standards Say? Responsibilities of Test Users 11.11If the integrity of a test taker’s scores is challenged, (sponsors) should inform test takers of their rights, including the possibility of appeal and representation by counsel Standards, page 115

12 Candidate Information If irregular behavior has occurred during a test, the (sponsor) will make every effort to withhold the scores of only those candidates directly implicated in the irregularity. Nevertheless, in some instances … the (sponsor) may not be able to identify the specific candidates involved in the irregularity. Scores of candidates not directly involved in the irregularity may be required to take an additional examination at a later date to ensure the validity of all scores. Candidate Information Bulletin, Certified Software Development Professional, 2002

13 Recovering from Harvested Items 1.Is retesting candidates who were administered tests with harvested items the only alternative? 2.How can you determine who among past candidates may have assisted in harvesting test items? 3.What action can be taken against candidates who already are licensed and may have harvested items? 4.How do you determine who may have used or benefited from harvested items? 5.How can a school that distributes harvested items or practice tests be sanctioned?

14 What are risks of doing nothing? Client’s attorneys may respond that “everybody does it for the Bar exam, so why does it matter?” 1.Bar Exams require complex reasoning and essays that are difficult to harvest accurately. 2.Many licensure exams rely primarily on recall of basic sanitation facts and procedures designed to protect public health and safety. 3.Licensure exams often are the primary means to protect public from applicants who may not be adequately skilled to practice safely – especially in occupations open to applicants with limited training.

15 What are risks of doing nothing? What Price Beauty - Its Your Choice Real Stories - Real Information Fox 31 Denver with Kristina Fox 31 Denver with Kristina 20/20 Investigation Washington Post NBC Philadelphia Story The Web MD CBS 2 News American Academy of Dermatology HealthBeyondBeauty.com

16 What are risks of doing nothing?  Nightmare Manicure May 29, 2002 A Westminster, Colo., woman won a jury award of $3.1 million against a nail salon that she claims gave her herpes through the use of non-sterile instruments. Good Morning America (ABC-TV), WGN National News  Woman's Finger Amputated After Getting Nails Done After Infection Sets In April 23, 2002 A Kansas City woman won a settlement against a nail salon after she was forced to amputate her index finger after getting her nails done. KMBC-TV, Kansas City

17 What are risks of doing nothing?  A 20/20 investigation found that 24 of 27 discount salons tested by a laboratory that specializes in infection and disease control had possible disease-causing fungi and bacteria.  California investigators found similar bacteria at 16 of 18 salons. 20/20 also went with inspectors to manicure stations in Dallas, Phoenix, Boston and Houston. Almost all tested positive for potentially harmful bacteria. 20/20, ABC TV, 2002

18 What are risks of doing nothing? §News2Houston Hidden cameras caught the illegal use of cutting blades on feet, the use of unsanitary nail utensils and then beauticians shared the same instruments with different customers. §According to the Texas Cosmetology Commission, each of the five nail salon inspectors in the Houston area is responsible for inspecting 1,000 to 1,400 salons, which makes it virtually impossible for each salon to be inspected yearly.Texas Cosmetology Commission News2Houston – Hidden Camera Investigations, 1999 & 2002