1. 1 23 rd Annual Conference September 11-13, 2003 Westin Harbour Castle Toronto, Canada

2. 2 Technology, Service and Delivery September 12 th, 2003 1:30 – 3:00 p.m. Dan Luk CTO, Capstar

3. 3 First Topic: “Point and Click – Using the Internet to Enhance your Testing Program”

4. 4 Why the Internet? Convenient Fast 24 X 7 Efficient Cost-Effective

5. 5 Why Not the Internet? Security Control Fairness

6. 6 Web Usage and Growth Source http://www.nielsen-netratings.com/new.jsp on 1/8/2003http://www.nielsen-netratings.com/new.jsp

7. 7 When to use the Internet? Pre-Test Post-Test

8. 8 Test Development Web-based Test Development Tools Collaborative Computing Item Development and Review Resolve Geographical Differences Security Authentication Access Restrictions Encryption

9. 9 Communications Types of Technology Push Technology e.g. e-mail Pull Technology e.g. Web Site

10. 10 Push Technology Main Advantages Selective or Targeted Customized Contents Concise Main Disadvantages “Spam” Frequent Address Changes

11. 11 Pull Technology Main Advantages General Availability Broad-based Information Information Available On-Demand Consistent Messaging Interactive Contents Main Disadvantages Must be Easy to Navigate Must Visit Frequently to View New Contents Random Arrival

12. 12 Types of Information Bulletins Application Procedures Application Forms Study Materials Practice Exams Etc.

13. 13 Registration/Scheduling Self-Service Vs. Call Center Accessibility Skills Online Help Vs. Help Desk High Availability Easy to Review Information or to Make Changes Implementation Methods Real-time Back-end Processed

14. 14 Registration/Scheduling (Continued) Business Rule Enforcement Upper Vs. Lower Case Data Consistency Vs. Data Cleansing Browser Compatibility ADA Accommodation such as Section 508 Compliancy E-commerce (Electronic Payment Required) Identity Theft

15. 15 Test Delivery High Stake Vs. Low Stake Anyone Can Test Vs. Positive ID Proctored Vs. Self-proctored Connection Performance Timed Exams Test Driver Features Test Security Browser Compatibility

16. 16 Data Management On-Demand Vs. FTP Dynamic Vs. PDF Registration/Scheduling Records Eligibility Processing Test Reporting Psychometric Analysis

17. 17 Capstar Internet Usage Exceeded Expectations Anticipate Continued Growth Continued Area of Focus

18. 18 Second Topic: “Innovative Uses of Technology In Testing”

19. 19 When to use Technology? Pre-Test Post-Test

20. 20 When to use Technology? Lie! Cheat! Steal!

21. 21 Lie Whom Are You Really Testing?

22. 22 Lie Typical Preventive Measures Policies and Procedures as Deterrent Positive Identification Via Proctor

23. 23 Traditional Identification Techniques Government Issued Pictured Ids ATT Tamper Resistance Paper and Printing Smart Cards Holographic Images Microprints Bar Code Data Strip

24. 24 Biometrics ‘Recently, the term "Biometrics" has also been used to refer to the emerging field of technology devoted to identification of individuals using biological traits, such as those based on retinal or iris scanning, fingerprints, or face recognition.’ http://stat.tamu.edu/Biometrics/

25. 25 Biometrics (Continued) Access to Testing Area Authentication Process Re-entry to Testing Area (Same Day Vs. Future) Re-authentication Process (Same Day Vs. Future) Initial Testing Vs. Repeat Testing Same Person Vs. Different Person

26. 26 Cheat Typical Preventive Measures Policies and Procedures as Deterrent Rules for Taking Breaks Business, Eligibility or Re-take Rules Proctored Exams Clean Test Rooms

27. 27 Traditional Monitoring Techniques Active Proctoring Proctor Window Video and Audio Surveillance

28. 28 Other Monitoring Techniques Remote Proctoring Remote Video and Audio Surveillance Back-end Monitoring Extreme Low Score Extreme High Score Sudden Score Fluctuation Time-to-respond Item Performance Form/Test Performance DB Cross-checking

29. 29 Other Monitoring Techniques Biometrics Candidate’s Photo Access to Testing Area Authentication Process Re-entry to testing area Re-Authentication Process

30. 30 Steal Reasons for Prevention to Include: Intellectual Property Protection Exam Integrity Public Safety

31. 31 Steal Typical Preventive Measures Policies and Procedures Business, Eligibility or Re-take Rules Rules for Taking Breaks Active Proctoring Proctor Window Video and Audio Surveillance

32. 32 Other Techniques Remote Proctoring Remote Video and Audio Surveillance Back-end Monitoring Extreme Low Score Extreme High Score Sudden Score Fluctuation Time-to-respond Item Performance Form/Test Performance DB Cross-checking

33. 33 Other Techniques (Continued) Initial Testing Vs. Repeat Testing Same Person Vs. Different Person New Items New Forms Continuous Testing Vs. Windowed Testing Software Algorithms Randomization Computer Adaptive Item Algorithms

34. 34 Trade-off Cost Security

35. 35 Other Technologies Used in Testing Improved Paper-and-pencil Process Dynamic Scanning and Scoring Fax-based Scoring Simulation-based Testing Innovative Item Types Computer-based Practical Testing 3-D Display

36. 36 Other Technologies Used in Testing (Continued) Standards SCORM IMS QTI Other Testing Devices PDAs Tablet PCs IVR/Cell Phones Alternate Devices Wireless Technologies

37. 37 Q & A

38. 38 Presentation Follow-up Please pick up a handout from this presentation –AND/OR- Please give me your business card to receive an e-mail of the presentation materials –OR- Presentation materials will be posted on CLEAR’s website

39. 39 Technology, Service and Delivery September 12 th, 2003 1:30 – 3:00 p.m. Dan Luk CTO, Capstar