University of Wisconsin Placement Testing with ExamSoft Readying ourselves for the future Jim Wollack University of Wisconsin-Madison Amy Smith ExamSoft Worldwide, Inc.

ExamSoft Company Overview ExamSoft is a leading provider of end-to-end CBT solutions with 13 YEARS experience delivering ROBUST DESIGN, DELIVERY, SCORING & ANALYSIS TOOLS Used by 250+ Academic, licensing, & certifying institutions Including: Health Professionals Financial Services Academia ~1 MILLION Exams delivered each year 38 of 50 State Bars

Leader in High-Stakes CBT

UW System and ExamSoft The University of Wisconsin System has partnered with ExamSoft Worldwide, Inc. to offer computer based testing for freshman placement exams. The new hybrid testing model maximizes: Flexibility Convenience Efficiency Validity Security

Overview of the UW System 26 campus state system o 13 4-year campuses o 13 2-year transfer institutions o Distinct from statewide technical colleges system

Placement Testing and the UW System Placement testing is a critical element in establishing uniformity and facilitating transfers between institutions

The UW Placement Testing Model Math, English, French, German, and Spanish Tests developed collaboratively across all 26 UW campuses o Content of tests is reflective of UW curricula o New forms are published every year Same tests used for all campuses o Cutscores and other particulars are campus-specific

UW Center for Placement Testing Center coordinates test development, validation, and administration systemwide Center is funded by all campuses o Each campus’ contribution is proportional to the size of campus’ incoming freshman class o Budgetary oversight by a Systemwide Advisory Board  6 Provosts  5 chairs of placement test development committees  1 UW System Vice President  Center Director, ex officio

Center’s Testing Cycle JanMarFebAprMayJunJulAugSepOctNovDec Regional Testing Orientation Testing Residual Testing Regional Testing Orientation Testing Residual Testing Tests offered at all UW campuses on fixed dates (mostly on Saturdays) Students test at any campus on specified date 69% test at one campus but attend another Center’s budget covers costs for Reg. Testing Summer Orientation testing is a special case of Residual Testing 5 ½ hours during orientation At campus’s own expense Each campus handles their own Resid. Testing Conducted at campus’ own expense Models for Resid. Testing vary across campuses Students: 13,047 Exams: 30,760 Students: 10,538 Exams: 20,525 Students: 7,049 Exams: 10,512

Challenges Today Capacity Late admits Out-of-state students & International students Student who missed regional testing Logistics Scheduling 130 administrations between June & August Orientation impact (5 ½ hours) Cost Residual testing costs extra Access Limited to campus testing Not all campuses offer residual testing Administrative Challenges Security Must have a proctored environment University System Total Legend > 20,000 > 1,000 < 1,000 < 100 Students: 30,634 Exams: 61,797

Initial Concerns with CBT Security—exams must be proctored Cost o Exam is currently free for students o Start-up/maintenance costs to campuses Switch to CBT may be disruptive for some campuses o Reliability/process with service failures

Changes for 2012 JanMarFebAprMayJunJulAugSepOctNovDec Regional Testing (CBT and PP) Computer Based Testing (Testing Centers) Residual Testing (CBT & PP) Convenient Year Round CBT Expanded Regional Dates Enhanced Residual Testing Large scale CBT (reduced fee) Continued P&P (free) Extended dates through July on more campuses provides greater access to free tests More dates: CBT-only and P&P-only dates Test at a nearby testing center (additional fee) Students can test anytime All test are proctored Offer CBT (additional fee) Cover proctor costs (CBT) P&P still available New International Options Take proctored CBT exams at local locations (additional fee) International Computer-based Testing

Overcoming CBT Concerns Flexibility o Test on and off campus (domestic & international) o Deliver exams on any computer, including student-owned laptops Security o all exams are fully secure and proctored Costs o Designed an equitable partnership to keep costs at a minimum for students Support & Services o Full support for students & administrators o Custom built exam registration site Reliability o Unique delivery method eliminates internet dependence during the exam

Reasons out of state students can’t use regional testing Flight$382 (per person)$763 Hotel$99 (person)$297 Car$42 (per day)$126 Expenses$100 (per day)$300 Total$1486 Students: 30,634 Exams: 61,797 University System Total Student Access Scheduling Overcommitted Tend to procrastinate Changing schedules Capacity Limited dates Limited seats Access Restrained by campus testing only Travel Student Challenges Legend > 20,000 > 1,000 < 1,000 < 100 Incremental cost for out of state students ( Parent & Student). Hotel$99 (person)$99 Car$42 (per day)$42 Expenses$100 (per day)$100 Total$241 Convenience Scheduling Costs Travel Student Challenges

Significantly Expanded Coverage On Campus Off Campus Even Globally

How is this possible? ExamSoft Rethink Traditional CBT

How the Software Works Auto Upload SECURE ANSWERS Issue Keys FOR EXAMS Download SECURE FILE Creates & Posts the Exam Wisconsin Students Register online for their exams Prior to the Exam Encrypted exam file is downloaded During the Exam Internet, applications, and local files are block. After the Exam Encrypted exam file is uploaded Wisconsin Students Results of exam delivered Analyzes exam results

Change/Home Registration/Home Special/Home Send to Human Registration Form Where would you like to take your exam On Campus (CBT)On Campus (P&P)Off Campus (Testing Ctr)Off Int’l Select Location Select Date & Time Shopping Cart Confirm Select Location Select Date & Time Confirm Select Exam Get Center (API) Get Date (API) Get Appointment (API) Save/Another Exam Shopping Cart Confirm/Schedule Appointments (API) Basic Form Shopping Cart Int’l Download Screens Proctor Form/Download Screen Ch. Date/Time (Campus) Ch. Date/Time (ACT) Ch. Location/ Channel - Instructions Buy Extra Exam Slots (ACT) Retake Instructions Help/FAQs Login Update RSP Other times New Student Registration Site

New Placement Exam Site

1.Streamline the process for all methods of testing 2.Increase capacity and options 3.Robust item banking tool for CBT and P&P exams with item statistics 4.Greater control and security 5.Increase flexibility 6.Easier scoring and reporting 7.Enhance cost recovery without mandating 8.Improve students’ first impression of UW system by offering technology and convenience options for first assessments Benefits to the University

1.Expanded regional testing gives greater access to the free test 2.Take test(s) at a location closer to home, reducing travel expenses for international and out of state students 3.Take test(s) at a convenient time, scheduling appointments for exams on dates and times that work within students’ busy academic and athletic schedules 4.CBT could provide an option for immediate feedback on exams 5.Take tests early - begin to register for courses earlier, which makes orientation easier and more fun 6.Use the latest technology to take the test(s) – and test with the comfort of your own laptop if you desire Benefits to the Students

Why We Chose ExamSoft’s Model Proven security and reliability - No “server is down” concerns during the exams Viable economic model for CBT migration Robust item banking and scoring tools Superior customer support Personalized client services - above and beyond Tremendous flexibility o Students get comfort of testing on their own computers o Delivery model greatly reduces normal capacity issue associated with CBT o Allows us to entertain moving towards inexpensive CBT for WI students by changing some Regional Testing dates to be CBT sites o Possibility of using ExamSoft for classroom testing

Sharing this Model Jim Wollack Amy Smith

Appendix

How the Software Works Auto Upload SECURE ANSWERS Internet Availability (Testing Center or after exam) Issue Keys FOR EXAMS Secure Test Center or Campus Location Download SECURE FILE Internet Availability (home, dorm, hotel, Starbucks)

Unprecedented Test Site Flexibility 230 State-of-the-Art Testing Centers

Deeper Look at Laptop Security Over 60 security check processes running during the exam!! Completely lock down the laptop at operating system level Block internet access System level tracking of exam taker activity No access to local files, applications Forced exit of exam and lock out of answer files

Easy, Intuitive Item Banking

Question Navigation is easy by using these icons. Easy, Reliable, Secure Exams

Item Validity and Reliability

Robust Reporting

Benchmarking Students Benefits: Measure & drive learning outcomes with remediation, self directed learning, teaching/ curriculum changes, etc. Provide valuable data for accreditation process

Web-Based Essay Scoring View questions here Review exam taker response here Provide grader feedback here Enter points here

Close the Loop – Item Analysis

Support From a Partner You Trust