Assessments & Testing 4 July 2004CAA Conference1 of 19 Implementing large scale and rapid growth assessment programmes Suzana Lopes Commercial Director

Assessments & Testing 4 July 2004CAA Conference2 of 19 Presentation Outline Pearson Company Overview Case Studies: Large Scale Assessment Programme  DSA – Driving Standards Agency (UK)  DVTA – Driver and Vehicle Testing Agency (Northern Ireland) Rapid Growth Assessment Programme  City and Guilds

Assessments & Testing 4 July 2004CAA Conference3 of 19 Pearson Company Overview Pearson plc £4 billion annual sales World’s # 1 education business Over 30,000 employees Pearson VUE - The UK Market Leader in e-Assessment Acquired Promissor in January 2006 to expand assessment service Flexible and robust content development and delivery solutions Test centres in 145 countries and easy to deploy customer site testing A commitment to quality and outstanding service

Assessments & Testing 4 July 2004CAA Conference4 of 19 Case Study: The Driving Theory Test Contract awarded 31 October 2003 by Joint Authorities Our contract is for delivery of theory tests Nearly 1.8 million tests delivered annually Learner tests for:  Car  Motorcycle  Truck  Buses  Driving instructor tests  Driving examiner tests

Assessments & Testing 4 July 2004CAA Conference5 of 19 What needed to be achieved Establish a contact centre to handle:  Telephone bookings  Mail and fax bookings Build DDA compliant test centres at 151 locations  England  Scotland  Northern Ireland  Wales Build and publish the tests Develop bespoke booking and reporting systems Real-time eligibility checks Hire, train and transfer staff from previous vendor

Assessments & Testing 4 July 2004CAA Conference6 of 19 What Needed To Be Achieved - The Timeframe

Assessments & Testing 4 July 2004CAA Conference7 of 19 Test Centres Requirement: 80% of the UK population to be no more than 20 miles from their nearest test centre Difficult to find DDA-compliant (accessible) locations in some areas Northern Scotland was particularly a problem Decision was made to use coaches as mobile test centres Eventually found locations for 151 test centres  Space that worked with the test centre plans  Safe and appropriate  Easily accessible by public transport Flexible opening hours

Assessments & Testing 4 July 2004CAA Conference8 of 19 Publishing The Tests Requirement: Publish tests in English and Welsh - with voice overs in 20 languages and available in British Sign Language First understand the presentation and scoring requirements Multiple choice and hazard perception components Thousands of the items built into a variety of tests Multiple forms of each test Hundreds of video clips for HPT test with specialized scoring Inventory the items and materials carefully Identify missing components Build the tests Test, test and test

Assessments & Testing 4 July 2004CAA Conference9 of 19 Test Centre Design and Test Delivery Requirement: Design highly secure test centre with invigilator station CCTV and audio monitoring and biometrics during check in process Produced working prototype of test centre meeting all requirements Opened on 1 March 2004 Loaded Hazard Perception Test video clips on 30 stations to demonstrate smooth operation Test registration and scheduling Standardised procedures at the centres Hired contractors to kit out 151 test centres and mobile centres Hired and trained staff

Assessments & Testing 4 July 2004CAA Conference10 of 19 Example Test Centre Floor Plan

Assessments & Testing 4 July 2004CAA Conference11 of 19 Critical Success Factors Excellent Project Management Working groups  Software  Testing centres  Contact centre  Test publishing  Security Senior business group Experienced test development and delivery staff  Members from Pearson VUE, DSA and DVTA Weekly update meetings

Assessments & Testing 4 July 2004CAA Conference12 of 19 Rapid Growth Assessment Case Study: City & Guilds 125 year old not for profit company Royal Charter UK’s largest vocational awarding body 1.2 million learners 8500 centres 600 awards 102 Countries Operating in 22 sectors

Assessments & Testing 4 July 2004CAA Conference13 of 19 Overview 2001 City & Guilds decided to launch assessment and learning portal Plan to move centres to online registration and booking and develop computer based tests Global search for test partner Detailed specification Reliability, scalability and flexibility ranked highly among the attributes required 6 companies competed Promissor awarded the contract in 2002

Assessments & Testing 4 July 2004CAA Conference14 of 19 Attributes Of The Market Almost entirely paper based. Candidate choice was limited but growing Centres were cost focussed Constant policy and funding changes Backdrop of slow uptake of on line services across industry

Assessments & Testing 4 July 2004CAA Conference15 of 19 Test Development Deployment team established CBT item writers appointed by City and Guilds Training in Promissor Builder  Easy to Use – non expert interface  Web based tool – collaborative  Track and view content changes  Store multiple classification schemes  Create tests based on balanced content criteria  Allows randomized tests, test pooled from an item pool and adaptive test

Assessments & Testing 4 July 2004CAA Conference16 of 19 Test Delivery Customers consulted about design of system Regional training events held for customers Online training provided to use registration system Documentation provided for surveillance and invigilation at centre Technical support line Customer service support Centre Approval Promissor provide Centre Criteria Test Centre Installation Test Centre Demo Test Promissor make LIVE

Assessments & Testing 4 July 2004CAA Conference17 of 19 Solution Overview (ENTERPRISE) SITE MANAGER Candidate Checks-in Booking Test Administrator Test Administrator Test Administrator Candidate Takes Test Test Published for Delivery Item Bank BUILDER Candidate Registration C&G System Scheduling Candidates scheduled on-line Results C&G System Test Centre Set-up Test Centre

Assessments & Testing 4 July 2004CAA Conference18 of 19 Rapid Growth 2002: 200 tests delivered 2003: Plan to move 80% of objective assessment to CBT   65k tests delivered 2004: 187k tests delivered k test per month Over 250 computer based tests developed 6000 registered centres

Assessments & Testing 4 July 2004CAA Conference19 of 19 Critical Success Factors Senior management support Efficiency and value has to be proven internally as well as externally On-screen must be mandatory Technology right first time Training must be easily available Extensive support is needed