Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager Date 18th July 2011 Version 2.0 Not protectively marked 1

Overview of IPS Identity and Passport Service (IPS) is a Public Sector Organisation  Approximately 3800 employees spread across the UK  Core Business Products: ─ Passports & Identity Management ─ Birth, Marriage, Death & Civil Partnership Certificates 2

Overview of Learning & Development Function in IPS Centralised Learning & Development function with a wide geographical remit. Classroom trainer-led events E-learning delivered by Kallidus Learning Management System (LMS) Blended learning solutions managed via the Learning Management System (LMS). Approximately days of training delivered for financial year

5 Levels of Evaluation Model used in IPS (Jack Philips Model)

Learning Measurement & MTM Strategy MTM is integrated with the IPS LMS –Estimate, Isolate & Adjust technique used by MTM All training courses including some e-learning are evaluated Internal targets and an Organisational Development Score Card has been set to drive strategy Three key indicators that are monitored monthly: –Trainer Performance –ROI: Cost Benefit Ratio –Learning Impact & Value (business results) IPS targets are incorporated into KPOs for L&D staff who are performance managed with these 5

Learning Measurement & MTM Strategy Implementing Tactical Effectiveness Time IPS-Strategic Optimized  Collecting evaluation data and measure business results  LMS dashboards presentation to business  Presenting monthly analyses to business  Collecting evaluation data and measure business results  LMS dashboards presentation to business  Presenting monthly analyses to business Comprehensive 6

1 st Quarter of 2010 Figures AprilMayJune No of Events No of Delegates Training days Average delivery per trainer (Target: 100 days per yr i.e. 9.0 average per month) Cost benefit Ratio (Target: £1:3.00) £ 1:1.74£ 1:1.86£ 1:2.44 Average Trainer Score (Target: 6.25) Evaluation Returns (Target: 65%) 52.02%58.08%56.78% 7

Impact on Business To improve impact and value to business: information is displayed on the LMS dashboard 8

Practical Approach to Data Analysis 9 MacroMicro Evaluation Data Analysis

Review & Analysis Cycle Collect/Measure Training Results Review Evaluation Data Take Action Measure Results 10

Human Capital ROI Card Business Results ROI Card Instructor Performance Summary Report Training Improvement Summary Report by Trainers Project Initiation Documents/Action plan by Training Managers Course Effectiveness Summary Report (Director & board Level) Talent Development Executive Report (Director & board Level) 11 Reports Used for Practical Analysis

Challenges & Tips There was initial resistance when the system was implemented as it was a cultural change IPS has put in place an Organisational Development balance score card accepted at board level The internal benchmark targets have been incorporated into the organisations Key Performance Indicators Align evaluation strategy to business objectives Note: Its is a Process not a project expect results over time. Simple continuous marketing of the benefits to business units helps improve evaluation return rates 12

Summary Aspirations- Evaluation to become second nature and part of the culture MTM is providing an invaluable resource for: –demonstrating the worth of training to organisation –provides performance information for making crucial strategic decisions –Could have a positive impact on the bottom line of any organisation

Questions

Contact and Further Information Andrew Somiah Systems Training Manager Identity and Passport Service HQ Learning & Development 4th Floor Mailpoint B NW Peel Building 2 Marsham Street London SW1P 4DF T: E: Stuart Halliday EMEA Sales Director KnowledgeAdvisors P: +44 (0) M: +44 (0)