1. Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager Date 18th July 2011 Version 2.0 Not protectively marked 1

2. Overview of IPS Identity and Passport Service (IPS) is a Public Sector Organisation  Approximately 3800 employees spread across the UK  Core Business Products: ─ Passports & Identity Management ─ Birth, Marriage, Death & Civil Partnership Certificates 2

3. Overview of Learning & Development Function in IPS Centralised Learning & Development function with a wide geographical remit. Classroom trainer-led events E-learning delivered by Kallidus Learning Management System (LMS) Blended learning solutions managed via the Learning Management System (LMS). Approximately 589.6 days of training delivered for 2009-2010 financial year

4. 5 Levels of Evaluation Model used in IPS (Jack Philips Model)

5. Learning Measurement & MTM Strategy MTM is integrated with the IPS LMS –Estimate, Isolate & Adjust technique used by MTM All training courses including some e-learning are evaluated Internal targets and an Organisational Development Score Card has been set to drive strategy Three key indicators that are monitored monthly: –Trainer Performance –ROI: Cost Benefit Ratio –Learning Impact & Value (business results) IPS targets are incorporated into KPOs for L&D staff who are performance managed with these 5

6. Learning Measurement & MTM Strategy Implementing Tactical Effectiveness Time IPS-Strategic Optimized 1 2 3 4 5  Collecting evaluation data and measure business results  LMS dashboards presentation to business  Presenting monthly analyses to business  Collecting evaluation data and measure business results  LMS dashboards presentation to business  Presenting monthly analyses to business Comprehensive 6

7. 1 st Quarter of 2010 Figures AprilMayJune No of Events 858490 No of Delegates 322395650 Training days 59.979.938.5 Average delivery per trainer (Target: 100 days per yr i.e. 9.0 average per month) 2.76.35.4 Cost benefit Ratio (Target: £1:3.00) £ 1:1.74£ 1:1.86£ 1:2.44 Average Trainer Score (Target: 6.25) 5.416.376.43 Evaluation Returns (Target: 65%) 52.02%58.08%56.78% 7

8. Impact on Business To improve impact and value to business: information is displayed on the LMS dashboard 8

9. Practical Approach to Data Analysis 9 MacroMicro Evaluation Data Analysis

10. Review & Analysis Cycle Collect/Measure Training Results Review Evaluation Data Take Action Measure Results 10

11. Human Capital ROI Card Business Results ROI Card Instructor Performance Summary Report Training Improvement Summary Report by Trainers Project Initiation Documents/Action plan by Training Managers Course Effectiveness Summary Report (Director & board Level) Talent Development Executive Report (Director & board Level) 11 Reports Used for Practical Analysis

12. Challenges & Tips There was initial resistance when the system was implemented as it was a cultural change IPS has put in place an Organisational Development balance score card accepted at board level The internal benchmark targets have been incorporated into the organisations Key Performance Indicators Align evaluation strategy to business objectives Note: Its is a Process not a project expect results over time. Simple continuous marketing of the benefits to business units helps improve evaluation return rates 12

13. Summary Aspirations- Evaluation to become second nature and part of the culture MTM is providing an invaluable resource for: –demonstrating the worth of training to organisation –provides performance information for making crucial strategic decisions –Could have a positive impact on the bottom line of any organisation

14. Questions

15. Contact and Further Information Andrew Somiah Systems Training Manager Identity and Passport Service HQ Learning & Development 4th Floor Mailpoint B NW Peel Building 2 Marsham Street London SW1P 4DF T: 0207 0356 798 E: andrew.somiah@ips.gsi.gov.ukandrew.somiah@ips.gsi.gov.uk Stuart Halliday EMEA Sales Director KnowledgeAdvisors P: +44 (0) 203 371 1165 M: +44 (0) 771 7507 945 shalliday@knowledgeadvisors.com