Chief Learning Officer Webinar Sponsored by Skillsoft September 13, 2012 David Vance

 The basics of running learning like a business  Talent Development Reporting Principles (TDRP) ◦ What is it? ◦ How can it help?  Q&A September 13, 2012 Chief Learning Officer Webinar2

 Help our organizations achieve their goals ◦ Achieve their goals more quickly ◦ Achieve their goals at lower cost  Best accomplished by running learning like a business focusing on ◦ Outcomes (Are we doing the right things?) ◦ Effectiveness (How well?) ◦ Efficiency (How much and at what cost?) September 13, 2012 Chief Learning Officer Webinar3

1. Align learning to business goals 2. Plan it carefully 3. Execute and report with discipline 4. Measure and evaluate September 13, 2012 Chief Learning Officer Webinar4

 Proactive, strategic exercise ◦ Meet with CEO or business unit leader ◦ Learn about next year’s goals and priorities ◦ Learn who the stakeholders are  If not done proactively, at least be sure the requested learning does support the highest- priority business goals ◦ May mean saying “No” to some requests September 13, 2012 Chief Learning Officer Webinar5

 A: Proactively to align learning to business goals  B: Align to business goals as requests for learning come in  C: No process to ensure alignment but most of our learning is aligned to business goals  D: No process and much of our learning is probably not aligned to business goals  E: We really don’t know what our business goals are September 13, 2012 Chief Learning Officer Webinar6

 Meet with stakeholders to understand ◦ Business need ◦ Learning’s role (if any)  Agree with stakeholder on ◦ Specific programs, type of learning ◦ Target audience, location, completion dates ◦ Number of participants ◦ Roles and responsibilities ◦ Measures of success  Expected impact of learning on business goal ◦ Costs September 13, 2012 Chief Learning Officer Webinar7

 If done proactively ◦ Develop a business case for key learning  All costs  Benefits ◦ Create a business plan for learning including  Strategic alignment  Expected results from learning  Business case  Measurement and evaluation strategy ◦ Have plan approved by CEO and senior leaders September 13, 2012 Chief Learning Officer Webinar8

 A: Written business plan for learning approved by the CEO, governing body or other senior leaders  B: Written business plan for learning but not approved by senior leaders  C: PowerPoint business plan for learning approved or shared with senior leaders  D: We have L&D goals but no formal plan. We respond to requests as they come in. September 13, 2012 Chief Learning Officer Webinar9

 Manage your execution to deliver planned outcomes, effectiveness, and efficiency  Review progress monthly within L&D ◦ Year-to-date results versus plan ◦ Will you make plan? ◦ If not, what is the forecast? ◦ What action should you take?  Review progress at least quarterly with CEO, senior leaders ◦ Focusing on most important business goals, key measures September 13, 2012 Chief Learning Officer Webinar10

 A: Report results to our CEO or top-level leader at least quarterly  B: Report results to our SVP of HR or other direct report to the top-level leader at least quarterly  C: Report semi-annually or annually  D: Don’t report results to others September 13, 2012 Chief Learning Officer Webinar11

 Execute your measurement and evaluation strategy. For example, ◦ Level 1 for all courses ◦ Level 2 where appropriate ◦ Level 3 for 10 key programs ◦ Levels 4 and 5 for three key programs  Did you achieve your goals?  If not, what can you learn? What can you improve?  If you exceeded goals, what can you learn? September 13, 2012 Chief Learning Officer Webinar12

 Conceptually not difficult  Challenging in practice  Requires new skills for some, confidence to apply them for others  Some common objections ◦ No standards in place to provide guidance ◦ No internal processes to support it ◦ Not comfortable with business-like approach  TDRp can help September 13, 2012 Chief Learning Officer Webinar13

 Brings principles, standards, definitions and recommended statements and reports to talent development  Answers key questions: ◦ What does the CLO need to run learning like a business?  What data should we collect?  How are the measures defined? ◦ What do the CEO and senior leaders want to see and how should it be presented? September 13, 2012 Chief Learning Officer Webinar14

 Began in fall 2010 by Kent Barnett (CEO, Knowledge Advisors) and Tamar Elkeles (VP of Learning and Organization Development, Qualcomm)  Engaged industry thought leaders like Fitz-enz, Bassi, Phillips, Brinkerhoff, Bersin and CLOs of major organizations  TDRp for L&D completed in March Now being extended to rest of HR. September 13, 2012 Chief Learning Officer Webinar15

September 13, 2012 Chief Learning Officer Webinar16 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology (e.g. Phillips, Brinkerhoff) Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Standard Measures: Outcome measures: What is the impact on the business? Efficiency measures: How much? How many? At what cost? Effectiveness measures: How well?

September 13, 2012 Chief Learning Officer Webinar17 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology (e.g. Phillips, Brinkerhoff) Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Three Statements: Outcome Statement Effectiveness Statement Efficiency Statement

September 13, 2012 Chief Learning Officer Webinar18 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology (e.g. Phillips, Brinkerhoff) Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Three Reports: Summary Report Program Report Operations Report

19 September 13, 2012 Chief Learning Officer Webinar Three standard statements ◦ Outcome ◦ Effectiveness ◦ Efficiency Standard measures are used but choice of measures depends on organization Summary statements show ◦ Last year’s actual ◦ Plan (or goal) for this year ◦ Year-to-date results Detail statements show ◦ Monthly, quarterly, trend data ◦ Granularity ◦ Without plan

 Shows alignment of learning to goals  Expected impact of learning on those goals ◦ Isolated impact of learning (level 4)  Participant’s forecast  Sponsor’s forecast, estimate  Statistical estimate (after the training is completed) ◦ Application (level 3) ◦ Proxies for impact  In some cases may be number of participants ◦ Agreed upon with stakeholder upfront September 13, 2012 Chief Learning Officer Webinar20

September 13, 2012 Chief Learning Officer Webinar

 Kirkpatrick/Phillips five levels ◦ Level 1: Satisfaction, Forecasts of levels 3-5  Participant  Sponsor ◦ Level 2: Amount learned ◦ Level 3: Application rate ◦ Level 4: Impact ◦ Level 5: Value (net benefit or ROI )  Some will have only L1, L2 September 13, 2012 Chief Learning Officer Webinar22

September 13, 2012 Chief Learning Officer Webinar

24 September 13, 2012 Chief Learning Officer Webinar Volume ◦ Participants ◦ Courses, classes, hours -Offered, taken ◦ By ILT, vILT, WBT, etc. Costs ◦ Development, delivery, management ◦ Tuition ◦ Opportunity ◦ Cost reduction due to e-learning Utilization ◦ Courses, classes, instructors ◦ By ILT, vILT Effort to develop and maintain Reach Cycle times ◦ Performance consulting, development, delivery *See a complete list in the Definition of Terms and Measurements document

September 13, 2012 Chief Learning Officer Webinar

26 September 13, 2012 Chief Learning Officer Webinar Three levels of reports ◦ Summary Report (for CEO) ◦ Program Report (for CLO) ◦ Operations Report (for CLO) ◦ Highly customized, pulling the most important measures from the statements Executive-level reports show ◦ Last year’s actual ◦ Plan (or goal) for this year ◦ Year-to-date results ◦ Forecast for this year Detailed reports for managers may show ◦ Monthly, quarterly, trend data ◦ Granularity ◦ Without plan or forecast

September 13, 2012 Chief Learning Officer Webinar

September 13, 2012 Chief Learning Officer Webinar

September 13, 2012 Chief Learning Officer Webinar

September 13, 2012 Chief Learning Officer Webinar

September 13, 2012 Chief Learning Officer Webinar

 Simple, consistent approach to reporting ◦ Use standard definitions to extract and convert data into three types of measures  Outcome, effectiveness, and efficiency ◦ Create three standard statements with these measures  Outcome, Effectiveness, and Efficiency ◦ Create three types of customized reports from these statements  Summary, Program (Initiative), and Operations September 13, 2012 Chief Learning Officer Webinar32

September 13, 2012 Chief Learning Officer Webinar 33 Senior ExecutivesLearning Executives Executive Reports Statements Data Sets Extract, convert and calculate Standard Measures Data Sources Executive Reporting Process Guiding Principles Financial Data Evaluation System Learning Management System Other Sources: HRIS, CRM, ERP Outcome Statement Learning Effectiveness Statement Learning Efficiency Statement OutcomesEffectivenessEfficiency L&D Summary Report Quarterly L&D Program Report Monthly L&D Operational Report Monthly

 Step 1: Align learning to goals ◦ => The Outcome Statement  Step 2: Plan carefully ◦ => The three statements  Step 3: Execute with discipline ◦ => The three reports  Step 4: Measure and evaluate ◦ => All statements and reports September 13, 2012 Chief Learning Officer Webinar

 Running learning like a business ◦ The Business of Learning by David Vance ◦ The Chief Learning Officer by Tamar Elkeles and Jack Phillips ◦ Strategic Learning Alignment by Rita Smith  Resources Available at TDRprinciples.org ◦ 56-page white paper, 18-page overview, 4-page summary ◦ Definition of terms and measures for L&D ◦ PowerPoints ◦ Sample write-up of an L&D Summary Report for a CEO  Contact Dave Vance for more information: September 13, 2012 Chief Learning Officer Webinar35