Using Talent Development Reporting Principles (TDRp) to Run Learning Like a Business 2013 Skillsoft Perspectives Orlando May 13, 2013 Dave Vance, Executive Director, CTR Dawn Campbell Borland, TD Bank

Workshop Objectives 2 May 13, 2013 Skillsoft Perspectives At the conclusion of the workshop, you will be able to: Articulate the principles of TDR and its benefits to your organization Identify the type of measures you should gather in your organization to populate the statements and reports Explain the Talent Development statements and reports and why each is important Understand how the statements and reports can help you rum learning like a business Appreciate the change that will be required to implement TDRp and run learning like a business Get started!

Agenda & Guidance 3 May 13, 2013 Skillsoft Perspectives Overview »TDRp background and context »Running Talent like a business »Executive reporting framework Break The Specifics »Recommended measures »TDRp Statements & Reports Managing the Change Learn More Wrap Up Our guidance to you »Maintain an aspirational view »Think about what you can do in 2013 to make progress The perfect is the enemy of the good. --- Voltaire

Interactive Introductions 4 May 13, 2013 Skillsoft Perspectives Organization size ( 100K employees) Sector (Private, public) Organization (L&D, HR, Business) Role (Individual contributor, manager/director, senior leader) Experience with Executive Reporting (yes, no) # of L&D Employees ( =250) Structure of L&D (centralized, decentralized, hybrid) Common Evaluation Platform (Yes, No)

TDRp Background & Context 5 May 13, 2013 Skillsoft Perspectives

Why TDRp? 6 May 13, 2013 Skillsoft Perspectives Need for commonly-agreed upon standards for measuring, reporting, and managing human capital Need to run talent like a business: deliver planned, agreed-upon results that help achieve company goals Need to clearly show alignment and value of human capital initiatives TDRp is as much about management as it is about measurement »Measurement, analysis and reporting are all means to the end »The end is effective, efficient management to deliver important outcomes

Talent Development Reporting Principles (TDRp) Bring standardization and consistency to talent development for »Internal reporting »Demonstrating the value of human capital »Running talent like a business Answer the common questions »What to collect »How to define it »How to report it »What to do with it Leveraging work done by others 7 May 13, 2013 Skillsoft Perspectives

A Brief History of the Movement May 13, 2013 Skillsoft Perspectives 8 Launched in Oct by Kent Barnett, CEO of Knowledge Advisors and Tamar Elkeles, VP of Learning & Communication at Qualcomm Industry thought leaders engaged Leading industry practitioners form Executive Council Knowledge Advisors provides initial funding Development of TDRp for L&D First whitepaper approved by Council Presentation at Learning Analytics Symposium TDRp extended to all key talent processes Presented to over 1500 people ASTD ICE and Singapore Creation of Center for Talent Reporting (October) New website goes live Membership Credentialing, Accreditation available Workshops Webinars User group conference SHRM/ANSI standard

The Center for Talent Reporting: The Home of TDRp Established October 2012 »Not-for-profit, 501c(6) organization (used for trade associations) Funded by sponsors and members Governed by nine-member Board of Directors »Including a Standards Committee to provide continued guidance on standards and reporting and an Advisory Council Mission »Develop and implement reporting standards for human capital »Business model: Others to provide consulting and products 9 May 13, 2013 Skillsoft Perspectives

Plans for May 13, 2013 Skillsoft Perspectives Two-day workshops offered in 2013 »Feb Denver »Mar Irvine »Jun 6-7 Alexandria, VA »Oct Atlanta Webinars offered monthly »Intro to TDRp »Implementation CSFs »Implementation Guidance (members only) First TDRp conference planned for October 17 Certification »Individuals »Vendors providing software products employing TDRp Accreditation »Organizations implementing TDRp »Consultancies providing services SHRM will establish a taskforce in 2013 to consider recommending TDRp for L&D as a SHRM/ANSI standard

11 Sponsors We want to thank the following sponsors for their support of the Center for Talent Reporting. Center For Talent Reporting FOUNDING ORGANIZATION SILVER SPONSORS BRONZE SPONSORS 9-May-13

Running HR (Talent) Like a Business 12 May 13, 2013 Skillsoft Perspectives

Why does the talent function exist? Why do organizations invest in: »Learning and Development »Leadership Development »Capability Management »Performance Management »Talent Acquisition »Total Rewards First: Agree on Our Role May 13, 2013 Skillsoft Perspectives 13

First: Agree on Our Role Help our organizations achieve their goals »More quickly »At lower cost »More effectively Best accomplished by running HR like a business focusing on: »Outcomes (Are we doing the right things?) »Effectiveness (How well?) »Efficiency (How much and at what cost?) May 13, 2013 Skillsoft Perspectives 14

1.Identify the business goals 2.Align HR initiatives to business goals and plan the initiatives carefully 3.Execute and report with discipline 4.Measure and evaluate May 13, 2013 Skillsoft Perspectives Four Steps to Run HR Like a Business 15

Step 1: Identify Business Goals 16 May 13, 2013 Skillsoft Perspectives Proactive, strategic exercise »Meet with CEO or business unit leader »Learn about next year’s goals and priorities »Learn who the sponsors are Results from Step 1 »List of CEO-prioritized SMART goals and sponsors We need specific, measurable goals expressed in terms of change from the previous year Example »The goal is to increase sales from $50M to $55M -Goal: Increase sales by $5M or 10% »The goal is to increase the employee engagement score from 60 points to 63 points -Goal: Increase engagement by 3 points (don’t say by 5% since some may interpret that to be 5 percentage points)

Step 2: Align HR Initiatives to Business Goals; Plan the Initiatives Carefully 17 May 13, 2013 Skillsoft Perspectives Meet with sponsors to understand »Business need in greater depth, the challenge, past success & failure »HR’s role (if any) If HR has a role, agree with sponsor on »Specific initiatives, programs, systems, or processes »Target audience, location, completion dates »Number of participants »Roles and responsibilities for sponsor and HR »Measures of success: Expected impact of initiative on business goal Expected impact of HR on business goal »This is the contribution of just the talent initiative(s) on the business goal »In L&D terms, it is the expected isolated impact of learning. In other words, it is a forecast of the level 4 (impact) »Our aim: a reasonable, roughly right forecast that is mutually agreed upon between the sponsor and HR. It needs to be “close enough” to help us make the right decision about whether the initiative is worth pursuing »Both parties need to own it

The business goal: 10% increase in sales for next year »L&D and the sponsor agree training could help achieve the 10% increase »Specific programs, target audiences, completion dates are agreed »What portion of the 10% increase might reasonably come from this agreed-upon program? »Perhaps 20%. »Then the expected isolated impact of sales training on the goal is 20% x 10% = 2% increase in sales due to training Step 2: Align HR Initiatives to Business Goals; Plan the Initiatives Carefully May 13, 2013 Skillsoft Perspectives 18

Reach agreement on the expected isolated impact »Ask sponsor to list all drivers of higher sales »Now ask sponsor to prioritize the drivers »Then ask sponsor to list the expected % contribution of each. Work the percentages until they add up to 100% »The percentage next to the sales training initiative is what we are looking for. This is the expected isolated impact of training on achieving the business goal. »Note that the expected impact came from the sponsor, not HR If impractical or impossible to get a % contribution, assign a High, Medium, or Low impact based on where the sales training fell in the prioritized list. Or agree on a proxy for impact and success Step 2: Align HR Initiatives to Business Goals and Plan the Initiatives Carefully May 13, 2013 Skillsoft Perspectives 19

Start with a list of drivers Potential drivers to increase sales by 10% Hiring two new salespeople Growth in economy Opening new sales office New salesperson incentive system New advertising campaign Growth in market share due to new products Failing competitor New sales tracking system Consultative sales and product features training Step 2: Example of Impact Determination with Sponsor May 13, 2013 Skillsoft Perspectives 20

Step 2: Example of Impact Determination with Sponsor Prioritize and assign contribution % Prioritized Drivers% contribution HR Initiative? 1.Growth in economy25% 2.Consultative sales and product features training 20%Yes 3.New salesperson incentive system 15%Yes 4.Growth in market share due to new products 10% 5.Hiring two new salespeople10%Yes 6.Opening new sales office5% 7.New advertising campaign5% 8.Failing competitor5% 9.New sales tracking system5% May 13, 2013 Skillsoft Perspectives 21 Prioritized list of drivers to increase sales by 10% Prioritized list of drivers with percentage contribution to increase sales by 10% (These add to 100%) Notice that HR initiatives in total are expected to contribute 45% or almost half to the goal of increasing sales by 10%

Manage your execution to deliver planned outcomes, effectiveness, and efficiency Review progress monthly within HR »Year-to-date results versus plan »Will you make plan? »If not, what is the forecast? »What action should you take? Review progress at least quarterly with CEO, senior leaders, sponsors »Focusing on most important business goals, key measures Step 3: Execute and Report with Discipline May 13, 2013 Skillsoft Perspectives 22

Step 4: Measure and Evaluate 23 May 13, 2013 Skillsoft Perspectives Execute your measurement and evaluation strategy. Did you achieve your goals? If not, what can you learn? What can you improve? If you exceeded goals, what can you learn?

Review: Four Steps to Run HR Like a Business 24 May 13, 2013 Skillsoft Perspectives 1.Identify Business Goals 2.Align HR initiatives to business goals and plan it carefully 3.Execute and report with discipline 4.Measure and evaluate

Our Experience in Running HR Like a Business 25 May 13, 2013 Skillsoft Perspectives Conceptually not difficult Challenging in practice Requires new skills for some, confidence to apply them for others Some common objections »No standards in place to provide guidance »No internal processes to support it »Not comfortable with business-like approach TDRp can help

Exercise 26 May 13, 2013 Skillsoft Perspectives Turn to your neighbor »Identify 3 ways that you are running L&D like a business »Identify 3 areas where you need to improve your processes to more effectively run L&D like a business Be prepared to share one idea with the group

Executive Reporting Framework 27 May 13, 2013 Skillsoft Perspectives

TDRp: Designed to Facilitate the Management of Human Capital 28 May 13, 2013 Skillsoft Perspectives Alignment of human capital initiatives to organization goals Upfront agreement on the expected impact and other measures of success Monthly reports for management of initiatives and operations Quarterly reports for the Department Heads (like the CLO), SVPHR, CEO to assess progress and manage to planned results Focus on three types of measures: Outcomes, Effectiveness, Efficiency

The Target Audience for TDRp 29 May 13, 2013 Skillsoft Perspectives Talent leaders and managers Those responsible for programs, people, and budgets Senior talent leaders SVP of HR, CLO, Head of Talent Acquisition, Heads of other talent processes (e.g leadership development, total rewards, capability management) Senior organizational leaders CEO, CFO, EVPs, SVPs, governing boards Different reports are required for different audiences

Operational vs Executive Reporting 30 May 13, 2013 Skillsoft Perspectives Operational ReportingExecutive Reporting Reporting on specific programs A holistic picture of the business Reporting on what happened Reporting on what happened and projections for the year Reporting results onlySetting goals and reporting results against goals Identifying specific improvement opportunities Identifying how L&D is contributing to the business and how it can provide even greater value Talking to ourselves using L&D terminology Engaging business leaders in business oriented discussions about L&D performance

Reports Are Designed to Answer: 31 May 13, 2013 Skillsoft Perspectives What is the goal or plan? »Organization goal (e.g., 10% increase in sales) »Impact of initiative on it (e.g., 20% contribution or 2% increase in sales) What key programs are planned to achieve it? How are we doing? (Year-to-date results versus plan) Are we going to make plan?(What is the forecast?) Focus: Managing the function versus merely Presenting results

The Four Assumptions 32 May 13, 2013 Skillsoft Perspectives 1.Primary purpose of human capital initiatives and processes: build organizational capability that enables the organization to achieve its goals or achieve its goals more quickly or at lower cost. 2.Human capital initiatives and processes should be aligned strategically to the goals of the organization 3.Business environment characterized by significant uncertainty; plans are made with the best information available. Waiting for absolute certainty and perfection is not an option. 4.Recommended reports and the underlying data will be used appropriately by competent, experienced leaders to manage the function to meet agreed- upon goals and to continuously improve.

The Eight Principles 33 May 13, Employ concise and balanced measures reported in a consistent and clearly defined format 2.Produce and communicate Executive Reporting with a frequency and thoroughness to enable appropriate management of the function 3.Include actionable recommendations in all Executive Reporting 4.Maintain data integrity and completeness 5.Employ appropriate analytical methods 6.Provide the impact and value or benefit of programs and initiatives whenever appropriate 7.Capture and report the full costs of Talent 8.Support continuous improvement of executive reporting and the underlying data bases Skillsoft Perspectives

The TDRp Process 34 May 13, 2013 Skillsoft Perspectives 1.Use standard definitions to extract and convert data into three types of measures -Outcome -Effectiveness -Efficiency 2.Create three standard statements with these measures -Outcome -Effectiveness -Efficiency 3.Create three types of customized reports from these statements -Summary -Program (Initiative) -Operations

Executive Reporting Focus: Standard Measures 35 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Standard Measures: Efficiency measures: How much? How many? At what cost? Effectiveness measures: How well? Outcome measures: What is the impact on the business? May 13, 2013 Skillsoft Perspectives

Executive Reporting Focus: Three Statements 36 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Three Statements: Outcome Statement Effectiveness Statement Efficiency Statement May 13, 2013 Skillsoft Perspectives

Executive Reporting Focus 37 Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics Three Reports: Summary Report Program Report Operations Report May 13, 2013 Skillsoft Perspectives

Executive Reporting Focus 38 Summary Conclusions, Actionable Recommendations, Issues for Further Analysis Data Sources and Systems Systems and processes for organizing data, calculating measures Specific methodology (e.g. Phillips, Brinkerhoff) Scope of TDRp Statements Standard Measures Data Reports System Wide Analytics Program/ Initiative Analytics May 13, 2013 Skillsoft Perspectives

Executive Reporting Process 39 Senior ExecutivesTalent Development Executives Executive Reports Statements Note: Data sets can be organized by processes and/or efficiency, effectiveness & outcomes Data Sources Executive Reporting Process Guiding Principles Financial Systems Evaluation, EOS Systems Non-Financial, non-TDR Systems Others: HRIS, LMS, CRM, ERP OutcomesEffectivenessEfficiency Talent Development Summary Report Quarterly Talent Development Program Report (s) Monthly Talent Development Operations Report (s) Monthly Effectiveness Statement (s) Outcome Statement Efficiency Statement (s) Talent Development Processes Talent Acquisition Leadership Development Learning & Development Capability Management Performance Management Total Rewards Extract, convert and calculate Standard Measures Data Sets May 13, 2013 Skillsoft Perspectives

Recommended Measures 40 May 13, 2013 Skillsoft Perspectives

41 Optimizing Outcomes Efficiency Effectiveness Optimized Outcomes Optimize SET LEARNING GOALS MONITOR & REPORT ANALYZEIMPROVE

Categories of Success Measures Effectiveness measures: Measures how well a task/process/function is performed Efficiency measures: Measures the extent to which time or effort is well used for the intended task or purpose (e.g. output relative to the input) Outcome measures: Measures the relationship and/or value of a program or activity to strategic business goals May 13, 2013 Skillsoft Perspectives 42

L&D Efficiency Measures TypesExamples Activity# courses available # classes delivered # learners attending training (total and unique) # hours of training consumed Utilization / reachCourse fill rate No shows % of target audience reached CostCost of development/ delivery External costs (vendors, course libraries) Travel costs TimeDevelopment time Time from request to fulfillment May 13, 2013 Skillsoft Perspectives 43 These measures answer the questions: how much, when, how long?

What Other Efficiency Measures Do You Use? May 13, 2013 Skillsoft Perspectives 44

L&D Effectiveness Measures TypesExamples Level 1: ReactionCourseware Instructors/ Online delivery Level 2: LearningPre vs post test difference % of tests passed Level 3: Application% of learning applied (or not applied, “scrap”) Perceived improvement in job performance Level 4: Business Results Perceived impact on business results (post or follow up) Actual business results SupportManager support Level 5: ValueValue for time spent May 13, 2013 Skillsoft Perspectives 45 These measures answer the questions: how well?

What Other Effectiveness Measures Do You Use? May 13, 2013 Skillsoft Perspectives 46

Outcome Measures TypeExamples FinancialRevenue, cost of goods sold, operating expenses, profit, margins, growth Customer: satisfaction, loyalty Satisfaction, loyalty, share of wallet, market share, complaints, referrals, reference accounts OperationalProductivity, product quality, new product development, patents EmployeeEngagement, turnover (voluntary, involuntary), leadership quality index May 13, 2013 Skillsoft Perspectives 47 These measures answer the questions: what difference are we making?

What Outcomes Do Your Programs Support? Sales? Productivity or cost reduction? Quality or safety? Compliance? Engagement? Retention? Attraction? Others? May 13, 2013 Skillsoft Perspectives 48

How Do You Determine What to Measure? 49 May 13, 2013 Skillsoft Perspectives L&D outcome measures »Reflected in the top strategic goals of the company »Reflected in important operational goals supported by L&D »Are measured and reported by the business Efficiency and effectiveness measures »Reflect the needs of L&D executives to manage the function efficiently and effectively in pursuit of -Company goals -Department goals »Are actionable »Provide insights Be open to new measures. Just because you haven’t measured something before doesn’t mean it’s not worth measuring

Build a Measures Library 50 May 13, 2013 Skillsoft Perspectives What is it? The single source of truth of your measures »Based on TDRp measures library and consistent with standard industry definitions »Can contain other measures relevant to your industry or company (particularly with respect to outcome measures) Why is it important? »Ensures a common language »Creates accountability »Builds maturity What’s in it? »The measure, type and definition »Criticality of the measure »Frequency of data collection/availability (e.g. monthly, quarterly, annually) »Source of data (system captured, manual) and data owner (organization or individual) »Ease of accessing the data (availability, security) »Quality of the data Who owns it? »A standards body that maintains, updates and publishes the library

Characteristics of a Good Measures Library A Solid Library:Description Measures what is importantFocuses on key performance indicators of real value to the organization Is predominantly quantitative Most measures can be expressed as objective values Is clearly defined and mutually understood Defined by and/or agreed to by process owners or governance body Encourages appropriate behavior Balanced to reward productive behavior and discourage game playing Is multi-dimensionalBalances between efficiency, effectiveness & outcomes; balances rear-view mirror and forward looking measures Articulates ownershipData quality and integrity is owned by specific individuals for specific measures May 13, 2013 Skillsoft Perspectives 51

Considerations for Identifying Measures 52 May 13, 2013 Skillsoft Perspectives Take time to understand the types of measures available »Review the measures library for the relevant talent process »Think about what each measure tells you and how it might provide insights Consider proxy measures »To simplify the process »To advance your progress Not everything is equally critical. Create a prioritization matrix Criticality »High = Critical to demonstrate tangible outcomes. Typically reserved for strategic, costly or visible programs »Medium = Important to demonstrate outcomes. Helps make tradeoffs among alternative programs. »Low = Won’t provide sufficient information for demonstrating outcomes

Practitioner’s Perspective on Building a Measures Library May 13, 2013 Skillsoft Perspectives 53

Exercise 54 May 13, 2013 Skillsoft Perspectives Take 5 minutes For each category (efficiency, effectiveness and outcomes), identify 1-2 measures that you aren’t using today Identify how each measure could enable you to better manage the business of L&D Be prepared to share with the larger group

Question: Are there any measures unique to e-learning? May 13, 2013 Skillsoft Perspectives 55

TDRp Measures Summary Question: What measures are right for my organization? Answer: »Outcome measures that relate to key company goals »Effectiveness measures that support company and department goals »Efficiency measures that support company and department goals Target 3-5 for each to start Check Questions »Why are we collecting this? »What will we do with it? Who will use it? May 13, 2013 Skillsoft Perspectives 56

TDRp Statements 57 May 13, 2013 Skillsoft Perspectives Yes, but what are your goals?

Executive Reporting Process 58 Senior ExecutivesTalent Development Executives Executive Reports Statements Note: Data sets can be organized by processes and/or efficiency, effectiveness & outcomes Data Sources Executive Reporting Process Guiding Principles Financial Systems Evaluation, EOS Systems Non-Financial, non-TDR Systems Others: HRIS, LMS, CRM, ERP OutcomesEffectivenessEfficiency Talent Development Summary Report Quarterly Talent Development Program Report (s) Monthly Talent Development Operations Report (s) Monthly Effectiveness Statement (s) Outcome Statement Efficiency Statement (s) Talent Development Processes Talent Acquisition Leadership Development Learning & Development Capability Management Performance Management Total Rewards Extract, convert and calculate Standard Measures Data Sets May 13, 2013 Skillsoft Perspectives

The Statements 59 Three standard statements »Outcome »Effectiveness »Efficiency Uses standard measures but choice of measures depends on organization Summary statements show »Last year’s actual »Plan (or goal) for this year »Year-to-date results Detail statements show »Monthly, quarterly, trend data »Granularity »Without plan May 13, 2013 Skillsoft Perspectives

Considerations in Creating the Outcome Statement 60 May 13, 2013 Skillsoft Perspectives Requirements »Summary statement »Lists organization’s top goals in priority order »Identifies impact of talent initiative on goal »Four columns -Last year’s actual (LYA) -Current-Year Plan -YTD Actual -YTD Actual as % of Plan Options »Quantitative versus qualitative »Isolated impact versus proxy »Detailed statements Statements

Outcome Statement for L&D Quantitative & Qualitative with Proxies May 13, 2013 Skillsoft Perspectives 61

Experience with Outcome Measures Use of forecasted, isolated impacts of learning? »Quantitative »Qualitative like high/medium/low or essential? Proxies? May 13, 2013 Skillsoft Perspectives 62

Considerations in Creating the Effectiveness Statement 63 May 13, 2013 Skillsoft Perspectives Start with effectiveness measures directly related to the initiatives in support of organization goals Add effectiveness measures which relate to other organizational and department goals (if any) Requirements »Must be some effectiveness measures »Columns: LYA, Plan, YTD, YTD as % of Plan Options »Selected measures »Grouping and labeling

Effectiveness Statement for L&D May 13, 2013 Skillsoft Perspectives 64

Detailed Effectiveness Statement for L&D May 13, 2013 Skillsoft Perspectives 65 What’s added? Quarterly data Breakout measures within the quality category Regional level data

Experience with Effectiveness Measures How many use L1? L2? L3? How many set goals for them? How many actively manage through the year? May 13, 2013 Skillsoft Perspectives 66

Considerations in Creating the Efficiency Statement 67 May 13, 2013 Skillsoft Perspectives Start with efficiency measures directly related to the initiatives in support of organization goals »Number of participants »Number of employees Add efficiency measures which relate to other organizational and department goals Requirements »Must be some efficiency measures »Columns: LYA, Plan, YTD, YTD as % of Plan Options »Selected measures »Their grouping and labeling

Efficiency Statement for L&D May 13, 2013 Skillsoft Perspectives 68

Detailed Efficiency Statement for L&D 69 May 13, 2013 Skillsoft Perspectives What’s added? Monthly data More detailed measures Breakdown by type of learning

Experience with Efficiency Measures How many report Number of participants? »Unique and total? »Set goals? Number of courses, hours? »Set goals? Progress against completion dates? Utilization rates? »Classrooms? Instructors? System usage? Reach? »Set goals? Development/delivery/opportunity cost reductions? May 13, 2013 Skillsoft Perspectives 70

Exercise: Start to Create Efficiency Statement 71 May 13, 2013 Skillsoft Perspectives List the efficiency measures you want to report Most efficiency statements will include: »Number of participants »Number of courses or classes or hours (perhaps segmented type of learning) »Reach (% of employees touched by learning) »Cost and/or cost reduction »Utilization ratios of courses, classes, instructors, or rooms

Practitioner’s Perspective on Using the Three Types of Measures May 13, 2013 Skillsoft Perspectives 72

TDRp Statements Summary Question: What do we do with the measures? Answer »Organize them into three standard statements »Key measures from the statements will be used in the reports to manage the function »Statements provide additional detail May 13, 2013 Skillsoft Perspectives 73

TDR Reports 74 May 13, 2013 Skillsoft Perspectives

Executive Reporting Process 75 Senior ExecutivesTalent Development Executives Executive Reports Statements Note: Data sets can be organized by processes and/or efficiency, effectiveness & outcomes Data Sources Executive Reporting Process Guiding Principles Financial Systems Evaluation, EOS Systems Non-Financial, non-TDR Systems Others: HRIS, LMS, CRM, ERP OutcomesEffectivenessEfficiency Talent Development Summary Report Quarterly Talent Development Program Report (s) Monthly Talent Development Operations Report (s) Monthly Effectiveness Statement (s) Outcome Statement Efficiency Statement (s) Talent Development Processes Talent Acquisition Leadership Development Learning & Development Capability Management Performance Management Total Rewards Extract, convert and calculate Standard Measures Data Sets May 13, 2013 Skillsoft Perspectives

The Reports 76 May 13, 2013 Skillsoft Perspectives Three executive reports for the active management of human capital to deliver planned results Constructed from the three statements just as management reports in an organization are constructed from underlying financial and activity reports The three executive reports: »One targeted for use with senior executives (CEO, SVP, etc.) »Two targeted for use with talent executives and program managers (those with specific responsibility for programs, budget and people)

The Reports (continued) 77 May 13, 2013 Skillsoft Perspectives Three levels of reports »Summary Report (for CEO) »Program Report (for CLO) »Operations Report (for CLO) »Highly customized, pulling the most important measures from the statements Executive-level reports show »Last year’s actual »Plan (or goal) for this year »Year-to-date results »Forecast for this year Detailed reports for managers may show »Monthly, quarterly, trend data »Granularity »Without plan or forecast

Bringing it All Together: The Summary Report 78 May 13, 2013 Skillsoft Perspectives Audience: senior business and HR executives Most significant measures from the effectiveness, efficiency and outcome statements aggregated at the highest level Measures will vary by organization and year to year based on organizational priorities Generated at least quarterly Must be read holistically to obtain an accurate picture of progress against goals Senior Business Executives Outcome Statement Learning Effectiveness Statement Talent Development Summary Report Quarterly Effectiveness Statement (s) Outcome Statement Efficiency Statement (s)

Considerations in Creating the Summary Report 79 May 13, 2013 Skillsoft Perspectives Requirements »Summary report »Top goals for organization impact by talent initiatives »Most important effectiveness and efficiency measures »Columns: -LYA -Plan -YTD -YTD as % of Plan -Forecast Options »Quantitative versus qualitative »Isolated impact versus proxy »Detailed report »Names of measures, how they are grouped, group names New!

L&D Summary Report May 13, 2013 Skillsoft Perspectives 80

Creating the Summary Report 81 May 13, 2013 Skillsoft Perspectives Copy your three goals and impact of learning from the Outcome Statement Copy 3-4 effectiveness measures from your Effectiveness Statement Copy 3-4 efficiency measures from your Efficiency Statement

The Program Report 82 May 13, 2013 Skillsoft Perspectives Purpose: Manage key programs to deliver planned results Audience: Senior Talent Executives Generated for each major Talent program »Includes measures necessary for senior talent leaders and managers to manage program, initiative, or process results on a monthly basis. »Typically includes outcome measures relevant to the program and the key effectiveness and efficiency measures. Senior Talent Executives Outcome Statement Learning Effectiveness Statement Talent Development Program Report Monthly Effectiveness Statement (s) Outcome Statement Efficiency Statement (s)

Considerations in Creating the Program Report 83 May 13, 2013 Skillsoft Perspectives Requirements »Organizational goal the initiative supports »Impact of initiative on goal »Sponsor’s name »Key effectiveness measures »Key efficiency measures »In either rows or columns -Plan, YTD actual, YTD as % of Plan, Forecast -For all listed outcome, effectiveness, and efficiency measures Options »Level of detail (simple, medium, complex or multi year)

Simple Program Report for L&D (page 1) May 13, 2013 Skillsoft Perspectives 84

Simple Program Report for L&D (page 2) May 13, 2013 Skillsoft Perspectives 85

The Operations Report 86 May 13, 2013 Skillsoft Perspectives Purpose: Manage Talent operations to deliver planned results Audience: Talent Executives and managers Pulls key measures from the Efficiency Statement Adds a column for forecast Senior Talent Executives Outcome Statement Talent Development Operations Report (s) Monthly Efficiency Statement (s)

Considerations when Creating the Operations Report 87 May 13, 2013 Skillsoft Perspectives Select the most important efficiency measures from the efficiency statement Requirements »Key efficiency measures that will be actively managed on a monthly basis »Columns for LYA, Plan, YTD, YTD as % of Plan, Forecast Options »Names of the measures »How the measures are grouped »Names of the groups

Operations Report for L&D May 13, 2013 Skillsoft Perspectives 88

TDRp Reports Summary Question: How do the measures help run learning like a business? Answer »Key measures are shown in three executive reports »The Summary Report is used by the CEO, governing body and CLO to manage the function at a high level on a quarterly basis »Program Reports are used by the CLO, Directors, and Program Managers to manage program execution on a monthly basis »The Operations Report is used by the CLO and Directors to manage key efficiency measures on a monthly basis May 13, 2013 Skillsoft Perspectives 89

Practitioner’s Perspective on Executive Reporting May 13, 2013 Skillsoft Perspectives 90 Value Dashboard – monitor, manage and communicate training results to key audiences

Managing the Change 91 May 13, 2013 Skillsoft Perspectives

Making it Happen Technically, TDRp is pretty straight forward It’s about gathering relevant data and displaying it in a succinct way on reports for executives But if it’s so easy, why don’t we see TDRp (or some variant thereof) implemented within most Talent functions? May 13, 2013 Skillsoft Perspectives 92 TDRp requires a fair amount of change in the way the L&D function operates. And let’s face it, change can be difficult. TDRp requires a fair amount of change in the way the L&D function operates. And let’s face it, change can be difficult.

Exercise: What’s Changing? 93 May 13, 2013 Skillsoft Perspectives Identify and share some changes that you believe organizations will experience as a result of TDRp

Innovation Adoption Curve May 13, 2013 Skillsoft Perspectives Innovators Early Adopters Early MajorityLate MajorityLaggards The Chasm I don’t understand why we need to do this. No one is asking for this data This is a good idea. With support, I’m willing to move forward It sounds like a good idea, but we have too many other things that we need to tackle right now. I’ll put this on the agenda for next year. “We need to act now. If we don’t, we risk our ability to positively impact the business and become a true business partner Individuals are at different points on the adoption curve. Sustained change happens when employees ‘cross the chasm’. 94

The Change Model 95 May 13, 2013 Skillsoft Perspectives From John P. Kotter: Leading Change Driving Change 8. Make Change Stick 7. Don’t let up 5. Empower Action6. Create Quick Wins 4. Communicate for Buy-In 3. Get the Vision Right 2. Build the Leadership Team 1. Create a Sense of Urgency People start telling each other: “Let’s go, we need to change things!” A group powerful enough to guide the change is formed and they work together well The team develops the right vision and strategy for the change effort People begin to buy into the change; this shows in their behavior More people feel able to act and do act on the strategy Momentum builds as people try to fulfill the strategy; fewer people resist People make wave after wave of changes, fulfilling the vision New and winning behavior continues despite the pull of tradition and turnover of change leaders

Create a Sense of Urgency Risks of doing nothing? »Senior L&D leaders will lack a holistic view of L&D performance »L&D leaders will not have the information to prioritize investment decisions and funding allocation »L&D leaders will lack a systematic approach to improving training results »The L&D function will have a lower impact on business outcomes »L&D will continue to be viewed as a cost center and will be subject to ongoing cost cutting Opportunity if you act? »L&D will build collaborative partnerships with the business ensure it focuses on the most critical priorities, drives application and produces measureable results »Senior L&D leaders run training like business including -An annual business plan -Agreed upon KPIs -Goals and performance targets »L&D will systematically improve training results and their impact on business outcomes »L&D will have the data, tools and methods to communicate its value to the business May 13, 2013 Skillsoft Perspectives 96

Sponsors vs Stakeholders? 97 May 13, 2013 Skillsoft Perspectives Sponsor: responsible to the CEO for delivering the business goal »The only person with whom you can have a meaningful discussion about the impact of talent on the business »Usually a direct report to the CEO at most one level further down (e.g. SVP of Sales, President of Manufacturing) Stakeholder: An individual who has a "stake" in the program »Can influence it »Wants to know if it is successful »Is directly affected by it »Typically includes -Head of the HR function (like the CLO) -Managers/staff in the HR function responsible for the initiative -Managers/staff in the client function (like Sales) responsible for the initiative -May include the SVP of HR

The Anatomy of Change May 13, 2013 Skillsoft Perspectives 98 Response Skeptical Enthusiasm This is taking time Results aren't visible Is it worth it? Maybe not a bad idea It works! The Dark Night of the Innovator Optimism Pessimism Time Investment Phase Pay-off Phase Start to see pay-offs Brass Bands and Fireworks

Implementing TDRp 99 May 13, 2013 Skillsoft Perspectives

Ten Steps to Implementing TDRp 100 May 13, Socialize TDRp concepts 2.Commit to implement 3.Engage a guiding team 4.Develop a multi-year plan 5.Identify business goals and establish common measures with sponsor(s) and stakeholders 6.Identify goals for each measure including the contribution of Talent to the business goals 7.Create draft statements and reports »Outcome Statement »Effectiveness and Efficiency Statements »Summary Report »Program Reports »Operations Report 8.Manage results to plan; conduct sponsor and stakeholder reviews 9.Review and improve 10.Integrate into business process Skillsoft Perspectives

TDRp Implementation Journey 101 Time Create Awareness (2) Commit to implement (1) Socialize TDRp concepts (5) Implement: Identify business goals and establish measures (6) Implement: Identify goals (7) Implement: Create draft statements & reports (8) Manage: monthly/quarterly reviews Institutionalize Advocate, Champion Degree of Support for Change May 13, 2013 Expand, refine (10) Integrate into business processes ( 9) Review: improve our processes (4) Plan: Establish multi-year plan This is a change process. Build capability over time (3) Engage a guiding team Skillsoft Perspectives

Critical Success Factors Talent Business Plan Leadership Results Drive Action Resources Governance / Alignment Data May 13, 2013 Skillsoft Perspectives 102 Technology & Integration Leadership Exec Reporting driven from the top (the department head) Talent leaders engage in ongoing dialogue with senior business leaders Governance /Alignment Align with business needs Establish Talent priorities Talent Business Plan Driven by business priorities Includes efficiency, effectiveness & outcome measures with SMART goals Results Inform annual planning process Drive continuous improvement Data Access to relevant data Confidence in data quality Availability matches reporting cadence Technology Systems and tools enable scalable, repeatable process Drill down capability to explore anomalies Resources Bandwidth & accountability Knowledge & skills Credibility with the business

Balance Supply with Demand Assess your organization’s strengths, weaknesses and blind spots In your implementation plan: »Leverage your strengths »Develop a plan to mitigate your weaknesses »Determine how you will reveal and address your blind spots Balance the supply of data with the demand (pull) for information that drives accountability and continuous improvement May 13, 2013 Skillsoft Perspectives 103

Track Your Progress 104 May 13, 2013 Skillsoft Perspectives TDRp Progress Assessment

TDRp Progress Assessment: Example 105 May 13, 2013 Skillsoft Perspectives

106 Progress to optimizing training’s value Measurement and Evaluation Strategic Roadmap Reactive M&E Strategic M&E We are here Skillsoft Perspectives

Learn More 107 May 13, 2013 Skillsoft Perspectives

CenterforTalentReporting.org Introduction to TDRp White paper Over 500 measures with definitions Over 60 sample statements and reports Detailed written guidance Sign up for May 13, 2013 Skillsoft Perspectives 108

The CTR Website 109 May 13, 2013 Skillsoft Perspectives For L&D, start here. Go to Learning & Development to find all the whitepapers, definition of terms document, sample statements and reports, and detailed implementation guidance

CTR Website: L&D Resources May 13, 2013 Skillsoft Perspectives 110 Tips & Tools  Constructing reports: L&D Summary Report, Executive Program Report, Outcome Report  Customizing TDRp for your organization  The L&D Summary Report and the Written Executive Report  Multiple Learning Programs in Support of One or More Goals  Outcome Discussions with the Stakeholder Tips & Tools  Constructing reports: L&D Summary Report, Executive Program Report, Outcome Report  Customizing TDRp for your organization  The L&D Summary Report and the Written Executive Report  Multiple Learning Programs in Support of One or More Goals  Outcome Discussions with the Stakeholder

Accelerate Your Mastery 111 May 13, 2013 Skillsoft Perspectives Two-day workshops offered in 2013 »Jun 6-7 Alexandria, VA »Oct Atlanta »Use code SKWKSP1 to get $200 discount Webinars offered monthly »Intro to TDRp »Implementation CSFs »Implementation Guidance (members only) First TDRp conference planned for October 17 Certification »Individuals »Vendors providing software products employing TDRp Accreditation »Organizations implementing TDRp »Consultancies providing services

Become a Member of the Center for Talent Reporting Benefits »Access to Enhanced Content on the website -Formulas and references for measures -Excel versions of sample statements and reports -Detailed implementation guidance »$300 discount on workshops and conference »Priority for TDRp certification Investment »Only $299 May 13, 2013 Skillsoft Perspectives 112

Wrap Up: Objectives Review 113 May 13, 2013 Skillsoft Perspectives By now, you should be able to: Articulate the principles of TDR and its benefits to your organization Identify the type of measures that are right for your organization to populate the statements and reports Explain the Talent Development statements and reports and why each is important Understand how the statements and reports can help you rum learning like a business Appreciate the change that will be required to implement TDRp and run learning like a business

Advice: Just Get Started! Identify where you might begin to implement »One project »One business unit or region Start small and grow »Begin with 3-5 measures from each group »Start with what you already have Don’t wait for perfection, new data systems »You can do much manually »You will learn as you go So, just start! May 13, 2013 Skillsoft Perspectives 114

Contact Information Dave Vance: Executive Director » »Phone: (970) Peggy Parskey: Assistant Director » »Phone: (323) Dawn Campbell Borland » May 13, 2013 Skillsoft Perspectives 115

Appendix: Job Aids 116 May 13, 2013 Skillsoft Perspectives

Sample Measures for L&D 117 May 13, 2013 Skillsoft Perspectives TypePurposeExamples EfficiencyDemonstrate the ability of the learning function to deliver learning solutions: Cost effectively In a timely manner That reach the target audience That use learning resources wisely Costs to design, develop and deliver content Course activity and volume Time from request to deployment Utilization of courses EffectivenessDemonstrate the ability of the learning function to deliver learning solutions: Aligned with business needs Satisfy learners Producing increased learning, new behaviors and improved performance Providing value to the business Level 1: Courseware quality, delivery effectiveness, learning effectiveness Level 2: Amount learned Level 3: % of learning applied on the job Level 4: Job impact Level 5: ROI or ROE OutcomesDemonstrate the ability of the learning function to deliver learning solutions: That produce tangible business outcomes Result in high levels of satisfaction Increase employee engagement and retention Financials: revenue growth, profitability, cost of goods sold Customer satisfaction and loyalty Operational: space utilization, inventory turns Employee engagement, turnover

The Outcome Statement 118 The QuestionThe Answer What is it?Depicts the key goals / desired outcomes (results) of the organization along with the impact the initiatives are expected to have on achieving those outcomes Who is the audience?Senior HR Leaders Why do it?The Business Outcome Statement depicts the value of the learning organization to the business What reports does it feed? Quarterly Summary Report Monthly Program Report May 13, 2013 Skillsoft Perspectives

The Effectiveness Statement 119 The QuestionThe Answer What is it?Focuses on how effectively or how well the business outcomes are achieved. Who is the audience?Senior HR Leaders Why do it?The effectiveness statement provides insight into areas for continuous performance improvement of learning programs What reports does it feed Quarterly Summary Report Monthly Program Report May 13, 2013 Skillsoft Perspectives

The Efficiency Statement 120 The QuestionThe Answer What is it?Brings together all the activity and cost elements necessary to judge how efficiently the outcomes were achieved Who is the audience?Senior HR Leaders Why do it?The efficiency statement helps ensure the learning function is well managed and reaching the desired audience What reports does it feed Quarterly Summary Report Monthly Operational Report May 13, 2013 Skillsoft Perspectives

The Statements Compared May 13, 2013 Skillsoft Perspectives QuestionEffectiveness Statement Efficiency StatementOutcome Statement What is it?Focuses on how effectively or how well the business outcomes are achieved. Brings together all the activity and cost elements necessary to judge how efficiently the outcomes were achieved Depicts key goals / desired outcomes of the organization along with the expected impact of L&D on achieving those outcomes Who is the audience? Senior L&D Leaders Why do it?The effectiveness statement provides insight into areas for continuous performance improvement of learning programs The efficiency statement helps ensure the learning function is well managed and reaching the desired audience The Outcome Statement depicts the value of the learning organization to the business What reports does it feed? Quarterly L&D Summary Report Monthly L&D Program Report Quarterly L&D Summary Report Monthly L&D Operational Report Quarterly L&D Summary Report Monthly L&D Program Report 121

The Summary Report 122 The QuestionThe Answer What is it?This Summary Report contains the most significant measures from the business outcome, effectiveness and efficiency statements aggregated at the highest level. This report would be generated quarterly. Who is the audience?Senior Organization Leaders & Senior HR Leaders Why do it?The Summary Report provides a holistic view of the overall impact, effectiveness and efficiency of the learning organization What statements feed it?Outcome Statement Effectiveness Statement Efficiency Statement May 13, 2013 Skillsoft Perspectives

The Program Report 123 The QuestionThe Answer What is it?This Program Report highlights the connections between programs/ initiatives, the organizational goal, and the impacts on those goals. Who is the audience?Senior HR Leaders and Managers Why do it?The Program Report enables HR Leaders to manage programs and initiative What statements feed it?Outcome Statement Effectiveness Statement Efficiency Statement May 13, 2013 Skillsoft Perspectives

The Operations Report 124 The QuestionThe Answer What is it?This Operations Report include efficiency details on the costs and activities of the HR function. This report shows actuals and performance relative to plan. Who is the audience?Senior HR Leaders and Mangers Why do it?The Operations Report enables HR Leaders to manage the efficiency of the HR operation What statements feed it?Efficiency Statement May 13, 2013 Skillsoft Perspectives

The Reports Compared May 13, 2013 Skillsoft Perspectives QuestionL&D Program ReportL&D Operations ReportL&D Summary Report What is it?Highlights the connections between learning programs/ initiatives, the organizational goal and the impact of learning on those goals. Includes efficiency details on the costs and activities of the learning function. This report shows actuals and performance relative to plan. Contains the most significant measures from the outcome, effectiveness and efficiency statements aggregated at the highest level. This report should be generated quarterly. Who is the audience? Senior L&D Leaders and their staff Senior Business Leaders & Senior L&D Leaders Why do it?Enables Learning Leaders to manage programs and initiative Enables Learning Leaders to manage the efficiency of the L&D operation Provides a holistic view of the overall impact, effectiveness and efficiency of the learning organization What statements feed it? Outcome Statement Learning Effectiveness Statement Learning Efficiency Statement Outcome Statement Learning Effectiveness Statement Learning Efficiency Statement 125

Variations of L&D Statements Type Statement or GraphLevel of DetailSector Quant, Qual, or Mixed/Proxies Outcome StatementHighPrivateQuantitative StatementHighPrivateQualitative StatementHighPrivateMixed, Proxies StatementDetailedPrivateQuantitative StatementHighGovernmentMixed, Proxies Effectiveness StatementHighAllQuantitative StatementDetailedAllQuantitative GraphDetailedAllQuantitative Efficiency StatementHighAllQuantitative StatementDetailedAllQuantitative May 13, 2013 Skillsoft Perspectives 126

Variations of Reports: L&D TypeLevel of DetailSector Quant, Qual, or Mixed/Proxies SummaryHighPrivateQuantitative (1) SummaryHighPrivateQuantitative (2) SummaryHighPrivateQualitative SummaryHighGovernmentMixed, Proxies ProgramSimpleAllQuantitative ProgramMediumAllQuantitative ProgramMulti YearAllQuantitative ProgramDetailedAllQuantitative OperationsHighAllQuantitative May 13, 2013 Skillsoft Perspectives 127