Introduction to Executive Reporting for CLOs Tamar Elkeles Vice President of QUALCOMM’s Corporate Learning Center Version: 26-Feb-11

Learning measurement: Key performance indicators Does your organization have a standard set of key performance indicators that you measure and monitor regularly? Source: KnowledgeAdvisors Learning Analytics Best Practices Research Study, 2004. © 2011 KnowledgeAdvisors

The Measurement Gap for CEOs 2009 ROI Institute Study based on responses from 96 CEOs © 2011 KnowledgeAdvisors 3

The Idea is Born Conversation between Kent and Tamar in Spring 2010 Executive Council created to provide guidance Dave Vance asked to provide leadership Industry thought leaders have provided advice, insight, and further guidance © 2011 KnowledgeAdvisors

Executive Council Members Tamar Elkeles, Qualcomm, Chair Terry Bickham, Deloitte Cedric Coco, Lowe’s Marilyn Figlar, Lockheed Martin Karen Kochler, CIGNA Sundar Nagarathnam, Net App Sandy Shaw, Sodexo Don Shoultz, BP Tom Simon, CNA Insurance Carrie Beckstrom, ADP © 2011 KnowledgeAdvisors

Industry Thought Leaders Providing Advice and Counsel to the Project Laurie Bassi Josh Bersin Rob Brinkerhoff Jack Phillips Frank Anderson Jac Fitz-enz © 2011 KnowledgeAdvisors

The Reporting Effort Goals Practical, relevant guidance for reporting in the talent development field Standard measures and definitions Standard reports Adopted by many organizations and becomes the standard for the industry Greater clarity and efficiency for each organization Ability to compare across the industry © 2011 KnowledgeAdvisors

Phase I Phase II Phase III The Three Phase Plan Expanded Reporting for L&D Reporting for Talent Development Basic Reporting for L&D Focus of today’s presentation: progress on Phase I © 2011 KnowledgeAdvisors

Executive Reporting for CLO’s: What They Want and How to Get It Dave Vance President, Manage Learning LLC

The Need Clear guidance for L&D Reporting Data measures and definitions Similar to what GAAP provides for accounting Generally Accepted Accounting Principles Reports: Income Statement, Balance Sheet and Cash Flow Statement Measures and definitions: Income, expenses How to calculate and report Examples: Depreciation, value of an asset © 2011 KnowledgeAdvisors

Benefits Talent Development Reporting Principles (TDRP) Clear, practical reporting guidance for CLOs to use within their own organization Manage the L&D function to produce desired outcomes effectively and efficiently Provide appropriate, meaningful reports to the CEO Standard measures and definitions to benchmark and learn from other organizations © 2011 KnowledgeAdvisors

Phase 1 Deliverable Reflects Efforts of Many Kent’s vision and commitment Executive Council input and support Thought leader’s guidance and counsel Key leaders at Knowledge Advisors Peggy Parskey Jeffrey Berk Kendall Kerekes © 2011 KnowledgeAdvisors

Overview Two primary audiences for reporting Four recommendations CLO, including managers and staff CEO, including governing boards and senior leaders Four recommendations Adopt a set of guiding principles Adopt standard measures and definitions Adopt three standard statements Adopt three standard reports © 2011 KnowledgeAdvisors

Focus of the Executive Reporting Process Summary Conclusions, Actionable Recommendations, Issues for Further Analysis Reports Scope of White Paper Statements Program/ Initiative Analytics Standard Measures System Wide Analytics Systems and processes for organizing data, calculating measures Data Specific methodology (e.g. Phillips, Brinkerhoff) Data Sources and Systems

Data Aggregation and Reporting High Scope of White Paper Summary Reports Summary Statements Detailed Reports Aggregation of Data Standard Measures Detailed Statements Low Low High Level of Customization © 2011 KnowledgeAdvisors

The Assumptions L&D broadly defined: All programs and initiatives designed to increase human capital to enable greater organizational and personal capacity, effectiveness, and efficiency Primary purpose of L&D: Enable organization to achieve its goals Learning is strategically aligned to goals Recommended reports used appropriately © 2011 KnowledgeAdvisors

The Eight Principles Employ concise and balanced measures reported in a consistent and clearly defined format Produce and communicate Executive Reporting with a frequency and thoroughness to enable appropriate management of the function Include actionable recommendations in all Executive Reporting Maintain data integrity and completeness © 2011 KnowledgeAdvisors

The Eight Principles Employ appropriate analytical methods Provide the impact and value or benefit of programs and initiatives whenever appropriate Capture and report the full costs of L&D Support continuous improvement of executive reporting and the underlying data bases © 2011 KnowledgeAdvisors

Executive Reporting Process Business Executive Summary Report Learning Executive Reports Guiding Principles Executive Reports L&D Summary Report L&D Program Report L&D Operational Report Business Outcomes Statement Learning Effectiveness Statement Learning Efficiency Statement Statements Data Sets Business Outcomes Effectiveness Efficiency Extract, convert and calculate Standard Measures Learning Management System Other Sources (e.g HRIS, ERP, CRM) Financial Data Evaluation System Data Sources © 2011 KnowledgeAdvisors

Standard Measures Business Outcomes Effectiveness Efficiency Organizational goals L&D’s impact on those goals Effectiveness Levels 1-5, timeliness Efficiency Costs Volume (participants, programs, classes, hours) Ratios Utilization rates Program and vendor management © 2011 KnowledgeAdvisors

The Statements Three standard statements Each statement shows Business Outcome Effectiveness Efficiency Each statement shows Last year’s actual Plan (or goal) for this year Year-to-date results Standard measures are used but choice of measures depends on organization © 2011 KnowledgeAdvisors

Business Outcome Statement © 2011 KnowledgeAdvisors

Effectiveness Statement December 9, 2018 © 2011 KnowledgeAdvisors 23

Efficiency Statement December 9, 2018 © 2011 KnowledgeAdvisors 24

The Reports Three levels of reports Each report shows L&D Summary Report L&D Program Report L&D Operations Report Each report shows Last year’s actual Plan (or goal) for this year Year-to-date results Forecast for this year Highly customized, pulling measures from the statements © 2011 KnowledgeAdvisors

Data Aggregation and Reporting High Scope of White Paper Summary Reports Summary Statements Detailed Reports Aggregation of Data Standard Measures Detailed Statements Low Low High Level of Customization © 2011 KnowledgeAdvisors

L&D Summary Report December 9, 2018 © 2011 KnowledgeAdvisors 27

L&D Program Report December 9, 2018 © 2011 KnowledgeAdvisors 28

L&D Operations Report December 9, 2018 © 2011 KnowledgeAdvisors 29

Summary: Talent Development Reporting Principles TDRP provides Guiding principles for reporting Standard measures and definitions Standard statements Executive reports Benefits Provide industry guidance on reporting Support better management of the function Support benchmarking and cross company comparisons © 2011 KnowledgeAdvisors

Next Steps Continue to share and socialize TDRP Webinars Workshops Industry begins to implement Refine and expand the effort for L&D (Phase II) Extend the effort to the rest of talent development (Phase III) This is an open, collaborative effort; no copyrights © 2011 KnowledgeAdvisors

Conclusion Please share your thoughts with us Other suggested measures Difficulty in applying standard measures Need for better definitions Write to Dave Vance: dave@poudrerivergroup.com At www.KnowledgeAdvisors.com This PowerPoint deck White Paper (70-page) Overview Paper (25-page) © 2011 KnowledgeAdvisors