Implementing & Sustaining ROI Patti Phillips, Ph.D.

Poll Question BUSINESS 2.0 Dream Team*  CEO  COO  CFO  CMO  CIO  SVP – Industrial Design  SVP – R&D  SVP – Online Sales  SVP - Procurement *BUSINESS 2.0 July 2004 Who’s missing? a.No one b.SVP – Corporate Services c.Chief Learning Officer d.Michael Jordan

Sample of Dream Team Accomplishments Saved $1.8 billion in 18 months and increased sales from $10 million to more than $85 million in five years Increased shareholder base from less than 15 percent institutional ownership to more than 90 percent Increased number of new patents Reduced 4 payroll systems to 1 Reduced 11 billing systems to 2 in 24 months

Meet Donald Hodges President, GlobalCom University 15 years experience in adult learning Handpicked by previous CEO Wants to impress new CEO with program results Donald’s results as reported to new CEO Developed 10 new programs Offered 1,120 hours of training Had 1,500 employees attend training Received on average 4.5 out of 5 on the program satisfaction rating Donald’s results as reported to new CEO Developed 10 new programs Offered 1,120 hours of training Had 1,500 employees attend training Received on average 4.5 out of 5 on the program satisfaction rating

Poll Question H ow likely is Don to be recruited for a position on the Dream Team? a. No way. b. Possibly, with a little improvement. c. Absolutely!

1 Objective To help you begin Today we have 1.Effectively communicating results on a routine basis 2.Developing an ROI implementation strategy 3.Determining the ROI on the ROI

What do we mean by results? Level 1. Reaction & Planned Action 2. Learning 3. Application 4. Impact 5. Return on Investment Measurement Focus Measures participant satisfaction with the program and captures planned actions Measures changes in knowledge, skills, and attitudes Measures changes in on-the-job behavior Measures changes in business measures Compares program benefits to the costs Economic System Individual

ROI Methodology Adapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) The Human Resources Scorecard: Measuring the Return on Investment. Boston, MA: Butterworth- Heinemann Develop Evaluation Plans and Baseline Data Develop Evaluation Plans and Baseline Data Stage 1 Evaluation Planning Stage 2 Data Collection Collect Data During Solution Implementation Collect Data During Solution Implementation Collect Data After Solution Implementation Collect Data After Solution Implementation Develop Objectives Of Solution Develop Objectives Of Solution Level 1 Level 2 Level 3 Level 4

Tabulate Costs Of Solution Tabulate Costs Of Solution Isolate the Effects of Solution Isolate the Effects of Solution Convert Data to Monetary Value Convert Data to Monetary Value Calculate the Return On Investment Calculate the Return On Investment Implement Communication Process Implement Communication Process Stage 3 Data Analysis Identify Intangibles Identify Intangibles Stage 4 Communicate Results Level 5 Intangible Benefits

Reaction & Planned Action Learning Application & Implementation Impact ROI Isolate the Effects of the Program Report the complete story of success. Intangible Benefits

Developing a communication strategy requires us to answer three questions. 1.What do we want? 2.Who can give it to us? 3.How best can we ask?

Common Target Audiences What do we want?Who can give it to us? Secure approval for programClient, top executives Gain support for the programImmediate managers, team leaders Build credibility for the training staffTop executives Enhance reinforcement of the programImmediate managers Enhance results of future programsParticipants Show complete results of the programKey client team Stimulate interest in training programsTop executives Demonstrate accountability for client expendituresAll employees Market future training programsProspective clients

How best can we ask? Ongoing Summaries Complete ROI Impact Study Report Executive Summary One-page Report Training Scorecard

How?What?Who?When? Ongoing SummaryBrief report of program success Training staff Evaluation team Participants One week after program (L1 & L2) One week after data are collected (L3 & L4) Complete ReportDetails of evaluation project Training staff Evaluation team Client Senior Managers* 30 days after data are collected Executive SummaryBrief overview of evaluation project Training staff Evaluation team Client Participants Senior Managers* 30 days after data are collected One-Page ReportMicro-level scorecard of program success Training staff Evaluation team Senior Managers* Supervisors 30 days after data are collected (Use with caution!)

Complete Report General Information Methodology for Impact Study Data Analysis Costs Results Barriers and Enablers Conclusions and Recommendations Exhibits

When communicating with senior managers consider two questions: Do they believe you Can they take it

Communication Progression First 2 ROI Studies 3-5 ROI Studies 6 Plus ROI Studies Detailed Study Executive Summary One Page Summary Meeting No Meeting

Training Scorecard Provides macro-level perspective of success Serves as a brief report versus detailed study Shows connection of training’s contribution to business objectives Integrates various types of data Demonstrates alignment between programs, strategic objectives, and operating goals

Training scorecards include seven categories of data. Indicators Reaction and Planned Action Learning Application Business Impact ROI Intangibles

Developing an ROI strategy requires us to answer four questions. 1.Do we need it? a.Our assessment b.Our clients’ assessment 2.What are our desired outcomes? a.What are the consequences of our implementing an ROI strategy? b.What are the negative consequences if we do not implement an ROI strategy? 3.Why? a.What purpose will implementation success serve us? b.What purpose will implementation success serve our team? c.What purpose will implementation success serve the organization? 4.How can we get there? a.What are the perceived barriers? b.How can we overcome them? c.What is one action we can take today to move forward?

Are you a candidate for ROI? Sample Assessment Questions Is your organization large with a variety of programs? Is your organization undergoing significant change? Is there pressure from senior management to measure results? Is the image of your training function less than desirable? Is there an increased focus on linking training to strategy? For complete assessment go to Tools & Templates at

Potential Outcomes of ROI Implementation Eliminate unnecessary programs Increase (or maintain) training budget Improve image with line supervisors Reallocate resources to allow for staff development Reallocate resources to build evaluation infrastructure Expand implementation of successful programs

If you implement an ROI strategy, what outcomes do you expect? What are the consequences of your implementing this strategy? If you do not implement this strategy, what are the negative consequences?  Exercise 1

Purpose drives the initiative to achieve outcomes. Implementation of our evaluation strategy gives us the information we need to help develop our people, improve our processes, and contribute to the bottom line – literally! We ensure that taxpayer dollars are allocated to the programs that directly support the agency mission.

What purpose will achieving the ROI strategy outcomes serve you, your team, and your organization?  Exercise 2

Poll Question Which of the following represents the biggest barrier to your implementing ROI? 1.Confidence to implement ROI 2.Management support 3.Cost 4.Time 5.Resources (people, technology)

Cost-Saving Approaches to ROI Plan for evaluation early in the process Build evaluation into the training process Share the responsibilities for evaluation Require participants to conduct major steps Use short-cut methods for major steps Use sampling to select the most appropriate programs for ROI analysis

Use estimates in the collection and analysis of data Develop internal capability to implement the ROI process Utilize web-based software to reduce time Streamline the reporting process Cost-Saving Approaches to ROI

Actions to make ROI work include Planning and discipline –Establish goals and targets –Assign responsibilities Improve needs analysis process Implement cost-savings approaches Communicate progress Engage Management –In Programs –In Evaluation Develop staff skills –ROI Networks –Workshops –ROI Certification –Competency groups –Read books/case studies/articles –Develop a case study –Teach the process to others

What is one action you can take today to move forward with ROI implementation?  Exercise 3

So what is the ROI on ROI? Maximum Use $ High ROI $ Success with ROI implementation is related to the extent to which results are used for process improvement!

Individual Perspective (Levels 1 and 2) Are facilitation scores improving? Are programs perceived as more relevant and important? Do content and program design better reflect adult learning principles? Is there evidence that learning is taking place at a higher rate? System Perspective (Level 3) Do supervisors better support training? Are systems in place to better support the learning transfer? Is frequency of use of knowledge and skills increasing? Are there fewer barriers to learning transfer? When developing the ROI on the ROI consider results from all perspectives!

Economic Perspective (Level 4 and 5) How many unnecessary programs have we eliminated? Have we decreased the cost of training while improving the quality of our programs? How much has our budget increased? When comparing cost savings and budget increases to the cost of ROI implementation, do we see a positive economic return?

Join the Dream Team  CEO  COO  CFO  CMO  CIO  SVP – Industrial Design  SVP – R&D  SVP – Online Sales  SVP - Procurement