Copyright 2004 ROI Institute, Inc. how me the money! Moving from Impact to ROI Patti Phillips, Ph.D.

Copyright 2004 ROI Institute, Inc. In this session participants will: Identify program benefits Convert benefits to monetary value Tabulate fully-loaded costs Identify intangible benefits Calculate an ROI

Copyright 2004 ROI Institute, Inc. ROI is reported as BCR = Program Benefits Program Costs ROI = Net Program Benefits Program Costs X 100

Copyright 2004 ROI Institute, Inc. ROI is reported in the context of five levels of evaluation. Level 1. Reaction & Planned Action 2. Learning 3. Application 4. Impact 5. Return on Investment Measurement Focus Measures participant satisfaction with the program and captures planned actions Measures changes in knowledge, skills, and attitudes Measures changes in on-the-job behavior Measures changes in critical outcomes Compares program benefits to the costs

Copyright 2004 ROI Institute, Inc. Moving from Impact to ROI requires five steps. Level 4 Level 5 1.Identify program benefits 2.Convert benefits to monetary value 3.Tabulate program costs 4.Identify intangible benefits 5.Compare monetary benefits to program costs

Copyright 2004 ROI Institute, Inc. Level 4 1.Identify program benefits

Copyright 2004 ROI Institute, Inc. Level 4 measures are defined as … the consequence of doing something different as a result of the program

Copyright 2004 ROI Institute, Inc. Poll Question A. Decrease error rates B. Increase use of disciplinary discussion skills C. Initiate at least three cost reduction projects D. Achieve a 2:1 benefit to cost ratio E. Improve teamwork Which of the following measures represent Level 4 Impact measures?

Copyright 2004 ROI Institute, Inc. A. Decrease error rates E. Improve teamwork Which of the following measures represent Level 4 Impact measures?

Copyright 2004 ROI Institute, Inc. Hard Data versus Soft Data Level 4 measures include two type of data.

Copyright 2004 ROI Institute, Inc. Decrease error rates versus Improve teamwork Level 4 measures include two type of data.

Copyright 2004 ROI Institute, Inc. Tangible versus Intangible Level 4 measures include two type of data.

Copyright 2004 ROI Institute, Inc. All data can be converted. Productivity Quality Time Cost Savings Revenue Convert data to cost savings Sales converted to profit margin

Copyright 2004 ROI Institute, Inc. Poll Question Which one of the following is the most credible item? A.Vulcan Materials Company produced 195 million tons of crushed stone during 2001 Source: Annual Report B.Wachovia Bank receives a 932% ROI in a training program for relationship managers. Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 81. C.Verizon receives a -85% ROI in a telephonic customer service skills program. Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 131.

Copyright 2004 ROI Institute, Inc. Credibility of data is influenced by Reputation of the source –Source of data –Source of study Bias –Researcher bias –Audience bias Methodology –Assumptions made in the analysis –Type of data –Scope of analysis –Realism of the outcome data

Copyright 2004 ROI Institute, Inc. Data are converted by Converting output to contribution - standard value Converting the cost of quality - standard value Converting employee’s time – standard value Using historical costs Using internal and external experts Using data from external databases Linking with other measures Using participants’ estimates Using supervisors’ and managers’ estimates Using staff estimates

Copyright 2004 ROI Institute, Inc. Historical Costs and Expert Input are Used to Convert Data Actual CostsLegal Fees, Settlements, from RecordsLosses, Material, Direct Expenses Additional EstimatedEEO/AA Staff Time, Costs from StaffManagement Time The Cost of A Sexual Harassment Complaint 35 Complaints $852,000 Annually Cost per Complaint = $24,343 $852,000 35

Copyright 2004 ROI Institute, Inc. External Databases are Used to Convert Data Cost of one turnover* Middle Manager$70,000 annual salary Cost of turnover150% Total cost of turnover$105,000 * External data – value obtained from industry related study

Copyright 2004 ROI Institute, Inc. Supervisor Estimates Used to Convert Absenteeism Unit of Measure: Absenteeism Data Collection Technique: Focus Groups Source of Data: Supervisors 1.What happens when someone is unexpectedly absent? 2.In monetary terms, how much does it cost you when a person is unexpectedly absent for one day? 1.What is your basis for this estimate? 3.On a scale of 0% - 100% (0% = no confidence; 100% = certainty) how confident are you in your estimated value?

Copyright 2004 ROI Institute, Inc. Supervisor Estimates Used to Convert Absenteeism SupervisorEst. Per Day Cost % ConfidenceAdjusted Per Day Cost 1$1,00070%$700 2$1,50065%$975 3$2,30050%$1,150 4$2,00060%$1,200 5$1,60080%$1,280 $5,305 Average adjusted per day cost of 1 absence$1,061

Copyright 2004 ROI Institute, Inc.

To Convert or Not Convert Use the 4-Part Test  Is there a standard value?  Is there a method to get there?  Can we get there with minimum resources?  Can we convince our executive in 2 minutes that the value is credible?

Copyright 2004 ROI Institute, Inc. Data conversion requires 5 steps: 1.Focus on a unit of measure 2.Determine the value (V) of each unit 3.Calculate the improvement in the measure (∆P) 4.Determine the annual improvement in the measure (A∆P) 5.Calculate the total monetary value of the improvement (A∆P x V)

Copyright 2004 ROI Institute, Inc. Grievances Converted by Internal Experts Step 1: 1 Grievance Step 2: V = $6,500 Step 3: ∆P = 7 out of 10 grievances prevented per month due to program Step 4: A∆P = Step 5: A∆P x V =

Copyright 2004 ROI Institute, Inc. Grievances Converted by Internal Experts Step 1: 1 Grievance Step 2: V = $6,500 Step 3: ∆P = 7 out of 10 grievances prevented per month due to program Step 4: A∆P = Step 5: A∆P x V = 84 x $6,500 = $546,000 7 x 12 = 84

Copyright 2004 ROI Institute, Inc. AB  Operating Costs  Support Costs  Administrative Costs  Participant Compensation and Facility Costs  Classroom Costs CD  Program Development Costs  Administrative Costs  Classroom Costs  Participant Costs  Analysis Costs  Development Costs  Delivery Costs  Overhead Costs  Evaluation Costs Poll Question Which cost categories represent the fully-loaded costs of a program?

Copyright 2004 ROI Institute, Inc. Recommended Cost Items Analysis Costs (Prorated) Development Costs (Prorated) Program Materials Instructor/Facilitator Costs Facilities Costs Travel/Lodging/Meals Participant Salaries and Benefits Administrative/Overhead Costs Evaluation Costs

Copyright 2004 ROI Institute, Inc. Potential Intangible Benefits Increased Job Satisfaction Increased Organizational Commitment Improved Teamwork Improved Customer Service Reduced Complaints Reduced Conflicts Reduced Stress

Copyright 2004 ROI Institute, Inc. ROI is reported one of two ways: BCR = Program Benefits Program Costs ROI = Net Program Benefits Program Costs X 100

Copyright 2004 ROI Institute, Inc. Healthcare Inc. Program Costs –Fully loaded to include needs assessment, development, coordination, participant salaries and benefits, and evaluation –Total Costs = $277,987 Monetary Benefits –Complaint Reduction Value of one internal complaint = $24,343 Annual improvement related to program – 14.8 complaints prevented –Turnover Reduction Value of one turnover statistic = $20,887 Annual improvement related to program = 136 turnovers prevented Your assignment: Calculate the BCR and the ROI

Copyright 2004 ROI Institute, Inc. Healthcare Inc. Program Costs = $277,987 Monetary Benefits –Complaint Reduction Value of one internal complaint = $24,343 Annual improvement related to program – 14.8 complaints prevented 14.8 X $24,343 = $360,276 –Turnover Reduction Value of one turnover statistic = $20,887 Annual improvement related to program = 136 turnovers prevented 136 X $20,887 = $2,840,632

Copyright 2004 ROI Institute, Inc. Healthcare, Inc. Program Benefits = Program Costs = BCR = = ROI = x 100 = $3,200,908 $277, $2,922,921 $277,987 1,051.5% $360,276 + $2,840,632 $277,987

Copyright 2004 ROI Institute, Inc. What is a good ROI? Set the value as with other investments, e.g. 15% Set slightly above other investments, e.g. 25% Set at break even - 0% Set at client expectations

Copyright 2004 ROI Institute, Inc. During this session we Defined program benefits Converted benefits to monetary value Identified fully-loaded costs Identified intangible benefits Calculated an ROI

Copyright 2004 ROI Institute, Inc. Remember Data Conversion When in doubt, leave it out!  Use the most conservative alternative  Use the most credible data course  Use the four-part test!  Use only first-year benefits for short-term programs Cost Tabulation When in doubt, leave it in!  Tabulate the fully-loaded costs  Develop a consistent cost profile  Use the most conservative alternative  Use the most credible data source

Copyright 2004 ROI Institute, Inc. Go forth and…. how them the money! t