1. Return on Investment: Training and Development
ROI and Evaluation 1

2. ROI Contents Purpose of ROI, and Value of training What is ROI Why ROI
Methodology
Toolkit

3. Models Benefit/Cost Ratio ROI (%) 3 3
Benefit/Cost Ratio: This is a basic definition of ROI. This is a quantification of the relation between the benefits of a program and its costs. When BCR is greater than one, the benefits outweigh the costs and the program is considered a success.
When BCR is less than one, the cost exceeds the benefits and demonstrates that improvements or changes probably need to be made to justify continuation of the program.
Another useful and often used definition/formula expresses the ROI as the percentage return on the costs incurred. This has the advantage of speaking to many investors and stakeholders in their language.
A result greater than 100% means that the program has a net benefit after accounting for the costs involved in running it. For instance, an ROI% = 150% means that the program yields a 150% return on money invested; i.e., the program yields $1.50 for every dollar that the program costs.
A result less than 100% means the program has a net cost. This means that the program does not recoup its cost after accounting for the benefit. When this happens, there may be a “hidden” or social benefit that is not quantifiable, such as an increase in employee morale. In these cases, stakeholders and decision makers need to ascertain whether the scale of loss is justifiable given the money spent. A loss of 3 percent of several thousand dollars may be worth it to realize a happier workplace, but 3 percent of several million may not; there may be easier or less expensive ways to create a happier workplace. This is where ROI really becomes useful. Used properly it can be an objective method to compare the benefits, costs and returns for two or more programs.
Note – See Figure 2-2 on p. 37 and throughout the Phillips text for the model. This is a widely adopted and complete model for the process of calculating ROI for training and development. 3 3

4. The Value of Human Capital
We all know Human Capital (and its development) is vital
Human Capital is often much undervalued as an investment opportunity
ROI can be very high
Phillips refers to 800% being achievable regularly
Tennessee Valley authority claim 1000%+
A world-class corporate university claimed 5,612%

5. Why ROI? Pressure on training functions to show demonstrable return
Value for money from training activity
Value of HRD to the business; e.g.
Financial benefits
Staff productivity
Staff retention
Clear alignment with corporate goals
In-house vs outsourced training
ROI analysis is a tool to support the contention that training is valuable; demonstrate benefits vs cost
Pressure to reduce costs must be balanced by visibility of benefit
“Warm and fuzzy” is no longer any good! (in most cases)
Jay Cross alternative view

6. ROI as Process Use evaluation techniques to acquire performance data
Use Training Needs Analysis to direct the project to the right data
Use analytical techniques to calculate ROI from the data
Use ROI results to review training effectiveness
Use the reviews to inform future decisions

7. ROI as Perception
Which of these perceptions indicate your client’s view of ROI (which you need to address)?
The Board may see a big picture of how the training affects the company’s ability to achieve its corporate goals
The finance department may be looking to see how training stacks up financially against other ways to invest the company’s money, and whether the training, as carried out, was financially more effective than alternative forms of development
The business unit manager may be solely concerned with the impact on performance and productivity in achieving the goals of their department
The training and development manager may be concerned with the impact training programmes are having on the credibility and status as the training function within the company and its ability to secure investment in the future to drive further business performance enhancements

8. What is ROI? Elements of “Total ROI” Use of ROI Role of ROI
Reducing costs
Increasing benefits/performance/capability
Use of ROI
ROI of a training project’s contribution to business or project
ROI of one delivery method vs. another (e.g. ILT vs. e-learning)
Role of ROI
Pre-training assessment and expectation setting
Post-training feedback
Post-experience value measurement

9. The Process Model 1: Measure reaction &identify planned actions
Immediate reaction and response
2: Cognitive learning and retention
Skills, knowledge, attitude changes
3: Assessing application of learning to the job
Actual change in behaviour
4: Identifying business results
Quantify business value of change
5: Calculating ROI
Compare monetary value with costs

10. Getting Started Select your project to measure ROI measurement Plan
Select a significant project; align with significant goals
Start with a project that has clearly definable metrics
ROI measurement Plan
Data Collection Plan
Identify the “returns”
Identify the investment factors
Select the survey audiences and sources of data
Select your data collection methods
Analysis Plan

11. Investment Factors Costs incurred
Payments to suppliers and service providers
Time and attention to create and deliver the training
Opportunity cost
Time and costs involved in the ROI measurement exercise
Other Internal costs

12. Process Summary – Data Collection Plan
Data collected at different times to provide:
Pre-training baseline
Post-training change analysis
Post-experience change analysis
Data Collection Plan
State the objectives of the training / learning
State the objectives of each phase of data collection at each evaluation Level
Identify any previously used metrics, values or methodologies used by the client, and determine suitability for the current exercise
Select the appropriate evaluation methods
Identify the audiences who will be surveyed for data
Set the timing for the data collection
Allocate responsibilities for data collection and analysis

13. Process Summary – ROI Analysis Plan
Continuation of the Data Collection Plan, capturing information on the key items needed to develop the actual ROI calculation.
List Significant Data items (usually Level 4 or 3) to be collected
Benefit Factors
Cost Factors
Methods to isolate effects of the learning/training from other influences
Methods to convert data to numerical values
Intangible benefits
Other influences
Communication targets

14. Process Summary – Data Collection
Identify the purposes of the evaluation.
State clearly what the evaluations are to measure and what the goals of the training are intended to be.
Be as specific as possible about the goals of the training
Ensure goals address the performance enhancement, business improvement or cost savings expectations.
Select the evaluation instruments and methodology.
Establish the timing for the data collection.
Decide whether pre-training analysis is required, or post training analysis, or both. (e.g. pre-training and multiple post-training assessments may be necessary to effectively identify the skills changes in Levels 2, 3 and 4.)
Carry out the data collection at the levels 1-4

15. Process Summary – Sources of Data
Organisational Performance Records, showing outputs and measurements taken as part of the business’ normal reporting process
Testing and certification assessment records
Participant feedback
Instructor feedback
Feedback from participants’ supervisors/managers
Feedback from participants’ subordinates
Team/group peer feedback
Feedback from other internal or external groups (eg HR training departments)

16. Process Summary – Evaluation Methods
Identify how the data will be collected and analysed
Surveys
On the job Observation
Interviews
Focus groups
Action plans (or Performance contracts) and Program assignments
Performance data monitoring

17. Process Summary – Isolate effects of training
Separate training (personal productivity) component of performance change, from new software/systems/processes
Essential for credibility
Potential methods
Use control groups
Impact assessments and estimates by participants, managers, peers
Trend lines
Discount/adjust for over-estimates
Apply “inflation adjustment” for estimates and assessments

18. Process Summary – Convert to money
Convert data to monetary value
Specific costs and time incurred
Costs and time saved
Quality increase, reduced waste
Improved customer service and satisfaction
“Intangible” benefits (retention, commitment, fewer complaints, reduced conflicts etc)

19. Process Summary - ROI Calculate ROI
Benefit-cost ratio= Program Benefits
Program Costs
ROI % = Net Program Benefits
Program Costs x 100
Break-even time = Investment
Benefit x Period in months

20. Evaluation Levels 1. Reaction and Planned Action 2. Learning
3. Application and Implementation
4. Business Impact
5. Return on Investment
Kirkpatrick’s Four Levels of Evaluation; Phillips added ROI as the fifth.
Reaction and Planned Action – Frequency: each learning event.
Was the customer (trainee) satisfied with the experience? What did they like? What did they learn? Was there anything missing? Consider using a Likert rating scale for feedback.
Was the facilitator knowledgeable? Was the subject interesting? Were facilities adequate for the training? Was the training scheduled at a good time? Additional comments?
Learning – Frequency: pre- and post-training.
Was there a change in attitude, skills and/or knowledge? Assess learning before training, during training and after training to accurately assess learning.
Application and Implementation – Frequency: pre- and post-training and particular periods after training is complete (e.g., three months, six months, one year).
This speaks to behavior. Are the trainees doing things differently at work by using the trained knowledge and skills?
Pre-/post-test, observation, interview. Allow time for the change to occur. Ask employee, supervisor, subordinates for their perception of change in attitude or performance.
Business Impact – Frequency: regular intervals over the calendar or fiscal year – monthly or quarterly is typical.
This speaks to overall change for the business as a result of the training program—the results. Is there improved quality, improved production, decreased costs, increased job satisfaction, reduced problems or accidents, increased sales?
ROI – Frequency: With each new training event or when significant changes are made to existing events.
Consider the costs of training versus the benefits of training. How did the bottom line change? Were the benefits greater than the cost? 20 20

21. Writing Objectives Easy as A, B, C, D Audience: Who?
Behavior: What do “they” do?
Condition: What is the setting and method of evaluation?
Degree: Measurement to be met.
Objectives are very specific measures for an organization or an individual. In a training environment, we need to keep the organizational goals and the learning goals in mind as we write objectives for any single training session. Objectives guide the learning.
ABCD of writing objectives, using observable verbs:
Audience
The group of learners the objective is written for; “the learner…” or “the student…”
Behavior
The action or observable verb which describes what the learner will be able to do after completing the instruction (e.g., describe, compare/contrast, demonstrate). For a more complete list of verbs, please see the verb list in the readings.
Condition
The tools, resources, setting the students will have and the assessment method to be used.
Degree
The standard or degree of accuracy to be considered proficient. This can be based on a normative scale, measured against a standard of performance. 21 21

22. Example Objectives
Course objective: Learners will be able to make 15 entries in a customer database in 15 minutes with no more than 1 error.
Application objective: Learners will be able to reduce the data entry error rate by 50 percent over the next 6 months.
Impact objective: Employee time spent correcting database errors is reduced by 25 percent from last year’s rate.
Increasing Scope 22 22

23. Costs, Budgets, Accounting
Quantifying ROI means accounting for all the costs of the program.
Fixed costs: independent of the number of participants.
Variable costs: Dependent on the number of participants.
There are costs at every step – make sure to account for them all.
Costs, Budgets, Accounting (PowerPoint slides) - Fixed, variable, hidden, calculated, measured
Input measures – see budget sheet – give examples.
Fixed Costs vs. Variable Costs
Fixed costs are those that remain the same no matter how many individuals participate in the training. Examples include marketing and information distribution about the training, trainer’s time, employees’ time away from job for training,
Variable cost are those costs that change based on number of participants. Examples include training manuals and materials, meals (if provided), difference in cost between large room for 50 participants and small room for 15 (if applicable). 23 23

24. Benefits and Soft Skills
Change in:
Attitude, work climate, leadership, teamwork.
We desire these changes because they ultimately effect productivity.
Allow time for change in attitude or behavior, then measure these changes and report qualitatively.
Allow time for change in productivity, then measure for data and report quantitatively.
This is the soft data.
Typically, soft skills can be tied directly to hard data. For example, a better work climate reduces the amount of sick leave and turnover, and increases productivity; better leadership enhances new ideas, new revenue and output. 24

25. Create a Data Collection Plan
What?
New information that needs to be recorded?
Who?
When?
How?
Ask students to brainstorm: “What data do you need? What new information needs to be recorded? Who will record it? When will it be recorded? How will it be collected?”
Return to the course objectives, application objectives and impact objectives to ensure you know what information you will need to measure the effect of training.
Some of it may not be readily available and may require planning or coordination with other people to collect.
From whom will you collect the feedback/information?
Participants; Participants’ managers; Participants’ co-workers; Participants’ supervisees; Clients/Customers; Other
When will you collect the information?
During training; Right after training; After time has passed
How do you collect the information?
Surveys/Questionnaires; Interviews; Test performance (norm or criterion references) ; Simulations; Observations 25

26. Trend Line Example

27. Stakeholders: Review Customer Employee Supervisor Subordinate
President
Board of Directors
Stockholders
Review
Then, review who your stakeholders are – who needs or wants the information you have collected. 27

28. Thank You.. Any Questions: Dr. Pratik Surana +91-9890653263