1. Conceptual Developments in Evaluation of Training Effectiveness & Feedback
Dr. Shulagna Sarkar
Assistant Professor ,
Jt. Coordinator – PGDM – HRM
IPE, Hyderabad

2. EVALUATION OF TRAINING EFFECTIVENESS – THE SOLUTION

3. Why Should A Training Program Be Evaluated?
Identifying the program’s strengths and weaknesses
Assessing whether content, organization, and administration of the program contribute to learning and the use of training content on the job
Distinguishing trainees who benefited most or least from the program
Determining the financial benefits and costs of the programs
Comparing costs and benefits of training versus non- training investments
Comparing costs and benefits of different training programs to choose the best program

4. Challenges
Setting the right objectives and meeting those those objectives
Communicating outcome expectations to trainers as well as trainees.
Cooperation from each participants and people involved.
Change in Attitude: Shift from evaluation of the person to evaluation of the correctness of the program
Removing TRAINING WASTE - training that isn't needed is conducted, and the required training or other solution that would be truly beneficial is not even identified.

5. Introduction
Training effectiveness refers to the benefits that the company and the trainees receive from training
Training outcomes or criteria refer to measures that the trainer and the company use to evaluate training programs
Training evaluation refers to the process of collecting the outcomes needed to determine if training is effective
Evaluation design refers to from whom, what, when, and how information needed for determining the effectiveness of the training program will be collected

6. Evaluation
Formative evaluation – evaluation conducted to improve the training process
Helps to ensure that:
the training program is well organized and runs smoothly
trainees learn and are satisfied with the program
Provides information about how to make the program better
Summative evaluation – evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program
May also measure the return on investment (ROI) that the company receives from the training program

7. The Product – The training
What can be evaluated
Remember 3 Ps
The Plan
The Process
The Product – The training
S.M.Israr

8. How to evaluate the Plan
Course Objectives
Appropriate selection of participants
Timeframe
Teaching Methods

9. How to Evaluate the Process
Planning Vs. Implementation
Appropriate participants
Appropriate time
Effective use of time
Imparting training according to set objectives

10. How to Evaluate the Product
Is only evaluation of the product sufficient?
Time
Ultimately all stages require evaluation in any case
Changes in effectiveness
Impact Analysis
Achieving Targets
Attracting Resources
Satisfying Interested Parties

11. Kirkpatrick’s Four-Level Framework of Evaluation Criteria
Focus 1
Reactions
Trainee satisfaction 2
Learning
Acquisition of knowledge, skills, attitudes, behavior 3
Behavior
Improvement of behavior on the job 4
Results
Business results achieved by trainees

12. Only by assessing each level can we yield actionable results
Level 4 - Results Was it worth it?
Level 3 - Behavior KSA being used on the job?
Check Requirements, Systems and Processes
Level 2 - Knowledge Did they learn anything
Check Performance Environment
Level 1 – Reaction
Was the environment suitable for
learning?
Improve Knowledge/Skill transfer
Go from top down
What if we did nothing?
Applications of percentage – level 1, level 2, level 3 etc.
Improve Learning Environment

13. Outcomes Used in Evaluating Training Programs:
Return on Investment (ROI)
Comparing the training’s monetary benefits with the cost of the training
direct costs
indirect costs
Benefits

14. Return on Expectations: (ROE)
Showing the value of training begins before the program even starts.
The end is the beginning.
Value must be created before it can be demonstrated.
"Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program."

15. Achieving Targets Level of variation in product Productivity
Processing Time
Profit
Operating Cost
Rates of meeting deadlines
Cost/Income ratio
% of tasks incorrectly done
Level of variation in product
Ability to cope with circumstances
Time to reach job competency
levels of supervision required
Frequency and costs of accidents

16. CIRO Model Context Input Reaction
Outcomes – Immediate, Intermediate and Ultimate Outcome
Cognitive outcome
Skill based outcome
Affective outcome

17. CIPP Model
Content
Input
Process
Product

18. Hamblin’s Model
Reaction Level: It measures the reactions of the trainees to the content and methods of the training, not the trainer, and to any other factors perceived as relevant. It determines what the trainee thought about the training.
Learning Level: It measures the learning attitude of the trainees during the learning period. It collects information that did the trainees learn what was intended.

19. Job Behavior Level: The job behavior of the trainees in the work environment at the end of the training period i.e. did the training got transferred to the job?
Effect on the Department: Has the training helped the trainees in improving the department’s performance?
The Ultimate Level: It measures that has the training affected the ultimate well being of the organization in terms of the business objectives.

20. Kaufman’s Five level Level Evaluation area 1a Enabling
Quality input availability like, human and physical resources
1b Reaction
Methods, means and process acceptability and efficiency
2 Acquisition
Individual and small group mastery and competency
3 Application
Individual and small group utilization within the organization
4 Organizational Output
Payoff to the organization
5 Societal Outcomes
Payoff to the society

21. Outcomes Used in Evaluating Training Programs:
Return on Investment (ROI)
Comparing the training’s monetary benefits with the cost of the training
direct costs
indirect costs
Benefits
Simple ROI = Gains – Investment costs
Investment costs
ROI = Learning Experience + Work Environment = Business Results

22. Facilities rental (taking into account the opportunity cost)
Savings
Cost of training
Fewer errors
Reduced customer turnover
Less equipment downtime
Increased revenue collection
Faster equipment start – up time
Reduced employee turnover
Higher workplace morale
Reduced recruitment cost
Less time loss to grievance hearings
Facilitators fees
Training design
Course Material
Videos – Workbooks
Facilities rental (taking into account the opportunity cost)
Equipment rentals
Production downtime

23. Factors That Influence the Type of Evaluation Design
How Factor Influences Type of Evaluation Design
Change potential
Can program be modified?
Importance
Does ineffective training affect customer service, product development, or relationships between employees?
Scale
How many trainees are involved?
Purpose of training
Is training conducted for learning, results, or both?
Organization culture
Is demonstrating results part of company norms and expectations?
Expertise
Can a complex study be analyzed?
Cost
Is evaluation too expensive?
Time frame
When do we need the information?

24. The key lies in developing a Training Plan Training plan sets the platform for evaluating training programs.

25. Thank you…