Evaluating Training Effort Organisations are under pressure to justify various expenses. Business heads and training managers are under pressure to prove the effectiveness of training. Two basic issues to address when evaluating:-  How to design the evaluation study?  What to measure?

Measuring Training Effectiveness Basically three things that we can measure: Participants’ reactions to the program; What the trainees learned from the program; To what extent on-the-job behavior changed.

Designing the Evaluation Study Time Series:- Series of measures before and after the training program. Controlled Experimentation:- Formal methods for testing the effectiveness of a training program, preferably with before- and-after tests and a control group.

What to Measure? Kirkpatrick’s Taxonomy Reactions: Through “ Happy sheets questionnaire” or “360-degree questionnaire”. Learning Behavior Results

360 Degree Questionnaire Benefits of using a 360-degree format include: Workplace behaviour improvements The pre-program assessment can be used by the participant as a focus for their learning. The participant is given a much broader appreciation of the application of their behaviours. The reporting after the post assessment allows the individual to see how they have grown or developed as a result of the training program. It provides the facilitator with hard data for proving the cost- effectiveness of the training intervention to Management.

Issues against The Kirkpatrick framework represents a trainer’s notion of what constitutes effective evaluation, not a business manager’s. The second issue deals with hierarchy, the assumption that each level has more value than the previous one. The third issue deals with causality, the assumption that each level is caused by the previous one. Kirkpatrick’s taxonomic model is conceptually flawed as it does not fully identify all the constructs underlying the phenomena of interest. Furthermore, the results provided by the model can lead to incorrect decisions since major intervening variables are not included.

Other Approaches…. QAI India QAI is a leading global consulting organization (Asia’s largest and world’s third largest organization) addressing ‘Operational Excellence' in IT, BPO and Knowledge intensive organizations. Effectiveness of a training program is measured at 5 levels: I. Need for training II. Satisfaction of participants III. Learning at the classroom IV. Change in Behavior V. Improvement in results

ROI approach by Jack Phillips ROI = Net Program Benefits X 100 % Program Costs ROI couches results and expectations in the language of business and is often a hurdle a project must clear in order to secure funding. Given that learning is increasingly perceived as an investment by the organization, it seems only natural that it should be evaluated in the same way as other large investments, in terms of costs and benefits

Software Companies: They use Heuristic Evaluation Quality Score (HEQS) Steps of HEQS are:

1. Identify the application, parameters and the severity ratings 2. Standardize the scope of evaluation 3. Provide a knowledge transfer (optional) 4. Standardize the time for evaluation 5. Standardize the evaluation format 6. Proceed to evaluate 7. Use the individual evaluations to arrive at a benchmark 8. Use this benchmark to measure the skills of each evaluator 9. Derive Insights

The IBM Certified Trainer (ICT) program is a comprehensive program for Trainers' Preparation and Certification. Validates skills using three proficiency levels: IBM Authorized to Teach. (IAT) IBM Professional Trainer (IPT) IBM Training Expert (ITE) E.g. Blended learning programs encompasses multiple learning strategies and delivery formats, ranging from those that are solely technology-based to those that support interactions between people. Training effectiveness is measured by a control group of employees who receive no training. Provides real-world-based assessments with feedback Increase in its annual investment in learning programs that are blended (with increases between 30 and 40 percent each year for the past four years).

Management Development Programmes(MDPs) have been institutionalised. Wipro has different programmes catering to different life-cycle stages of the manager. Different people may get different amounts of training at different times. 360 degree feedback. Training Quality measured against six sigma.

Manufacturing Company Featherlite India Training given on various topics Training conducted every 3 months or 6 months depending on requirements In-house and external trainers Happy sheet Questionnaire known as “ Survey Forms” Monthly Performance Appraisal Interview Rating Sheets

Conclusion Measuring Training Effectiveness is different for software and manufacturing companies. The purpose of Evaluation is to improve the probability of investment providing a good return to the business. Measurement of training effectiveness is a hallmark of good management.