Evaluating the Effectiveness of Your Training: Kirkpatrick’s 4 Levels Left Click to move to next slide and begin next audio. PRESS the F5 key to begin the slideshow.

The Principles and Process of Good Training Development & Delivery 1.Training Needs Assessment 2.Establish Learning Objectives 3.Content 4.Delivery 5.Evaluation Is the training program a quality program? Does it follow...

National Institute of Corrections Model: Instructional Theory Into Practice (ITIP) Training Cycle Needs Assessment Evaluation Deliver Training Pilot Test Set Performance Objectives Revise Design Training Train Instructors

Assessment of Training Needs In order to be effective, a training must be based on the needs of the trainees, and the “needs” or goals of the organization. Identify “The Gap” -- between what skills employees actually have --and what they should be Who should determine need for training? Management, trainees themselves, co- workers, specialists in the field,…

Needs Assessment Sample Questionnaire Rate your competence in each of the following areas giving a 1 to the lowest and a 5 to the highest. Give a different number to each. RatingJuvenile Careworker Skill Establishing clear expectations Balancing positive & negative feedback Giving immediate and appropriate consequences How to use your facility’s behavior management plan

Establishing Learning Objectives Skills-based or competency-based objectives Knowledge-Based objectives: more likely to include what the learner will understand or be able to explain Attitudes: individual attitudes needed to fully perform

Learning Objectives Competency goals should be: Behaviorally anchored Observable Forward-thinking Discrete User-friendly Example: Supervisor frequently gives accurate and specific feedback to subordinates about their performance. Hackett

Training Content Does content meet best practices established by leading organizations in the field of juvenile confinement? (NIC, NJDA, CJCA, NCCHC) Does the content cover the needs of staff? Does it accurately reflect local policies and procedures? Is the content timely? Does it reflect current research and theory?

Training Delivery Meets the needs of all learning styles – active/kinetic, visual, social, audio Covers all parts of the learning cycle – perceiving, reflecting, applying, creating Organized presentation Positive learning environment Empathetic, knowledgeable, professional trainers Et cetera

Training Evaluation: Does the training actually work? Does the training leave your staff feel more motivated, more confident, more knowledgeable? Does the training affect the way staff do their job? How are youth affected by the change? Is it causing the change in staff’s performance that we want? To the degree that we want?

Evaluating Training Effectiveness: Kirkpatrick’s 4 levels Level 1: Reaction Level 2: Learning Level 3: Behavior Change Level 4: Results

Level 1 Evaluation: Reaction Training Evaluation Did you like this training? Was the material relevant to your job? Was the trainer knowledgeable? What do you think was the most important thing you learned? Are the handouts helpful? Comments: _______________ A positive reaction does not guarantee learning, but a negative reaction almost certainly reduces its possibility.

Level 2 Evaluation: Learning Other methods can be less formal, such as trainer observations during practice, team assessments, or self-assessments.

Level 3 Evaluation – Behavior Change on the Job Guidelines: 1.Use a control group if practical 2.Evaluate both before and after the training if possible 3.Survey and/or interview people who can observe the behavior 4.Get 100 percent response or a random sampling 5.Repeat the evaluation at appropriate times including after allowing a time lapse

For trainee: How much has the program improved the working relationship between you and your co-workers? Large extent Some extent No change Made worse For supervisor: Since the training, how well does the trainee do in making decisions? Much better little better The same Little worse Much worse For subordinates & trainees: Since the training, how much much does she/he ask for your opinion or support to your ideas? More often 15 trainees Little more 5 trainees The same 0 trainees Less often 0 5 staff15 staff10 staff 0 Level 3 Evaluation continued: Example questions

Level 4 Evaluation: Organizational Impact Measures broad-based changes in areas that are important to the agency as a whole, such as: –decreased legal costs –Safety improvements –Increased arrests Compare measures before and after training. Complicated and Expensive

Sample results of a Level 4 Evaluation Change in use of physical restraints as measure of Behavior Management training Number of incidents