1. Kaplan University Writing CenterSaturday, March 05, 2016 1 IX520 Needs Assessment and Evaluation in Education and Training Unit 7 Seminar September 13, 2012

2. Kaplan University Writing CenterSaturday, March 05, 2016 2 In this seminar, we will … Discuss evaluation strategies Apply the Rapid Evaluation model to a scenario Address any questions

3. Kaplan University Writing CenterSaturday, March 05, 2016 3 What is evaluation?

4. Kaplan University Writing CenterSaturday, March 05, 2016 4 What is evaluation? Evaluation is a process that critically examines a program. It involves collecting and analyzing information about a program’s activities, characteristics, and outcomes. Its purpose is to make judgments about a program, to improve its effectiveness, and/or to inform programming decisions (Patton, 1987).

5. Kaplan University Writing CenterSaturday, March 05, 2016 5 Why do we evaluate?

6. Kaplan University Writing CenterSaturday, March 05, 2016 6 Why do we evaluate? To determine the extent to which a program or intervention is effective (i.e., to determine if it is successful or how well it meets its objectives). It is necessary to routinely appraise, and subsequently adjust, learning strategies to ensure their effectiveness. Evaluation can help to identify areas for improvement and avenues for greater efficiently. Evaluation helps to validate a solution’s progress and success, provide an impetus for improving the program, or substituting with a different approach.

7. Kaplan University Writing CenterSaturday, March 05, 2016 7 How do we evaluate?

8. Kaplan University Writing CenterSaturday, March 05, 2016 8 A. D. D. I. E.

9. Kaplan University Writing CenterSaturday, March 05, 2016 9

10. Kaplan University Writing CenterSaturday, March 05, 2016 10 Alkins' UCLA Model CIRO (Context, Input, Reaction, Outcome) Daniel Stufflebeam's CIPP Model (Context, Input, Process, Product) Eisner's Connoisseurship Evaluation Model Jack Phillips' Five Level ROI Model Kaufman's Five Levels of Evaluation Michael Scriven's Goal-Free Evaluation Approach PERT (Program Evaluation and Review Technique) Portraiture Model Provus's Discrepancy Model Robert Stake's Responsive Evaluation Model Robert Stake's Congruence-Contingency Model Illuminative Evaluation Model Which Model? So many to choose from…

11. Kaplan University Writing CenterSaturday, March 05, 2016 11 LevelEvaluation TypeDescriptionSample Tools 1Reaction Reaction evaluation relates to how the participants felt about the training experience. How well did they “like” the learning process? Post-training surveys or questionnaires. Likert scale survey (quantitative). Verbal reactions (qualitative). 2Learning Results Learning evaluation is the measurement of the change in knowledge - before and after the event. What did they learn? (the extent to which the learners gain knowledge and skills) Pre & Post test Interviews & observation Kirkpatrick’s Evaluation Levels

12. Kaplan University Writing CenterSaturday, March 05, 2016 12 LevelEvaluation TypeDescriptionSample Tools 3Behavior Behavior evaluation is the extent to which new learning is applied. What changes in job performance resulted from the learning process? (capability to perform the newly acquired skills) Observation and interview over time are required to assess the degree, significance, and sustainability of change. 4Business Results Results evaluation refers to the resulting impact on the business or environment. What are the tangible results of the solution in terms of reduced cost, improved quality, increased production, efficiency, etc.? Employee appraisals and evaluations. Performance charting. Business reports. Key performance indicators. Kirkpatrick’s Evaluation Levels

13. Kaplan University Writing CenterSaturday, March 05, 2016 13 The purpose of the evaluation strategy for this University is to validate the leadership training program. Barksdale and Lund (2011) state that “the purpose of an evaluation strategy is to provide you and your stakeholders with a clear vision of what you want the evaluation to accomplish for your performance improvement solutions, products, and services (pg. 7).” The purpose of the evaluation strategy for Bailie University is to provide the stakeholders with reputable data on how the recently launched leadership program has impacted the demonstration of leadership characteristics and the retention of senior academic administrative staff. The information provided by the evaluation process will also be useful in the enhancement of future professional development initiatives and hiring practices of the university. Which one?

14. Kaplan University Writing CenterSaturday, March 05, 2016 14 The Evaluation Framework Step 1: Identify the Evaluation Strategy – Unit 6 Step 2: Establishing Measures – Unit 7 Step 3: Developing higher level analysis tools – Unit 8 Step 4: Data analysis strategies – Unit 8

15. Kaplan University Writing CenterSaturday, March 05, 2016 15 Questions? Please feel free to ask any questions that you might have regarding things that we are, or will be, doing. Please don ’ t be shy …

16. Kaplan University Writing CenterSaturday, March 05, 2016 16 Thank You Contact me: Dr. Jeff Bailie Jbailie@kaplan.edu 605.430.0509 Jbailie@kaplan.edu