Assessing Learning Outcomes Dr. Deb Gebeke, Facilitator

Apps’ Textbook O Original Extension Evaluation Textbook written in 1991…over 25 years ago! O Page …only 10 pages devoted to assessment/evaluation O Happiness Index vs. Outcomes and Impacts O How will you know you are making a difference? Spending resources wisely?

Apps’ Summary O Evaluation: Determine something’s value O Program Evaluation: Determine to what extent the objectives are realized O If objectives are realized, does that mean the class was a success? O That participants learned something of value?

Textbook: Apps’ Summary O Evaluation can be both formal and informal. O Evaluation can be conducted at different times and for different reasons: O Before: Needs Assessment O During: To make changes based on feedback O After: Assessing Outcomes O App’s approach is not vigorous enough for today

What are the Four Stages of Learning? O Stage 1: Ignorance O Stage 2: Confusion O Stage 3: Confidence O Stage 4: Mastery O’Connor and Seymour, 1993

Stage 1: Ignorance O Unconscious Incompetence O We are unaware of a skill. O Unaware that one cannot do a Task. “I don’t know that I don’t know.”

Stage 2: Confusion O Conscious Incompetence O Conscious attention is on the task and the results are variable. This is the stage when the learning rate is the greatest. O Aware of the task, but cannot do it. “I know that I don’t know.”

Stage 3: Confidence O Conscious Competence O Full conscious attention is still needed to carry out an activity. The skill is not yet fully integrated and habitual. O Able to think through a task step by step and do it “I know that I know. I get it”

Stage 4: Mastery O Unconscious Competence O Skill has been fully integrated and is habitual O Can do the task without thinking about intermediate steps “I can just do it. I have integrated what I learned into practice”

Four Stages of Learning UnconsciousConscious CompetentStage 4: MasteryStage 3: Confidence IncompetentStage 1: IgnoranceStage 2: Confusion

Four Levels of Evaluation O Dr. Donald Kirkpatrick: Model began over 50 years ago as a sales and training evaluation tool O Kirkpatrick’s Four Levels of Evaluation 1. Reaction 2. Learning 3. Behavior Change 4. Results O Print and Post Summary card- luation%20Cards.pdf luation%20Cards.pdf

Reaction to web-based summary clips (Dr. Myron Eighmy) O Why Evaluate (.51 min) O Four Levels (1.35 min) O Reaction or Satisfaction (3.22 min) O Learning (6.47 min) O Behavior Change (5.19 min) O Impact (4.25 min) O The Ripple Effect (3.19 min) Detailed Chart for Four Levels of Evaluation (business model) evaluation-in-detail

What resources are available to assist in overall planning and evaluation for programs? O NDSU: Worksheet and guidelines for completing program planning worksheet planning-resources O Evaluation Tool Kit (samples):

How do you know when your program is successful? O Objectives are met O Sample Basic Tool for evaluating programs O instruments/learningeval.docx instruments/learningeval.docx

Consider adding other items O Was the information helpful? O How will you use the information? O How effective were each of the strategies? O Demographic information O Open ‐ ended questions to allow for feedback on items that learner feels are important? O What behavior do you intend to change? Be ready for follow up survey.

Methods and Timing O Survey Monkey, Qualtrics and other electronic resources O During program, post then pre or retrospective, mail, phone, etc.

Resources O American Evaluation Association O Bloom’s Taxonomy of Action Verbs - uments/blooms-taxonomy-1 uments/blooms-taxonomy-1