D EVELOPING YOUR E VALUATION T OOLKIT Focus on Learning 2013 Paula Buskard and Sandy Odrowski

A GENDA  Session Outcomes  Overview of what/why we evaluate  Aligning evaluation methods to outcomes  Evaluation Planning  After the Evaluation: Debriefing  Q & A

S ESSION O UTCOMES By the end of this session the participant will have the ability to:  Select suitable evaluation tools that are learner– centred and align to learning outcomes  Identify best practice assessment tools that prepare the instructor for writing great evaluations.  Identify ways to prepare the learner for evaluation.  Discuss ways to conduct effective post test debriefing for both learner and instructor.

P LACE MAT A CTIVITY  Each member will takes a square/triangle and individually writes 3 questions/concerns they have about evaluation.

R ECORD THE GROUPS TOP 3 CONCERNS IN THE CENTRE

E VALUATION –W HAT IS IT ??

F ORMATIVE  diagnostic tool (Where are learners at? What have they got so far? Do I need to go over something again?)  help learners take stock of and assess their own progress towards learning goals

S UMMATIVE :  assess a learners achievement of meeting the learning outcomes  determine grades

A N A NALOGY OF L EARNING... 9 Formative Summative

S OMETHING TO THINK ABOUT  What is Needed to know (Learning Outcome)  What is Nice to know (Filler)

W HAT TO EVALUATE ?  The Course Learning Outcomes describe clearly what learners will know and be able to do at the end of the course.  They are performance-based and results oriented.

T HE G OLDEN C HAIN

S NOWBALL E XERCISE  Each group move to a posted Learning Outcome  Take 2 minutes (I will call time) to identify evaluation methods that could be used to measure learning.  When time is called, move to the next Learning Outcome and repeat.  Once each group has had the opportunity to identify evaluation methods for each Learning Outcome let us know.  Share in large group.

W HAT DID YOU LEARN TODAY  Working with a partner, fill in the Handout labeled Evaluation Strategy Matching Exercise.

E VALUATION M ETHODS A DVANTAGES AND C ONCERNS  In your group:  Take a moment to discuss the Advantages and Concerns regarding the evaluation method listed on the flip chart paper at your table.  Record your findings and be prepared to discuss with the larger group

T HE GOODS  “Will this be on the test?”  “What do I really have to know?”  “How do I answer this question?”  “Why did I get this question wrong?”

What do you want your learners to know?

F OUR P OINTS TO C ONSIDER :  1 – Measure what is most important rather than what is easiest to measure  2 - Be aware of the learning embedded in evaluation  3 – Understand the evaluator’s power  4 – Take care not to reduce the curriculum to only what is evaluated

G UIDELINES FOR PLANNING  Start early (holidays, sequence, scaffolding)  Determine the number of items and time limitations  Construct the question- make sure that it measures learning and is linked to content objectives/outcomes  Edit the questions for errors in spelling, sentence structure, etc.

G UIDELINES FOR PLANNING C ONT ’ D …  Group the questions according to type and topic (make easier ones first)  Write directions (include marking scheme)  Have the answers “ready” for post test debrief  Review

H OW TO WRITE A GOOD E VALUATION  Learner-Centered and Authentic Assessment ChecklistLearner-Centered and Authentic Assessment Checklist  Developing Good Test Questions Developing Good Test Questions

T HE G OLDEN CHAIN ? Develop a business plan for a small non profit organization. Lecture and homework: practice multiple choice questions from the text Evaluation: Final Exam: multiple choice. Course Learning Outcome Learning and practice methods Evaluation Method

T HE G OLDEN CHAIN ? Develop a business plan for a small non profit organization. Lecture: types of business and parts of a business plan/. Case studies Student develop a business plan for a mock or real non profit organization Course Learning Outcome Learning and practice methods Evaluation Method

T EACHER D EBRIEF Exercise: In your small groups answer these 2 questions:  What could teachers consider after a test?  What could teachers do to help their learners after a test?

T EACHER D EBRIEF : CLEAN UP  Did my learners understand what was asked on the test?  Were there questions that the learners missed that were the fault of the teaching? How will I remedy this?  Do the results indicate that the learners were prepared for the evaluation?

T EACHER D EBRIEF : CLEAN UP C ONT ’ D If I use this procedure again, what changes will I make? What other methods of evaluation might be more appropriate? Is this an evaluation that must be returned? Is there evidence of academic dishonesty? If so, what can I do about it (this time and in the future?)

L EARNERS D EBRIEF  Quickly go over the test and provide the correct answers to all of the questions.  Discuss the outstanding questions – those that most students answered well, those that some may have got stuck on, etc.  Open the floor for focused discussions (learning opportunity).  After a reasonable time is spent on a class discussion, invite students to make appointments with you to discuss individual concerns.  24 hour rule

E VALUATION WORKS BEST WHEN …  Provide clear, objective assignment guidelines  Provide samples, if possible, of desired product  Provide a marking rubric showing where the marks will come from  Tie evaluation strategy directly to the learning outcome being measured Domain/level

A NY Q UESTIONS /C ONCERNS LEFT ?

A NY OTHER Q UESTIONS ?

D O YOU W ANT TO L EARN M ORE ?  Durham College Assessment Using Technology Durham College Assessment Using Technology  Eastern Region ABC program Excellent Evaluation Resources Eastern Region ABC program Excellent Evaluation Resources  Developing Good Test Questions Developing Good Test Questions

D O YOU WANT TO L EARN M ORE ?  Andrus, C and Bloor, D (2005) Teaching tools  Gross-Davis, B. (2001). Tools for teaching. San Francisco: Jossey-Bass  Fenwick T. and Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Toronto: Thompson Educational Publishing  objectives.pdf objectives.pdf  