1. Learning and Technology Services Online Assessments at Sauder Not your parents’ multiple choice exams...

2. Learning and Technology Services Roadmap TODAY: Introductions and Assumptions Start at the start = Learning Objectives Overview of the Vista Assessment tool Breaking the mould/what is possible COMM 293 Case Analysis Publishers Resources

3. Learning and Technology Services Introductions Who are you? What do you do? Why are you here?

4. Learning and Technology Services Learning Objectives

5. Learning and Technology Services Tactics

6. Learning and Technology Services Learning Objectives The Springboard Workshop participants will be able: To challenge current perceptions about online assessments To acquire a working knowledge of the available components of an online assessment (Quiz) tool To stimulate creative thinking around the (almost) unlimited power available in using an Online Assessment (Quiz) tool To connect the value of how Online Assessments can support student learning To appreciate the vast resource pool available online and from publishers

7. Learning and Technology Services Assumptions Today’s focus is on leveraging the specific Quiz tool for two primary benefits: Automatic grading Extensive analysis of results for measurement of reliability/validity Not a technical workshop/presentation Encouraged to challenge/question/debate You are not alone Suspend what you believe about what online assessments are; and are not.

8. Learning and Technology Services Overview Key Points: Wide variety of objective question types We do not build the assessment first, we build a database of questions which we then link to an assessment Tracking and feedback for students, faculty and UGO for pro- active intervention Alleviates Marker issues Can contribute to both formative and summative assessment activities, as well as many non-traditional ones Multiple randomization variables Question answers (a,b,c,d) Question order (1-10) Question sets (10 of 20)

9. Learning and Technology Services Assessment Types (Vista Lexicon) Quiz Fully tracked and ‘controllable’ Results can populate the Gradebook for Formative grade aggregations and/or used for Selective Release triggers Survey Can contain exact same questions, but ALL responses are anonymous NOTE: we can ‘see’ if a student has completed, but NOT how they responded Self-Assessment Can utilize all available questions from the Database for self- access NOTE: NO RESULTS CAN BE TRACKED BY ANYONE OTHER THAN THE STUDENT

10. Learning and Technology Services Vista Question Types Hand out summary document

11. Learning and Technology Services Reporting Tools Ability to produce reports Overall stats by: Student Question (including discriminator data) Based on an external data point (i.e. Program Year) Full ability to export as CSV for higher- order analysis (i.e. SPSS)

12. Learning and Technology Services Current Sauder uses Summative drill and practice Formative midterm and final exams CUS elections CUS APT Midterm self-evaluations Selective Release triggers

13. Learning and Technology Services Examples of non-standard usage of the Assessment tool Student self-evaluation. Unfortunately, I cannot attend on Monday, but, please feel free to say that I am a big believer in the importance of participation grading not appearing like a black box and something that “happens to” students with a mystery grade at the end. Providing a rubric (and mine is simple!) and inviting students to self-evaluate half-way through, has, I have found, been fantastic for: improving student accountability; raising self-awareness; improving participation levels in the second half of term; reducing complaints on grading (to near zero.) changing the balance of my grading time (not all at the end.) I’ve done this for 6+ courses over the last 18 months and have some good data. Paul Cubbon, Marketing Instructor

14. Learning and Technology Services Tour of what is possible CSS samples Student self-evaluations Embedded media 3-dimensional question Interactive embedded Java Applet RSS feeds

15. Learning and Technology Services Comm 299 Case Study Overview of story Summary of Exam structure exam composition versions randomization administration Results Blended Assessment with rough work on paper Evidence (or lack thereof) of true concept acquisition

16. Learning and Technology Services Comm 293 Exam/Assessment results Midterm 2 March 4, 2011 Assignment Marks to-date Final Marks, 2009w Assignment Marks, 2009w

17. Learning and Technology Services Do I have to write my own questions? Respondus Publisher list http://www.respondus.com/partners/publisher_list.shtml NOTE: this list is not exhaustive as many other publishers/resources can be mined for questions