1. www.sakaiproject.org 1 Tests and Quizzes Tool (aka SAMigo) Marc Brierley, Stanford University T&Q Project Manager and Lead Designer Charles Kerns, Stanford University Learning Activity Designer

2. www.sakaiproject.org 2 Where did the design originate? Formative Assessment (Homework) –Stanford Assignment Tool for Human Biology –CourseWork CMS with Language Tools (Andrew W. Mellon Foundation funded) –OKI Tools Workshop to Specify Functionality (Andrew W. Mellon Foundation funded) –OKI Formative Assessment Tool (Andrew W. Mellon Foundation funded) OnCourse Quiz Tool –Online testing

3. www.sakaiproject.org 3 Where did the design originate? OKI Tools Workshop –Many schools met to define needs for a CMS –One group specified the needs for an Assessment Tool –Several hundred functional requirements identified

4. www.sakaiproject.org 4 Where did the design originate? Assignment and Assessment Manager –Supports teaching and learning in large lecture science courses and in language instruction –OKI Tool developed at Stanford –Andrew W. Mellon Foundation funded

5. www.sakaiproject.org 5 Where did the design originate? OnCourse CMS for Indiana University instructional support

6. www.sakaiproject.org 6 What are its features: Assessment Authoring Assessment Publishing Assessment-taking Management Assessment Grading Question Pools Management Publishing Template Management

7. www.sakaiproject.org 7 Organize & Publish Assessments Library of assessments for a course –Can be ex/imported in QTI XML format Published assessments with responses, grades –Can be published to sections, groups in course –Can have different release, due, retract dates

8. www.sakaiproject.org 8

9. 9 Question Types Multiple Choice (single and multiple correct) Multiple Choice Survey True/False Matching Essay/Short Answer Fill in the Blank File Upload Audio Recording* * in future Sakai 2.x release

10. www.sakaiproject.org 10

11. www.sakaiproject.org 11 Multimedia Everywhere WYSIWYG editor Fonts, style, layout, images, HTML, links In section headers, question texts, selections for responses, feedback, comments

12. www.sakaiproject.org 12

13. www.sakaiproject.org 13 Pedagogical Flexibility Online test Self-study questions Homework, problem-sets Essay, code, or project submission Language drills Quick knowledge probes Others???

14. www.sakaiproject.org 14 Online Test Unique tests for each student –Randomized order of questions –Randomized draw from question pools Timed test-taking during access window –Auto-submit at end of timed period One submission only per student Higher Security –IP Addresses restricted –Secondary Password (proctored) No late submissions accepted Scores transferred to gradebook

15. www.sakaiproject.org 15

16. www.sakaiproject.org 16 Self-Study Questions Immediate feedback Random access to Questions –Table of contents –Marked for review list No record of score in gradebook No due date; always available to student Unlimited submissions allowed

17. www.sakaiproject.org 17

18. www.sakaiproject.org 18 Homework/Problem Sets Published with multiple release and due dates for different sections Multiple methods for handling late submissions Students can save work during assignment period. Auto-grading with methods for diagnosing learning problems Quick-reviews of student aggregate performance by instructors viewing histograms, statistics. Pools for organizing questions for reuse

19. www.sakaiproject.org 19 Essay and Project Submission Use file upload question type for submitting any type of document Grader can download assignment; upload marked-up file to return to student Grades are recorded in gradebook

20. www.sakaiproject.org 20 Templates Create Assessment “Types” for different disciplines, different uses Select what which choices are available to the instructor. Hide complexity by making choices for the type (e.g., Self-study always has immediate feedback)

21. www.sakaiproject.org 21

22. www.sakaiproject.org22

23. www.sakaiproject.org 23 Question Pools Organize questions into multiple subpools Export and import questions in IMS QTI format. Author Questions from within Pool Mgr. Randomly draw from pools for an assessment

24. www.sakaiproject.org 24 Demo

26. www.sakaiproject.org26

27. www.sakaiproject.org27

28. www.sakaiproject.org28

29. www.sakaiproject.org29

30. www.sakaiproject.org30

31. www.sakaiproject.org31

32. www.sakaiproject.org32

33. www.sakaiproject.org33

34. www.sakaiproject.org34

35. www.sakaiproject.org35

36. www.sakaiproject.org36

37. www.sakaiproject.org37

38. www.sakaiproject.org38

39. www.sakaiproject.org39

40. www.sakaiproject.org40

41. www.sakaiproject.org41

42. www.sakaiproject.org42

43. www.sakaiproject.org43

44. www.sakaiproject.org44

45. www.sakaiproject.org45

46. www.sakaiproject.org46

47. www.sakaiproject.org47

48. www.sakaiproject.org48

49. www.sakaiproject.org 49 2.0 Highlights Full integration into Sakai framework Assessment scores to Gradebook File upload question type Play with it: http://nightly2.sakaiproject.org/portal

50. www.sakaiproject.org 50 T&Q 2.x Future Implementation of 2.0 framework –Release assessments to specific classes and sections –Department template –Question Pool sharing Audio-recording question type

51. www.sakaiproject.org 51 T&Q post 2.x Future Establish a T&Q sub-team (designers and developers) New features via Assessment DG (post Baltimore conference) and feature prioritization process New code via committer strategy that mirrors Sakai process See you in Austin! (Dec. SEPP conf.)

52. www.sakaiproject.org 52 Questions?

53. www.sakaiproject.org 53 Contacts Marc Brierley, Stanford University T&Q Project Manager and Lead Designer marc.brierley@stanford.edu Charles Kerns, Stanford University Learning Activity Designer charles.kerns@stanford.edu