1. SAMigo Assessment Tool (aka Tests and Quizzes ) Marc Brierley, Stanford University Interaction Designer http://bugs.sakaiproject.org/confluence/ display/SAM

2. Agenda Feature Overview –Focus on pedagogical flexibility using assessment types Short Demo Futures Q&A

3. What are T&Q’s features: Assessment Authoring Assessment Publishing Assessment-Taking Management Assessment Grading Question Pools Management Assessment-Type (formerly Template) Management

4. Why “Assessments”? More than just test and quizzes For Example: –Problem Sets –Formative Assessments –Surveys The name “Tests and Quizzes” is a compromise

5. Organize & Publish Assessments Core (pending) Assessments for a course –Starting page for assessment authoring –Can be ex/imported in QTI XML format –When ready, published via Settings Published Assessments for student assessment-taking –When students respond, scores link appears –Can change release, due, retract dates

7. Question Types Multiple Choice (single and multiple correct) Multiple Choice Survey True/False Matching Essay/Short Answer Fill in the Blank File Upload Audio Recording* * New for Sakai 2.2 release

9. Rich-Text & Attachments In section headers, question texts, selections for responses, and feedback

11. Pedagogical Flexibility Online test Self-study questions Homework, problem-sets Paper, code, or project submission Language drills Quick knowledge probes Surveys Others???

12. Assessment Types Create assessment “types” for different disciplines, different uses Hide complexity by selecting what which settings are visible to the instructor. Pre-set defaults by making setting choices for the type (e.g., Self-study always has immediate feedback)

13. Assessment Creation Workflow

19. Online Test Unique tests for each student –Randomized order of questions –Randomized draw from question pools Timed test-taking during access window –Auto-submit at end of timed period One submission only per student Higher Security –IP Addresses restricted –Secondary Password (proctored) No late submissions accepted Scores transferred to gradebook

21. Self-Study Questions Immediate feedback Random access to Questions –Table of contents –Marked for review list No record of score in gradebook No due date; always available to student Unlimited submissions allowed

23. Homework/Problem Sets Published with multiple release and due dates for different sections Multiple methods for handling late submissions Students can save work during assignment period. Auto-grading with methods for diagnosing learning problems Quick-reviews of student aggregate performance by instructors viewing histograms, statistics. Pools for organizing questions for reuse

24. Essay and Project Submission Use file upload question type for submitting any type of document Grader can download assignment; upload marked-up file to return to student Grades are recorded in gradebook

25. Question Pools Organize questions into multiple subpools Author Questions from within Pool Mgr. Randomly draw from pools for an assessment

28. Demo

58. 2.3 Highlights Attachment of files and URLs to Audio Question Type w/ signed certificate Bulk download of student files Filtered view in grading Page navigator and search capability for the Grading pages New question type: Numeric Responses –By Diego del Blanco Orobitg (Universidad Politécnica de Valencia)

59. 2.4 Roadmap and beyond 2.4 –email student feedback from grading screens SAK-7094 –performance enhancements to student assessment taking (delivery) –bug fixes Beyond?...stay for the BOF

60. Questions?

61. Contacts Marc Brierley, Stanford University T&Q Lead Designer marc.brierley@stanford.edu Lydia Li, Stanford University T&Q Project Manager lydial@stanford.edu Homepage: http://bugs.sakaiproject.org/confluence/display/SAM

62. Where did the design originate? Formative Assessment (Homework) –Stanford Assignment Tool for Human Biology –CourseWork CMS with Language Tools (Andrew W. Mellon Foundation funded) –OKI Tools Workshop to Specify Functionality (Andrew W. Mellon Foundation funded) –OKI Formative Assessment Tool (Andrew W. Mellon Foundation funded) Summative Assessment (High-Stakes tests) –OnCourse Quiz Tool

63. Where did the design originate? OKI Tools Workshop –Many schools met to define needs for a CMS –One group specified the needs for an Assessment Tool –Several hundred functional requirements identified

64. Where did the design originate? Assignment and Assessment Manager –Supports teaching and learning in large lecture science courses and in language instruction –This OKI Tool was developed at Stanford –Andrew W. Mellon Foundation funded

65. Where did the design originate? AAM design combined with OnCourse Quiz tool design “Navigo” (2003) Stanford and Indiana join Sakai project, Stanford rewrites tool to Sakai Style Guide TPP, tool renamed SAMigo (2004-present)