1. Assignment & Assessment Tool: Feature Review

2. Goals for Today Quick review of AAM (Assessment Assignment Manager) Introduce Template system for AAM Introduce proposed feature set for Assignment/Assessment Get Your Feedback on features Discuss design for document submission and peer review Audience for today’s presentation Instructional Tech Support Staff who can provide on features in AAM

3. AAM-What is it? Tool that replaces the Assignment Tool in CourseWork Design focus is on formative assessment (homework) Two areas of emphasis-large lecture courses and language instruction Working with Michigan and Indiana to build modular, plug-in tool for OKI frameworks

4. Timeline June 15, 2003 AAM Alpha Aug 1, 2003 AAM Beta Jan 1, 2004 AAM 1.0 Apr 1, 2004 AAM 1.1 (with language and large lecture interfaces and features)

5. AAM Team Design Marc Brierley Charles Kerns Julie Mai Huong Nguyen Development Rachel Gollub Sherwin Lu Huong Nguyen Qingru Zhang Ganapathy Subramaniam Daisy Flemming

6. Communication with AAM Team AAM Design Blog: http://www.stanford.edu/group/course work/AAMdesign/ http://www.stanford.edu/group/course work/AAMdesign/

7. Template Editor Authoring Templates and Assessments Template File ATS Creates Assessment Editor Assessment File (Based on Template) Instructor Creates

8. Templates All templates have default characteristics which define release, presentation, evaluation, and feedback. Template characteristics can be set to be: Either displayed or hidden from instructors and students Either locked or editable by instructors

9. 12 Item Access - Sequential If Sequential, then one must proceed from web page to web page with no return allowed. Nav buttons for Next page only. Assessment Header Part 1 Next >> Assessment Header Part 1 Assessment Header Part 1 Next >> Done

10. 12- Item Access – Random Access to Parts If Random Access to Parts, then one may proceed to any “chunk” from Table of Contents but sequential path is avail. Nav buttons for prev and next page. Assessment Header Part 1 Next >> Done Assessment Header Part 1 Assessment Table of Contents Part 1 Part 2 TOC<< PrevDone TOC

11. Assessment Header Part 1B Question 1 Assessment Header Part 2 Question 1 Question 2 12- Item Access – Random Access to Items If Random Access to Items, then one may proceed to any item from Table of Contents but sequential path is avail. Nav buttons for prev and next page. Assessment Header Table of Contents Part 1A Q1 Q2 Part 1B Q1 Part 2 Q1 Q2 << PrevDoneTOC Assessment Header Part 1A Question 1 Question 2 << PrevTOCNext >> TOCNext >>

12. Assessment Header Part 2 Question 1 Question 2 13- Bookmarking of Items If Bookmarking = TRUE, then toggling bookmark button/flag is located at each question, a Bookmarks button at bottom of each page, and A bookmark page has links to each of the bookmarked questions Assessment Header Bookmarks Part 1 Q2 Part 2 Q1 << PrevDoneTOC Assessment Header Part 2 Question 1 Question 2 TOCNext >>Bookmarks X BM - BM X BM - BM

13. 76- Display Chunking If DisplayChunking = byitem, then each item is displayed on a separate web page Assessment Header Part 1 Question 1 TOCNext >>Bookmarks Assessment Header Part 1 Question 2 << PrevDoneTOC Bookmarks - BM

14. 76- Display Chunking If DisplayChunking = byPart, then each part is displayed on a separate web page Assessment Header Part 2 Question 1 Question 2 TOCNext >>Bookmarks - BM Assessment Header Part 3 Question 1 Question 2 TOCDone Bookmarks - BM << Prev

15. 76- Display Chunking If DisplayChunking = byMultiPart, then each part is can displayed on a multiple web pages Assessment Header Part 2A Question 1 Question 2 TOCNext >>Bookmarks - BM Assessment Header Part 2B Question 3 Question 4 TOCDone Bookmarks - BM << Prev

16. 76- Display Chunking If DisplayChunking = byAssessment, then full assessment is displayed in one web page Assessment Header TOCBookmarks Part 1 Question 1 Question 2 - BM Done Part 2 Question 1 Question 2 - BM Question 3 - BM Question4 - BM

17. 11- Multipart Allowed If TRUE, then in Assessment Editor: Assessment Header Part 1 Create New Part If FALSE, then in Assessment Editor: Assessment Header Part 1

18. Group Access: Full Class Sections Any Class Groups (Can set access variables for each group)

19. Assessment Access: When does it become available Immediately By Date Release Date By Action Completion of another assessment Completion of another assessment with minimum grade

20. Assessment Access: When is it unavailable? Immediately By Date Retract Date By Action Completion of this assessment

21. Assessment Access: Where/How is it available? By IP Address Allowed IP Address List Forbidden IP Address List By Special Password (for site-located assessments)

22. Assessment Access: Due Date At Date No late submissions accepted Late accepted and tagged No Due Date

23. Assessment Access: Number of Submissions Only One Any number set by instructor Unlimited (When allowed number of submissions is made then response check boxes, radio buttons, text boxes, etc. change to fixed displays)

24. Assessment Access: Special Controls for Instructor Retry-allows instructor to override access settings and to give a student one more submission

25. Assessment Access: Timed Assessments Assessment can have time limit from first download. Questions and Parts can also have time limits.

26. Feedback Access: Immediate (while taking assessment) Feedback and Score Feedback Release Dates Score Release Date Solution and Comment Release Date No Feedback or Scores

27. 14- Feedback Chunking (only valid if ReleaseModel = immediate) If FeedbackChunking = byitem, then when a response is entered for question 1 the display is automatically updated for the assessment display to show feedback and score information. Most common use will probably be for sections to be displayed one item at a time, but this is not necessary. Assessment Header Part 2 Question 1 Question 2 TOCNext >>Bookmarks Assessment Header Part 2 Question 1 Question 2 << PrevDoneTOC Bookmarks Q1 Feedback (score, feedback) - BM

28. 14- Feedback Chunking (only valid if ReleaseModel = immediate) If FeedbackChunking = bypart, then when the Next>> button is clicked, the same part is displayed with feedback. Assessment Header Part 2 Question 1 Question 2 TOCNext >>Bookmarks - BM Assessment Header Part 2 Question 1 Question 2 << PrevNext>>TOC Bookmarks Q1 Feedback (score, feedback) - BM Q2 Feedback (score, feedback) Assessment Header Part 3 Question 1 Question 2 TOCDone Bookmarks - BM << Prev

29. 14- Feedback Chunking (only valid if ReleaseModel = immediate) If FeedbackChunking = byassessment, then when the Done button is clicked, the same part is displayed with feedback. After this time, all nav buttons take testee to pages with feedback. Same view as after feedback display date. Assessment Header Part 2 Question 1 Question 2 << PrevNext>>TOC Bookmarks Q1 Feedback (score, feedback) - BM Q2 Feedback (score, feedback) Assessment Header Part 3 Question 1 Question 2 TOCDone Bookmarks - BM << Prev Assessment Header Part 3 Question 1 Question 2 << PrevDoneTOC Bookmarks Q1 Feedback (score, feedback) - BM Q2 Feedback (score, feedback) To TOC

30. Evaluation Types: Item scores Numeric (negative values accepted) ABCDEF/+- Plus-check-minus Completed/not completed Comments For each question For each part For full assessment

31. Evaluation by: Instructor Section leader Peer or peer group Testee Auto-Scoring

32. Testee Info Available to Evaluators Anonymous (false name) Submission Report identifiees submitters Identified Name SUNet ID Email Address

33. Score and Total Assessment Values: Total = Sum of all question scores With default question score Total Value set with weighted Parts having equal-valued questions Weighting by points Weighting by percent Total Value set with equal-valued questions in all Parts

34. Results Distribution: to section leader to group to instructor to testee to grade book to secty

35. Types of Results that can be distributed: Combined statistics (summary data) Anon Resp and Scores Testee’s personal score All testees’ responses and scores Analytic statistics

36. Objective Metadata Rubric Question Response Answer 1 Answer 2 Answer 3Feedback HintEval Comments Score %Val Max Value KEY Initial Item Information Added by Instructor, Testee, Auto-scoring during use Italicized box can contain multiple types of Multimedia Data Structure of an Item Name Type

37. Name Type Related Files & Links Inline Multimedia Description hwork Study quest quizexam Keywords Objective Part 2 Part 1 Assessment Attributes Role-Rubric Template Template Description Structure of an Assessment

38. Name Layout Type Related Files & Links Inline Multimedia Description Keywords Objective Item 2 Item 1 Part Attributes Structure of a Part