1. Assignment Marking via Online Self Assess Margot Schuhmacher, Lecturer Higher Education Development Unit, Centre for Learning and Teaching Support, Monash University Robert Redpath, Lecturer School of Computer Science and Software Engineering, Faculty of Information Technology, Monash University

2. Overview The Role of Assessment The Assessment Process Assessment Criteria as a Quiz Evaluation of the New Process by Students and Tutors Conclusions

3. Introduction Assessment for formal accreditation and student learning Student as primary stakeholder in the process Assignment submission via a learning management system

4. The Assessment Process The LMS replaces a traditional paper based process This provides benefits: –Flexible student submission –Persistent record BUT the student view of the marking process is still as a black box

5. Black Box View

6. The Tutor and an LMS Assignment is online Marking guide is separate The process is tedious Mark entry is separate step

7. Marking Guides Represented in many forms What if it is presented as a simple rubric?

8. Rubric Score for Essay Structure AmateurCritical & Essential DesirableExtension Descriptor Essay seems to have coherent structure Essay structure has an introduction body and conclusion All critical elements and a discussion All desirable elements and extended with analysis Points 1 2 3 4 Select X

9. Desired Improvements Can the technology incorporate marking criteria into the assignment?

10. Assignment tool in Vista Undergraduate database unit Students needed to implement a database Assessed on logical and physical design –Use of constraints –Valid syntax –Results of queries

11. Assessments Tool in Vista Quizzes question types include –Multiple choice (single and multiple selection) –True False –Short Answer –Open-ended (paragraph) Feedback and results can be returned to students

12. Combine two tools?

13. Marking Criteria Examples The constraints were: 1. Not explained 2. Had a poor explanation 3. Had a good explanation 4. Had a very good explanation Did you use sensible sample data that allowed the queries to produce output? 1. Yes 2. No

14. Further Examples Which of the following SQL Features did you use correctly and successfully in one or more of your queries? Please select all features used.  Compound/complex search conditions  LIKE search criteria  Date function (such as TO_CHAR, TO_DATE)  Join 2 tables  Join 3 or more tables  ORDER BY clause  GROUP BY with aggregate function  Sub-queries

15. A Demonstration Using WebCT – Vista (MUSO)

16. A White Box View The student sees the evaluation of their assignment specifically related to the marking criteria

17. The Evaluation Process Trialled in undergraduate database unit Anonymous surveys of students and tutors 76 assignments 46 self assessments 12 student evaluations 3 (out of 4) tutor evaluations

18. Student Issues Preference for the new system? Ease of the process? Value of the process? Indication of performance from the quiz? Variation in grade expectation when result known?

19. Greater Student Satisfaction Early and strong indications of assessment Improved student feedback at both times of self assessment and assessment confirmation A greater understanding of the criteria applied Improved learning

20. Tutor Issues Ease of marking in comparison to previous approaches? Ease of management? Ease of providing feedback? Clarity of marking guide in comparison to other units? Preference for the process? Need to override the student score? Direction of adjustments

21. Tutors Prefer the Approach Ensures a clear assessment criteria Moderation carried out by confirming or overriding student self assessment An imposed structure exists for persistent records easing further moderation

22. Findings The criteria must be described clearly –Improves consistency The student receives immediate feedback –Self assessment is moderated by the tutor Persistent and central record –Ease of moderation

23. Conclusions An improved assessment process Clearer criteria More immediate feedback Improved learning experience A persistent record of the entire process Greater efficiencies in the process

24. Future Work The process trialled again in 2005 Process improvements Process training/education required Investigate other forms of assessment criteria