Consistency and Reliability in Rating Student’s Work

Challenges in rating student’s work Subjectivity Experience Personality

Consistency and Reliability The purpose of the procedure: Fairness Transparency Truthfulness Trust The integrity. The students’ future may rely on the marks you given. Fairness: to mark the quality of student’s work rather than the person. Truthfulness: true reflection of the student’s ability; Trust: link between the markers and the students. The marking criteria is available for bother markers and students – Transparency.

The process of rating students’ work Step 1: Validation Step 2: Pre-moderation Step 3: Moderation Step 4: Post-moderation Step 5: Audit There are five steps in this rating process.

Step 1: Validation Assessment Writer(s): Subject teachers Providing guidelines for answering the question Assessment Assessor(s): Check complexity and difficulty of the assessment tasks Rating starts from here. This is the preparation for the rating. The purpose of the step is to make sure the validity of the assessment tasks. The teachers agree to be the writers. This means that the students are tested for what they have learned and taught. The assessment tasks match the learning outcomes of the course. In the writing process, the assessment writers discuss and select the questions and provide a guideline for the question. Then, the validation form will be sent to the assessment assessors. The assessors will check the complexity and difficulty (clear, explicity, too hard or easy) of the assessment task.

Step 2: Pre-moderation Random selection of student’s work Understanding the assessment task Marking rubrics Individual rating Group discussion to get a group mark The general discussion to get a final mark The subject coordinator selects the student’s work randomly before the rating. Because we have students from different nationalities, we do pay attention to selecting students from different background. The markers have 10 to 15 minutes to read through the assessment tasks and requirements. After that, the markers discuss the assessment task and anticipate the possible answers and what should or should not be accepted. Then it is the individual marking. Objectivity! The work will be marked blinded, which means the student’s details are not to be revealed, to be objective. When the individual rating finishes, the markers discuss their rating in groups to get a group mark. Next, t he group marks are discussed to get a final mark.

Step 3: Moderation Marking individually According to the marking criteria Discussion Enter the results and inform the Subject Coordinator This is when the teachers start marking their the work of their own marking. During this phase, the teachers still discuss with each other if there is work that is unsure or different. When the rating finished, the results are to be entered on SMP and inform the Subject Coordinator.

Step 4: Post-moderation Cross-marking: Comparing the average results among all the tutorial groups Check the marks asking for a second opinion Final marks released Remark requests from students In the post-moderation, the Subject Coordinator compares the average results of each tutorial. If there is a tutorial group is too high or low, or there is a mark that is a bit too high or low, the SC informs the teacher to ask for a second opinion. In doing so, the teacher will select student work with highest, middle and lowest marks. Normally, the difference among the tutorial groups is about 5 marks or so. It is important to realise that we are not to keep the marks low or high. If there is a work with a very high mark, D or HD, or too low, below 35, the marker will be asked to check to mark for reassurance. There could be re-scaling up or low of the results. After the results are issued to the students, the students still may apply for remark within two working days if they are not happy with the results.

Audit Time of the audit Process of the audit Audit report Discussion of the audit report Feedback of the audit When all the moderations is finished, we also have the Audit. This will be conducted yearly. The Course Coordinator will be in charge of the audit. First, an auditor will be selected. This auditor should be someone who understands the course, but not involved in teaching this particular session. What the auditor will be is first to attend the pre-moderation meeting in order to know the requirements of the moderation. Then, the Course Coordinators will select the highest, middle and lowest marks and pass them on the auditor. The auditor will write an audit report in terms of question, marks given and make recommendations. The report will be given to the Academic Quality Coordinator who will pass the report to the course coordinator. In the end, the subject coordinator and teachers will get the feedback about the audit.

Summary of the Procedure Rigorous Fair Trustworthy Transparent Rick The whole process is very tight. Within a very limited time, all the tasks have to be done. We have to make sure that rating is based on the fairness and transparency so that the students trust us for the marks we have rated.

Any Questions?