Outcomes-based Teaching and Learning Pre-workshop 3 Designing Assessment Tasks and Grading Criteria

Intended Outcomes of Pre-workshop Exercises for members of each group 1. Review existing assessment tasks (ATs) in relation to alignment with subject ILOs. 2. Identify areas requiring changes. 3. Revise or re-design ATs to align to subject ILOs. 4. Discuss and share your ideas with your group members.

Before the Workshop Refer to the Pre-workshop reference provided and the revised subject ILOs in Pre-workshop Exercise 1, conduct Pre-workshop Exercises 4, 5 & 6 to: a. design assessment tasks to address the revised subject IlOs. Consider if the ATs require the students to engege in the ILO verbs? Do the weightings reflect the relative importance of the ILOs? b. design grading criteria for either the ILOs or the assessment tasks. Discuss with your group members when doing the above exercises. Complete the exercises to be ready for discussion and sharing in the workshops on 2 December 2006.

Outcomes-based Teaching and Learning (OBTL) using Constructive Alignment Engaging the student in the verb in the ILO ILO: What the student has to learn Assessment: How well the student has met the ILO

Designing Assessment Tasks (ATs) Steps: 1. Selecting a practicable task that embodies the target ILO verb. 2. Making a judgment on how well the ILO has been met by the students' performance on the ATs – developing grading criteria.

Assessment Tasks (ATs) . Provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what level their performance is in those ILOs. . Provide the evidence allowing teachers to make a judgment about the level of a student’s performance against the ILOs and to award a final grade. . Should be appropriately designed or selected to address the ILOs that we want to assess. . Different assessment tasks address different ILOs. There would therefore be different kinds of assessment task.

Selection of Assessment Tasks A range of different assessment tasks may be required to address the range of ILOs of a subject. Questions to be asked in selecting assessment tasks: 1. Are the assessment tasks able to address all ILOs? 2. Are the assessment tasks aligned to their appropriate ILOs? Are the students required to engage in the verbs identified in the ILOs? 3. Are the assessment tasks practicable with respect to available time and resources? 4. Do the assessment tasks reflect the relative importance of the subject ILOs? 5. Is the assessment workload realistic for teachers and students?

Constructive Alignment of ILOs and Assessment Tasks Subject ILO 5 Subject ILO 4 Subject ILO 3 Subject ILO 2 Subject ILO 1 AT 3 AT 2 AT 1 Assessment Tasks Subject ILOs Consider if : all ILOs are being addressed? there is a balanced coverage of the ILOs? the more important ILOs are given appropriate assessment emphasis.

Common ILOs Possible Assessment Tasks Describe assignment, essay question exam Explain assignment, essay question exam, oral, letter-to-a-friend Integrate project, assignment Analyse case study, assignment Apply project, case study, experiment Solve problem case study, project, experiment Design, create project, experiment Reflect reflective diary, portfolio, self-assessment Communicate a range of oral, writing or listening tasks addressing the ILOs, e.g. presentation, debate, role play, reporting, assignment, precis, paraphasing, answering questions etc.

Assessment tasks for the sample subject ILOs 1. Explain different communicative strategies. AT: Write an assignment or answer exam questions explaining the four most important strategies used in report writing. 2. Apply appropriate strategies in different modes of communication. AT: Write a report on a tutorial case study discussion on a selected topic. 3. Reflect and improve own communicative strategies. AT: Self-assessment of the report, identifying strengths and areas needing improvement. Revise the report based on self-assessment. The three ILOs could be assessed through one assessment task – writing a report on a tutorial case study discussion followed by a self-assessment and revised report. a. Write a report on a tutorial group discussion; b. Explain the strategies used in writing the report; c. Self-assess the report, identifying strengths and strategies to be improved. d. Revise the report based on c. and submit the revised report together with the first version.

Examination is a very commonly used assessment task especially for large classes. We need to consider if 1. examinations involving answering essay type of questions under invigilated conditions is able to assess students' performance in some high level ILOs, e.g. apply, reflect, create etc. 2. there are other alternative assessments tasks which will more appropriately addess those high level ILOs. If we must have an invigilated “final exam”, there are far better formats than the conventional written essays; for example gobbets, critical incidents (e.g. comment on a video segement), letter-to-a-friend, and so on. If we must have exams, consider if the weighting (usually 50% or more) is appropriate to reflect the relative importance of the ILOs being addressed.

Exercise 4 - Designing Assessment Tasks Consider the intended learning outcomes of your subject in Exercise 1, suggest appropriate assessment task(s) to address these ILOs. For each of the tasks, explain what the students are expected to do in completing the task. Are the students required to enact the target ILO verbs? Discuss with your group members. Subject ILOs Assessment Tasks Student activities in completing the task

After deciding on the assessment tasks, we have to After deciding on the assessment tasks, we have to consider the following issues. 1. Specification of the tasks Especially when continuous assessment is used, the different assessment tasks making up the continuous assessment will need to specified. 2. Assessment weightings If assessment tasks are given different assessment weightings, ensure that these weightings relfect the relative importance of the ILOs. Take the four sample subject ILOs – explain, apply reflect and improve. Are they of equal importance? If not, then the more important ILOs should be given heavier assessment weighting.

Exercise 5 – Assessment Weighting Go back to the ATs designed in Exercise 4, now consider assessment weightings to ensure that they reflect the relative importance of the ILOs. Discuss with your group members. Subject ILOs Assessment Tasks Student activities in Wt completing the task Does the weighting of the ATs reflect the relative importance of the ILOs they address?

Grading Criteria Student performance in assessment tasks should be judged against a set of clear grading criteria defining the quality of performance expected of each of the grades.

Grading ILOs Direct Indirect (Grading the ILOs) (Grading the assessment tasks which are aligned with the ILOs) Grading criteria (rubrics) Grading criteria (rubrics) of individual ILOs of individual assessment tasks Derive final grade Derive final grade

Example of Direct Grading Criteria of ILOs Marginal Pass Satisfactory Good Excellent D C- C C+ B- B B+ A- A Grade 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 point/unit ILOs Explain Able to identify and briefly Able to identify a number Able to identify a full As in “Good” but write about limited points. relevant points with some range of relevant provides views on Very little evidence of details. Uses these points Points with details. possible alternative using these points to to provide a fair reasoning Supported by relevant causes and/or results provide reasoning to or causality. No evidence literature. Points are under changing why they are inter- of a comprehensive organized to provide a conditions. Able to related. overview of reasoning comprehensive and link current or causality. cohesive reasoning or reasoning to causality. situations in real- life professional contexts. Reflect Able to use available Able to use available Able to use available As in “Good”. Able information to self- information to self- information to self- to generalize self- evaluate and identify evaluate and identify evaluate and identify evaluation to beyond limited aspects of own more aspects of own the full range of owm existing context. strengths and weaknesses strengths and weaknesses strengthes and weak- Suggest ways of in a general sense. No in a general sense. Little nesses. Self-evaluation improving perform- evidence of suggestions application of theory in is based on theory. ance to real-life of ways to improve self-evaluation and limited Increasingly able to professional performance. No evidence suggestions of ways to suggest ways to contest. of theory being used in improve performance. improve performance self-evaluation. in a specific context.

Example of Indirect Grading Criteria of Assessment Tasks in a Portfolio Marginal Pass Satisfactory Good Excellent D C- C C+ B- B B+ A- A 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 The pieces of evidence The evidence is relevant, The evidence presents a As in “B” but with are relevant and accurate and covers good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality and isolated, addressing subject. Little evidence of subject. Good coverage evidence of inter- one aspect of the an overall view of the with relevant and nalization into subject. Demonstration subject. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of aspects of the subject evidence of reflect- way. Poor coverage, no content. Able to discuss integrate to form a ion on own originality, weak content meaningfully. thrust or purpose. performance based justification of portfolio Good coverage but little Good evidence of on theory. items. Inappropriate Application or integration. application of subject Generalizes subject self-evaluation. Fair justification of items. Content to practice. content to new and Attempted realistic self- Portfolio items well unfamiliar real- evaluation justified. Realistic life contexts. self-evaluation.

Departments need to decide whether to use Departments need to decide whether to use direct or indirect grading of the ILOs.

A holistic way of deriving a final grade from multiple grades ILOs AT1 AT2 AT3 AT4 Grade 1. Relate √ √ A – if all ILOs well achieved but 1 & 2 2. Apply √ √ excellent 3. Identify √ B – if all are well achieved 4. Explain √ C – if 3, 4 & 5 are well achieved 5. Communicate √ D – if marginal or only 2 ILOs are achieved

An example of a holistic way of deriving final grade Curriculum and Instruction: A subject in a course for Ed. Psychlsts. Grading will be based on your attaining the following ILOs 1. Apply the principles of good teaching and assessment to chosen contexts. 2. Relate selected aspects of curriculum design and management to the educational system in Hong Kong. 3. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist. 4. Show examples of your reflective decision-making as an educational psychologist. Final grades will depend on how well you can demonstrate that you have met all the ILOs: A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice. B Awarded when all ILOs have been met very well and effectively. C Awarded when the ILOs have been addressed satisfactorily, or where the evidence is strong in some ILOs, weaker but acceptable in others. F Less than C, work plagiarised, not submitted.

Example of Holistic Grading of Assessment Tasks in a Portfolio Marginal Adequate Good Excellent D C- C C+ B- B B+ A- A 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 The pieces of evidence The evidence is relevant, The evidence presents a As in “B” but with are relevant and accurate and covers good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality and isolated, addressing course. Little evidence of course. Good coverage evidence of inter- one aspect of the an overall view of the with relevant and nalization into course. Demonstration course. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of aspects of the course evidence of reflect- way. Poor coverage, no content. Able to discuss integrate to form a ion on own originality, weak content meaningfully. Thrust or purpose. Performance based justification of portfolio Good coverage but little Good evidence of on theory. items. Application or integration. application of course Generalizes course Fair justification of items. Content to practice. content to new and Portfolio items well Unfamiliar real- justified. life contexts.

Exercise 6 – Developing Grading Criteria Consider the intended learning outcomes and assessment tasks you have identified in Exercise 4, suggest appropriate grading criteria addressing either (a) the ILOs or (b) the assessment tasks. Discuss with your group members. Subject ILO and AT: Grade Grading criteria

Help our students understand the grading criteria . Remind students of the ILOs to be assessed. . Explain to students the standards expected of each grade. . Illustrate with previous assessment tasks on How the grading criteria have been applied. How grades have been awarded. How a better grade could have been achieved. . Clarify any problems that students may have.

After you have completed the pre-workshop. exercises,. 1 After you have completed the pre-workshop exercises, 1. select one group member to share your ideas at the workshop, and 2. jot down and issues that you would like to bring up for discussion at the workshop.