ITU Seminar on Teaching & Learning Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark The Danish Grade Scale and Grading with ILO's ( 14:50 – 15:45 )

The Danish Grade Scale and Grading with Intended Learning Outcomes Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark The Danish Grade Scale and Grading with ILO's ( 14:50 – 15:45 )

[ 3 ] Intended Learning Outcomes From content:...to competence: Intended Learning Outcomes (using verbs from The SOLO Taxonomy) ILO's: analyze... explain... compare... apply... Goal: understand: deadlock interference synchronization...  analyze explain Teacher Censor Student identify recite Understanding?!? Have the student do something; and then "observe" product and/or process vs. 

[ 4 ]  Make explicit ILO's ( )   (…and tell this to students) = ILO's = Teaching Learning Activities Assessment Constructive Alignment vs. ROBERT: extrinsically motivated ROBERT: extrinsically motivated SUSAN: intrinsically motivated SUSAN: intrinsically motivated Intended Learning Outcomes

[ 5 ] The Implementation Process 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2) Operationalize these goals: and express them as (SOLO) intended learning outcomes 3) Carefully design (TLA's): teaching/learning actitivites (  intended learning outcomes) 4) Carefully design (AT's): assessment tasks (  intended learning outcomes) alignment learning supportlearning incentive "What and how students learn depends to a major extent on how they think they will be assessed" (Biggs & Tang, 2007)

[ 6 ] The Danish Grade Scale ECTS SCALE A, B, C, D, E, Fx, F... 4 steps 8 steps 10 steps 21 steps... 4 steps 8 steps 10 steps 21 steps 7 steps:... Conversion (between EU countries): Motivations (for new grade scale): Comparability (DK vs. other countries)  mobility in EU Uniform usage (from primary school to university) "Wear and tear" (inflation, diff. subject conventions) Wanted: clearer steps between grades Wanted: grade = degree of fulfillment of objectives

[ 7 ] Key Legal Points Adoption of new grade scale in all of DK (Sep 2007) §1-8 Only absolute grading allowed (no relative grading) §9 Grade = degree of fulfillment of objectives §10 All courses must have explicit course objectives §10, §19

[ 8 ] The Danish Grade Scale Unacceptable -3 For a performance which is unacceptable in all respects F Inadequate 00 For an insufficient performance which does not meet the course objectives Fx Adequate 02 For a sufficient performance which barely meets the course objectives E Fair 4 For a fair performance which adequately meets the course objectives but also displays several major weaknesses D Good 7 For a good performance which meets the course objectives but also displays some weaknesses C Very good 10 For a very good performance which meets the course objectives, with only minor weaknesses B Excellent 12 For an excellent performance which completely meets the course objectives, with no or only a few insignificant weaknesses A Grade := Degree of fulfillment of course objectives!

[ 9 ] The Danish Grade Scale Passed A: 10% B: 25% C: 30% D: 25% E: 10% E: 10% D: 25% C: 30% B: 25% A: 10% Not passed 0 (lowest) 100 (highest) degree of fulfillment of course objectives 22,5 (interval mid) 50 (interval mid) 77,5 (interval mid) / symmetric transposition of lowest passed grades avoid “psychological effect” with negative grades ECTS +2

[ 10 ] Grading Formula No universal ”grading formula”: Grading is (of course) a very course-specific process: Depends on explicit learning objectives!

[ 11 ] Assessing by Marks For: People are used to it It is logistically easy Against: Quality = accumulating small quantities Sends undesirable messages to students (cf. the "backwash effect") Also, measurement errors accumulate no valid difference between 74 and 75, yet to the student it can make a big difference (pass/fail) ILO's: analyze... explain... compare... apply... ILO's: analyze... explain... compare... apply... Higher-level assessment [John Biggs]

[ 12 ] Intended Learning Outcomes: analyze... explain... compare... apply... Grading using ILO's §10: Grade = degree of fulfillment of objectives (overall evaluation) §9: Only absolute grading allowed (no relative grading) Grading: 12 = ‘Completely meets objectives’ with no or only a few insignificant weaknesses 10 = ‘Meets course objectives’ with only minor weaknesses 7 = ‘Meets course objectives’ but also displays some weaknesses etc....from a researcher’s perspective (as in research-based teaching)

[ 13 ] Group Exercise 1) Pick a course (e.g., your favorite) 2) Write down the ILO's (or see 'course base') 3) Design assessment task(s): Use what you have heard today as inspiration; in particular: "Different Types of Examination" (Hanne Leth Andersen) [ For simplicity, let's disregard resource constraints ] Q) What are the main difficulties (challenges)? 2-4 people per group AT's = ILO's

[ 14 ] SOLO Taxonomy SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … analyze compare contrast integrate relate explain causes apply theory (to its domain) … combine structure describe classify enumerate list do algorithm apply method … define identify count name recite paraphrase follow (simple) instructions … Graphic Legend problem / question / cue known related issue - given! hypothetical related issue - not given! student response Q R QUANTITATIVEQUALITATIVE R R' Q R Q R2R2 R3R3 R1R1 Q R Q R Q

[ 15 ] Two Types of Assessment Formative: During course Feedback  learning NB: revealing errors may improve learning ! (student strategic response) Summative: End of course (typically) Score  final grade NB: concealing errors may improve grades ! (student strategic response)

[ 16 ] Thank you ! Questions?