1. ITU Seminar on Teaching & Learning Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark The Danish Grade Scale and Grading with ILO's ( 14:50 – 15:45 )

2. The Danish Grade Scale and Grading with Intended Learning Outcomes Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark The Danish Grade Scale and Grading with ILO's ( 14:50 – 15:45 )

3. [ 3 ] Intended Learning Outcomes From content:...to competence: Intended Learning Outcomes (using verbs from The SOLO Taxonomy) ILO's: analyze... explain... compare... apply... Goal: understand: deadlock interference synchronization...  analyze explain Teacher Censor Student identify recite Understanding?!? Have the student do something; and then "observe" product and/or process vs. 

4. [ 4 ]  Make explicit ILO's ( )   (…and tell this to students) = ILO's = Teaching Learning Activities Assessment Constructive Alignment vs. ROBERT: extrinsically motivated ROBERT: extrinsically motivated SUSAN: intrinsically motivated SUSAN: intrinsically motivated Intended Learning Outcomes

5. [ 5 ] The Implementation Process 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2) Operationalize these goals: and express them as (SOLO) intended learning outcomes 3) Carefully design (TLA's): teaching/learning actitivites (  intended learning outcomes) 4) Carefully design (AT's): assessment tasks (  intended learning outcomes) alignment learning supportlearning incentive "What and how students learn depends to a major extent on how they think they will be assessed" (Biggs & Tang, 2007)

6. [ 6 ] The Danish Grade Scale ECTS SCALE A, B, C, D, E, Fx, F... 4 steps 8 steps 10 steps 21 steps... 4 steps 8 steps 10 steps 21 steps 7 steps:... Conversion (between EU countries): Motivations (for new grade scale): Comparability (DK vs. other countries)  mobility in EU Uniform usage (from primary school to university) "Wear and tear" (inflation, diff. subject conventions) Wanted: clearer steps between grades Wanted: grade = degree of fulfillment of objectives

7. [ 7 ] Key Legal Points Adoption of new grade scale in all of DK (Sep 2007) §1-8 Only absolute grading allowed (no relative grading) §9 Grade = degree of fulfillment of objectives §10 All courses must have explicit course objectives §10, §19

8. [ 8 ] The Danish Grade Scale Unacceptable -3 For a performance which is unacceptable in all respects F Inadequate 00 For an insufficient performance which does not meet the course objectives Fx Adequate 02 For a sufficient performance which barely meets the course objectives E Fair 4 For a fair performance which adequately meets the course objectives but also displays several major weaknesses D Good 7 For a good performance which meets the course objectives but also displays some weaknesses C Very good 10 For a very good performance which meets the course objectives, with only minor weaknesses B Excellent 12 For an excellent performance which completely meets the course objectives, with no or only a few insignificant weaknesses A Grade := Degree of fulfillment of course objectives!

9. [ 9 ] The Danish Grade Scale -- ++ Passed A: 10% B: 25% C: 30% D: 25% E: 10% E: 10% D: 25% C: 30% B: 25% A: 10% Not passed 0 (lowest) 100 (highest) degree of fulfillment of course objectives 22,5 (interval mid) 50 (interval mid) 77,5 (interval mid) /10 0 10 2 5 8 -2-5 symmetric transposition of lowest passed grades 02 12 4 7 10 00-3 avoid “psychological effect” with negative grades ECTS +2

10. [ 10 ] Grading Formula No universal ”grading formula”: Grading is (of course) a very course-specific process: Depends on explicit learning objectives!

11. [ 11 ] Assessing by Marks For: People are used to it It is logistically easy Against: Quality = accumulating small quantities Sends undesirable messages to students (cf. the "backwash effect") Also, measurement errors accumulate no valid difference between 74 and 75, yet to the student it can make a big difference (pass/fail) ILO's: analyze... explain... compare... apply... ILO's: analyze... explain... compare... apply... Higher-level assessment [John Biggs]

12. [ 12 ] Intended Learning Outcomes: analyze... explain... compare... apply... Grading using ILO's §10: Grade = degree of fulfillment of objectives (overall evaluation) §9: Only absolute grading allowed (no relative grading) Grading: 12 = ‘Completely meets objectives’ with no or only a few insignificant weaknesses 10 = ‘Meets course objectives’ with only minor weaknesses 7 = ‘Meets course objectives’ but also displays some weaknesses etc....from a researcher’s perspective (as in research-based teaching)

13. [ 13 ] Group Exercise 1) Pick a course (e.g., your favorite) 2) Write down the ILO's (or see 'course base') 3) Design assessment task(s): Use what you have heard today as inspiration; in particular: "Different Types of Examination" (Hanne Leth Andersen) [ For simplicity, let's disregard resource constraints ] Q) What are the main difficulties (challenges)? 2-4 people per group AT's = ILO's

14. [ 14 ] SOLO Taxonomy SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … analyze compare contrast integrate relate explain causes apply theory (to its domain) … combine structure describe classify enumerate list do algorithm apply method … define identify count name recite paraphrase follow (simple) instructions … Graphic Legend problem / question / cue known related issue - given! hypothetical related issue - not given! student response Q R QUANTITATIVEQUALITATIVE R R' Q R Q R2R2 R3R3 R1R1 Q R Q R Q

15. [ 15 ] Two Types of Assessment Formative: During course Feedback  learning NB: revealing errors may improve learning ! (student strategic response) Summative: End of course (typically) Score  final grade NB: concealing errors may improve grades ! (student strategic response)

16. [ 16 ] Thank you ! Questions?