1. ITU Seminar on Teaching & Learning Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark ( 09:00 – 10:00 ) ITU Teaching & Learning Seminar: Assessment and Exams

2. Constructive Alignment and Intended Learning Outcomes Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching & Learning Seminar: Assessment and Exams ( 09:00 – 10:00 ) Introduction to:

3. [ 3 ] "What is good teaching?" Exercise: T

4. [ 4 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

5. [ 5 ] “Teaching for Quality Learning at University - What the student does” “Teaching for Quality Learning at University - What the student does” Constructive Alignment & SOLO Taxonomy: Introduction to…: “Teaching Teaching & Understanding Understanding” “Teaching Teaching & Understanding Understanding” 19 min award-winning short-film on Constructive Alignment (available on DVD in 7 languages, epilogue by John Biggs) John Biggs’ popular and heavily cited book: Note: 3 rd Edition now available [J.Biggs & C.Tang, 2009]

6. [ 6 ] What are the film's main messages (in your opinion) ? Neighbour Discussion: T

7. [ 7 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

8. [ 8 ] From Content to Competence My old course descriptions (Concurrency 2004): Given in terms of a 'content description': Essentially: This is a bad idea for two reasons...! Goal is…: To understand: deadlock interference synchronization...

9. [ 9 ] Goal is…: To understand: deadlock interference synchronization... Problem 1 ! Problem with 'content' as goals ! analyse... theorize... define deadlock describe solutions name solutions recite conditons Stud. C Stud. A Stud. B analyze systems explain causes Censor Teacher  agreement analyze systems explain causes tacit knowledge from a research-based tradition not known by student

10. [ 10 ] Problem 2 ! Problem with 'understanding' as goals ! The answer is simple : 'concept of deadlock' ?!  Goal is…: To understand: deadlock interference synchronization... It cannot be observed !

11. [ 11 ] Competence ! 'Competence' as goals ! Have the student do something; and then "observe" the product and/or process 'SOLO' = Structure of the Observed Learning Outcome Note': inherently operational (~ verbs) Objective ! To learn how to: analyze systems for... explain cause/effects... prove properties of... compare methods of...... Note: 'understanding' is of course pre-requisitional ! Competence := knowledge + capacity to act upon it

12. [ 12 ] SOLO Advantages Advantages of The SOLO Taxonomy: Linear hierarchical structure Aimed at evaluating student learning Converges on research (at SOLO 5) Research: Production of new knowledge

13. [ 13 ] SOLO (elaborated) Note: the list is non-exhaustive SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … analyze compare contrast integrate relate explain causes apply theory (to its domain) … combine structure describe classify enumerate list do algorithm apply method … define identify count name recite paraphrase follow (simple) instructions … Graphic Legend problem / question / cue known related issue - given! hypothetical related issue - not given! student response Q R QUANTITATIVE QUALITATIVE R R' Q R Q R Q R Q QUALITATIVE

14. [ 14 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

15. [ 15 ] Concrete Recommendations (4x) Intended Learning Outcomes [Genetics 101] After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies, in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria). describe and perform connexion-analysis describe fundamental genetic concepts : (e.g., mutation variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously 2) List ILO's as 'bullets': Clearer than text Student "learning checklist" Use to design Teaching/Learning Activities (TLA's) Use to design Assessment Tasks (AT's) 1) Use 'standard formulation': a) puts learning focus on the student b) competence formulation: "to be able to" 3) Use 'Verb + Noun' formulation: What the student is expected to do with a given matter. V N V V VV V V V V N N N N 4) Avoid 'understanding-goals': "To understand X", "Be familiar with Y", "Have a notion of Z",...! N N

16. [ 16 ] The Danish Grade Scale Unacceptable -3 For a performance which is unacceptable in all respects F Inadequate 00 For an insufficient performance which does not meet the course objectives Fx Adequate 02 For a sufficient performance which barely meets the course objectives E Fair 4 For a fair performance which adequately meets the course objectives but also displays several major weaknesses D Good 7 For a good performance which meets the course objectives but also displays some weaknesses C Very good 10 For a very good performance which meets the course objectives, with only minor weaknesses B Excellent 12 For an excellent performance which completely meets the course objectives, with no or only a few insignificant weaknesses A Grade := Degree of fulfillment of course objectives!

17. [ 17 ] Activation Exercise What are key ILO's in YOUR course ? T Concurrency: analyze systems compare models Concurrency: analyze systems compare models

18. [ 18 ] The Implementation Process 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2) Operationalize these goals: and express them as (SOLO) intended learning outcomes 3) Carefully design (TLA's): teaching/learning actitivites (  intended learning outcomes) 4) Carefully design (AT's): assessment tasks (  intended learning outcomes) alignment learning supportlearning incentive More information on implementing alignment: [ http://www1.itu.dk/sw79909.asp ]

19. [ 19 ] Questions... "What is good teaching?" The Short-Film Cognitive processes Association new ~ old The Book John Biggs "understanding" content  competence Student activation Susan & Robert Teacher models levels 1 - 2 - 3 Course descriptions Constructive Alignment Top Competences 15% programming CS v. NAT v. MAT Students at University My research and teaching 'TLA' Teaching / Learning Activities Tips'n'Tricks ? recite generalize R R' Q The SOLO Taxonomy

20. [ 20 ] Key References ”Teaching for Quality Learning at University” John Biggs & Catherine Tang Society for Research into Higher Education, 2007. McGraw-Hill. ”Evaluating the Quality of Learning: The SOLO Taxonomy” John Biggs & Kevin F. Collis London: Academic Press, 1982 ”Teaching Teaching & Understanding Understanding” Claus Brabrand & Jacob Andersen 19 minute award-winning short-film (DVD) Aarhus University Press, Aarhus University, 2006 "Constructive Alignment & The SOLO Taxonomy: a Comparative Study of University Competencies in Computer Science vs. Mathematics" Claus Brabrand & Bettina Dahl CRPIT, Vol. 88, ACS 3-17, R. Lister & Simon, Eds., 2007

21. [ 21 ] Common SOLO Competences From a study of 632x courses at NAT/(AU+SDU) ie. science context

22. [ 22 ] The BLOOM Taxonomy (1956) The BLOOM Taxonomy: Knowledge Comprehension Application AnalysisEvaluationSynthesis Qualitative Quantitative SOLO 4+5 SOLO 2+3 ”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item” -- (Biggs & Collis, 1982) ”