1. Claus Brabrand Reykjavik UniversitySeptember 22, 2009 How to make sure your students learn what you want them to Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark Háskólinn í Reykjavík Reykjavik University

2. Claus Brabrand Reykjavik UniversitySeptember 22, 2009 An Introduction to the Theory & Application of Constructive Alignment Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark Háskólinn í Reykjavík Reykjavik University

3. [ 3 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 "What is good teaching?" Exercise: T

4. [ 4 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Alignment in Practice Concrete recommendations Alignment implementation process

5. [ 5 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 “Teaching for Quality Learning at University - What the student does” “Teaching for Quality Learning at University - What the student does” Constructive Alignment & SOLO Taxonomy: Introduction to… “Teaching Teaching & Understanding Understanding” “Teaching Teaching & Understanding Understanding” 19 min award-winning short-film on Constructive Alignment (available on DVD in 7 languages, epilogue by John Biggs) John Biggs’ popular and heavily cited book: Note: 3 rd Edition now available [J.Biggs & C.Tang, 2009]

6. [ 6 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 "What are the film's main messages (in your opinion)"? Neighbour Discussion: T

7. [ 7 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Alignment in Practice Concrete recommendations Alignment implementation process

8. [ 8 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 From Content to Competence My old course descriptions (Concurrency 2004): Given in terms of a 'content description'. Essentially: This is a bad idea for two reasons...! Goal is…: To understand: deadlock interference synchronization...

9. [ 9 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Goal is…: To understand: deadlock interference synchronization... Problem 1 ! Problem with 'content' as goals ! analyze... theorize... define deadlock describe solutions name solutions recite conditons Stud. C Stud. A Stud. B analyze systems explain causes Censor Teacher P.S.: even if it were possible to agree, we know that the exam will dictate the learning anyway.  agreement analyze systems explain causes tacit knowledge from a research-based tradition not known by student

10. [ 10 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Problem 2 ! Problem with 'understanding' as goals ! The answer is simple : 'concept of deadlock' ?!  Goal is…: To understand: deadlock interference synchronization... It cannot be measured !

11. [ 11 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Competence ! 'Competence' as goals ! Have the student do something; and then "measure" the product and/or process 'SOLO' = Structure of the Observed Learning Outcome Note': inherently operational (~ verbs) Objective ! To learn how to: analyze systems for... explain cause/effects... prove properties of... compare methods of...... Note: 'understanding' is of course pre-requisitional ! Competence := knowledge + capacity to act upon it

12. [ 12 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Neighbour Discussion "How does this content vs. competence relate to YOUR courses?" T

13. [ 13 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 SOLO Advantages Advantages of The SOLO Taxonomy: Linear hierarchical structure (good for progression) Aimed at evaluating student learning Converges on research (at SOLO 5) Research: Production of new knowledge

14. [ 14 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 SOLO (elaborated) Note: the list is non-exhaustive SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … analyze compare contrast integrate relate explain causes apply theory (to its domain) … combine structure describe classify enumerate list do algorithm apply method … define identify count name recite paraphrase follow (simple) instructions … Graphic Legend problem / question / cue known related issue - given! hypothetical related issue - not given! student response Q R QUANTITATIVEQUALITATIVE R R' Q R Q R Q R Q

15. [ 15 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 SOLO verbs Mapped by:  B. Dahl & C. Brabrand (Natural science context!) With help from:  3 Educational research colleagues (medicine)  J. Biggs & C. Tang

16. [ 16 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Exercise "Key competences in YOUR course?" T Concurrency: analyze systems compare models Concurrency: analyze systems compare models

17. [ 17 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Alignment in Practice Concrete recommendations Alignment implementation process

18. [ 18 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Concrete Recommendations Intended Learning Outcomes [Genetics 101] After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies, in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria). describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously 2) List sub-goals as 'bullets': Clearer than text 1) Use 'standard formulation': a) puts learning focus on the student b) competence formulation: "to be able to" 3) Use 'Verb + Noun' formulation: What the student is expected to do with a given matter. V N V V VV V V V V N N N N 4) Avoid 'understanding-goals': "To understand X", "Be familiar with Y", "Have a notion of Z" ! N

19. [ 19 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Implementation Process 1) Think carefully about: overall goal of course (what students learn to do?) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes 3) Choose carefully the form(s) of assessment (~ intended learning outcomes ) 4) Choose carefully the form(s) of teaching (~ intended learning outcomes ) alignment learning incentivelearning support Think of teaching activities as ”training for exam”

20. [ 20 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 My Starting Point Content description (Concurrency '04+'05): What is the overall goal of the course...? (i.e., what are the students to learn to do?)

21. [ 21 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Overall Course Philosophy Model-Based Design for Concurrency:

22. [ 22 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Implementation Process 1) Think carefully about: overall goal of course (what students learn to do?) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes 3) Choose carefully the form(s) of assessment (~ intended learning outcomes ) 4) Choose carefully the form(s) of teaching (~ intended learning outcomes ) alignment learning incentivelearning support Think of teaching activities as ”training for exam”

23. [ 23 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Intended Learning Outcomes Model-based design for Concurrency... SM I #1 #2 #3 Intended Learning Outcomes (based on The SOLO Taxonomy): Note: explicitly included as a non-goal 

24. [ 24 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Implementation Process 1) Think carefully about: overall goal of course (what students learn to do?) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes 3) Choose carefully the form(s) of assessment (~ intended learning outcomes ) 4) Choose carefully the form(s) of teaching (~ intended learning outcomes ) alignment learning incentivelearning support Think of teaching activities as ”training for exam”

25. [ 25 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 2004 Multiple-Choice assessment: with a bunch of seemingly reasonable questions: 2006 Assessment (~ ILO's): Project (carefully assessing): construct, apply, relate, implement,... Multiple-Choice test (carefully assessing): analyze and compare MC-test (pre- vs. post-alignment) Bad Alignment Good Alignment

26. [ 26 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Implementation Process 1) Think carefully about: overall goal of course (what students learn to do?) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes 3) Choose carefully the form(s) of assessment (~ intended learning outcomes ) 4) Choose carefully the form(s) of teaching (~ intended learning outcomes ) alignment learning incentivelearning support Think of teaching activities as ”training for exam”

27. [ 27 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Teaching/Learning Activities Teaching/Learning Activities (~ ILO's)!!! Also, generally about moving from: To: Teacher-centric activities Student-centric activities e.g., lectures, monologues,... 

28. [ 28 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Questions... "What is good teaching?" The Short-Film Cognitive processes Association new ~ old The Book John Biggs "understanding" content  competence Student activation Susan & Robert Teacher models levels 1 - 2 - 3 Course descriptions Constructive Alignment Top Competences 15% programming CS v. NAT v. MAT Students at University My research and teaching 'TLA' Teaching / Learning Activities Tips'n'Tricks ? recite generalize R R' Q The SOLO Taxonomy

29. [ 29 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Now, please: "2-minute recap" Please spend 2' on thinking about and writing down the most important points from the talk – now!: After 1 day After 1 week After 3 weeks After 2 weeks Immediately

30. [ 30 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Key References ”Teaching for Quality Learning at University” John Biggs & Catherine Tang Society for Research into Higher Education, 2007. McGraw-Hill. ”Evaluating the Quality of Learning: The SOLO Taxonomy” John Biggs & Kevin F. Collis London: Academic Press, 1982 ”Teaching Teaching & Understanding Understanding” Claus Brabrand & Jacob Andersen 19 minute award-winning short-film (DVD) Aarhus University Press, Aarhus University, 2006 "Constructive Alignment & The SOLO Taxonomy: a Comparative Study of University Competencies in Computer Science vs. Mathematics" Claus Brabrand & Bettina Dahl CRPIT, Vol. 88, ACS 3-17, R. Lister & Simon, Eds., 2007

31. Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Thank You! ((( http://www.daimi.au.dk/~brabrand/short-film/ )))http://www.daimi.au.dk/~brabrand/short-film/ Film's homepage:

32. [ 32 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Tips'n'Tricks (activation) Neighbour discussions: Frequent breaks: Post-It exercise:  focus: zoom in  anonymous (!)  swap'able  everyone will engage  empathetic control  shared knowledge pool pulse reader measurements:  more questions (students dare ask them)  better questions (students had a chance to discuss) 1-2 min timeout [Phil Race] Form variation: lecturing blended with in-class activation exercises

33. [ 33 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Tips'n'Tricks (cont'd) "Less-is-more": Use many examples: (build on student pre-knowledge) Explicit structure:  analyze  compare  relate common deadlock, uncommon deadlock, A-synchronization, B- synchronization, hand-shake, multi-party synchronization, multi-party hand-shake, binary semaphores, generalized semaphores, blocking semaphores, recursive locks,... vs. Emphasize depth over breadth (coverage) NEW OLD 1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4. wwwwwww 1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4. wwwwwww 1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4. wwwwwww 1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4. wwwwwww  self evident to you [ teacher ]  not to a learner [ student ] (esp. during learning process)  Student 'recap' at end: after 1 day after 1 week after 3 weeks after 2 weeks now

34. [ 34 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 E.g. course: ”Databases” (at RUC/Roskilde): Note: almost entirely non-operational(!) i.e. measure how?!  obtain knowledge about the structure of database systems;  be familiar with design of databases by use of special notations like E/R and analysis through normalization;  get an overview of the most important database models and a detailed knowledge about the most important model - the relational model as well as the language SQL;  get an overview of database indexing and query processing;  obtain knowledge about application programming for DB systems. Problematic Courses Familiar with ?!

35. [ 35 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Teacher’s intention Student’s activity Exam’s assessment  e.g. - explain - relate - prove - apply e.g. - memorize - describe U NALIGNED C OURSE e.g. - memorize - describe "Dealing with the test"

36. [ 36 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Teacher’s intention Student’s activity Exam’s assessment e.g. - explain - relate - prove - apply A LIGNED C OURSE e.g. - explain - relate - prove - apply e.g. - explain - relate - prove - apply e.g. - explain - relate - prove - apply e.g. - explain - relate - prove - apply

37. [ 37 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Student Motivation  Susan: (”intrinsic motivation”) - wants to…: learn !  Robert: (”extrinsic motivation”) - to…: pass exams !

38. [ 38 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Constructivism  ”Transmission is Dead…” : (lectures = )  Knowledge is… Actively Constructed ! active teacher & passive students ! risk

39. [ 39 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 SOLO Taxonomy  Hierarchy for Competences:  Deep learning (not surface) ! 5: generalize, theorize, predict, … 4: explain, analyze, compare, … 3: describe, combine, classify, … 2: recite, identify, calculate, …

40. [ 40 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Stud Learning Focus  Focus on Student Learning ! (instead of ”what teacher does” & labelling students: ’good/bad’)  Student activitation  learning

41. [ 41 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Alignment  Make explicit ILO’s (Intended Learning Outcomes):   (…and tell this to students) Exam = ILO’s = Teaching

42. [ 42 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Acquisition of Competence Acquisition of competence progresses according to the following stages of learning: 1) Unconscious incompetence 2) Conscious incompetence 3) Conscious competence 4) Unconscious competence 5) Capacity for moving consciously between stages 3) and 4): (which is required by a teacher)

43. [ 43 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 On the Role of Examination Alignment: A systemic theory (a teaching system w/ cause/effects) A theory of planning (over the course of a course) A theory of motivation (and incentive) From the exam as a...:...to: "Necessary evil" Motivational and learning-guiding pedagogical tool for the teacher(!) application of alignment "The exam does not come after, but before the course!"

44. [ 44 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 Definition: “Good Teaching” Definition: Good news: We now know how to do this: Alignment!!! Explicitly defined course objectives (as verbs)! Discourage surface-learning! Encourage depth-learning! “Less-is-more”: depth rather than breadth of coverage! ”Good teaching is getting most students to use the higher cognitive level processes that the more academic students use spontaneously” -- “Teaching for Quality Learning at University”, John Biggs, 2003

45. [ 45 ] Claus Brabrand Reykjavik UniversitySeptember 22, 2009 The BLOOM Taxonomy (1956) The BLOOM Taxonomy: Knowledge Comprehension Application AnalysisEvaluationSynthesis Qualitative Quantitative SOLO 4+5 SOLO 2+3 ”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item” -- (Biggs & Collis, 1982) ”