ITU Seminar on Teaching & Learning Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching & Learning Seminar: Teaching/Learning Activities ( 09:00 – 10:00 )

Constructive Alignment and Intended Learning Outcomes Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching & Learning Seminar: Teaching/Learning Activities ( 09:00 – 10:00 ) Introduction to:

[ 3 ] "What is good teaching?" Exercise: T

[ 4 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

[ 5 ] “Teaching for Quality Learning at University - What the student does” “Teaching for Quality Learning at University - What the student does” Constructive Alignment & SOLO Taxonomy: Introduction to…: “Teaching Teaching & Understanding Understanding” “Teaching Teaching & Understanding Understanding” 19 min award-winning short-film on Constructive Alignment (available on DVD in 7 languages, epilogue by John Biggs) John Biggs’ popular and heavily cited book: Note: 3 rd Edition now available [J.Biggs & C.Tang, 2009]

[ 6 ] "What are the film's main messages (in your opinion)"? Neighbour Discussion: T

[ 7 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

[ 8 ] From Content to Competence My old course descriptions (Concurrency 2004): Given in terms of a 'content description': Essentially: This is a bad idea for two reasons...! Goal is…: To understand: deadlock interference synchronization...

[ 9 ] Goal is…: To understand: deadlock interference synchronization... Problem 1 ! Problem with 'content' as goals ! analyse... compare... define deadlock describe solutions name solutions recite conditons Stud. C Stud. A Stud. B analyze systems explain causes Censor Teacher  agreement analyze systems explain causes tacit knowledge from a research-based tradition not known by student

[ 10 ] Problem 2 ! Problem with 'understanding' as goals ! The answer is simple : 'concept of deadlock' ?!  Goal is…: To understand: deadlock interference synchronization... It cannot be observed !

[ 11 ] Competence ! 'Competence' as goals ! Have the student do something; and then "observe" the product and/or process 'SOLO' = Structure of the Observed Learning Outcome Note': inherently operational (~ verbs) Objective ! To learn how to: analyze systems for... explain cause/effects... prove properties of... compare methods of Note: 'understanding' is of course pre-requisitional ! Competence := knowledge + capacity to act upon it

[ 12 ] SOLO Advantages Advantages of The SOLO Taxonomy: Linear hierarchical structure Aimed at evaluating student learning Converges on research (at SOLO 5) Research: Production of new knowledge

[ 13 ] SOLO (elaborated) Note: the list is non-exhaustive SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … analyze compare contrast integrate relate explain causes apply theory (to its domain) … combine structure describe classify enumerate list do algorithm apply method … define identify count name recite paraphrase follow (simple) instructions … Graphic Legend problem / question / cue known related issue - given! hypothetical related issue - not given! student response Q R QUANTITATIVE QUALITATIVE R R' Q R Q R Q R Q QUALITATIVE

[ 14 ] Outline 1) Introduction Constructive Alignment The SOLO Taxonomy 2) From Content to Competence Advocate a shift in perspective Elaborate The SOLO Taxonomy 3) Intended Learning Outcomes How to use Intended Learning Outcomes Relation to new Danish Grading Scale

[ 15 ] Concrete Recommendations (4x) Intended Learning Outcomes [Genetics 101] After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies, in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria). describe and perform connexion-analysis describe fundamental genetic concepts : (e.g., mutation variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously 2) List ILO's as 'bullets': Clearer than text Student "learning checklist" Use to design Teaching/Learning Activities (TLA's) Use to design Assessment Tasks (AT's) 1) Use 'standard formulation': a) puts learning focus on the student b) competence formulation: "to be able to" 3) Use 'Verb + Noun' formulation: What the student is expected to do with a given matter. V N V V VV V V V V N N N N 4) Avoid 'understanding-goals': "To understand X", "Be familiar with Y", "Have a notion of Z",...! N N

[ 16 ] The Danish Grade Scale Unacceptable -3 For a performance which is unacceptable in all respects F Inadequate 00 For an insufficient performance which does not meet the course objectives Fx Adequate 02 For a sufficient performance which barely meets the course objectives E Fair 4 For a fair performance which adequately meets the course objectives but also displays several major weaknesses D Good 7 For a good performance which meets the course objectives but also displays some weaknesses C Very good 10 For a very good performance which meets the course objectives, with only minor weaknesses B Excellent 12 For an excellent performance which completely meets the course objectives, with no or only a few insignificant weaknesses. A Grade := Degree of realization of course objectives!

[ 17 ] Activation Exercise What are key ILO's in YOUR course ? T Concurrency: analyze systems compare models Concurrency: analyze systems compare models

[ 18 ] Questions... "What is good teaching?" The Short-Film Cognitive processes Association new ~ old The Book John Biggs "understanding" content  competence Student activation Susan & Robert Teacher models levels Course descriptions Constructive Alignment Top Competences 15% programming CS v. NAT v. MAT Students at University My research and teaching 'TLA' Teaching / Learning Activities Tips'n'Tricks ? recite generalize R R' Q The SOLO Taxonomy

From Intended Learning Outcomes to Teaching/Learning Activities Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching/Learning Seminar: Teaching/Learning Activities ( 10:30 – 10:45 )

From ILO's to TLA's Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching/Learning Seminar: Teaching/Learning Activities ( 10:30 – 10:45 )

Summary from Last Year Constructive Alignment and Intended Learning Outcomes

[ 22 ] Intended Learning Outcomes From content:...to competence: Intended Learning Outcomes (using verbs from The SOLO Taxonomy) ILO's: analyze... explain... compare... apply... Goal: understand: deadlock interference synchronization...  analyze explain Teacher Student name recite Understanding?!? Have the student do something; and then "observe" product and/or process vs. 

[ 23 ]  Make explicit ILO's ( )   (…and tell this to students) = ILO's = Teaching Learning Activities Assessment Constructive Alignment vs. ROBERT: extrinsically motivated ROBERT: extrinsically motivated SUSAN: intrinsically motivated SUSAN: intrinsically motivated Intended Learning Outcomes

[ 24 ] The Implementation Process 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2) Operationalize these goals: and express them as (SOLO) intended learning outcomes 3) Carefully design (TLA's): teaching/learning actitivites (  intended learning outcomes) 4) Carefully design (AT's): assessment tasks (  intended learning outcomes) alignment learning supportlearning incentive More information on implementing alignment: [ ]

Teaching/Learning Activities General issues

[ 26 ] Lecture (about Application) Teacher activity: Introduce Explain Elaborate Discuss application Give examples Show PPT slides Questions on slides Winding up Student activity: Listen Listen (maybe take notes) Understand? (correctly? deeply?) Listen (maybe take notes) Watch (maybe note points) Write answers to questions Possibly ask a question active teacher passive student vs. [ Biggs & Tang 2007, p.137 ] vs.

[ 27 ] Lecture functional knowledge declarative knowledge teacher centric student centric Lecture

[ 28 ] Student Activation Average retention rate 5% 10% 20% 30% 50% 75% 80% Lecture Reading Audiovisual Demonstration Discussion group Practice by doing Teaching others [ NTL Institute for Applied Behavioral Science, Bethel, Maine ] passive student active student activation "The (in-famous) Learning Pyramid": Doing: Learning to do Doing: Learning to do Listening: Learning about Listening: Learning about

[ 29 ] Listening vs. Doing Learning (about): about application about cooking about programming about designing about analysis about construction about relating... Learning (to do): to apply to cook to program to design to analyse to construct to relate... student listening student doing !! vs. (to something about something)

[ 30 ] Main Point !!! Note: there is a workshop on this later today (17: :20) Design TLA's = ILO's Given (carefully chosen) : student-centric ILO's: analyze... explain... compare... apply... FOCUS TODAY!

Thank you ! Have a great seminar !

Aligning your TLA's with your ILO's Claus Brabrand ((( ))) ((( ))) Associate Professor, IT University of Copenhagen Denmark ITU Teaching/Learning Seminar: Teaching/Learning Activities ( 17:35 – 18:20 ) WORKSHOP:

[ 33 ] The Implementation Process 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2) Operationalize these goals: and express them as (SOLO) intended learning outcomes 3) Carefully design (TLA's): teaching/learning actitivites (  intended learning outcomes) 4) Carefully design (AT's): assessment tasks (  intended learning outcomes) alignment learning supportlearning incentive ILO's: analyze systems for properties (e.g. deadlock) compare models of systems (wrt. behavior) [ ]

[ 34 ] Workshop (groups of 3) 1) Pair up (groups of 3) and choose 1-2 key ILO's from your courses: (e.g., 'to program', 'to design', 'to construct', 'to analyse', 'to relate', 'to compare',...) 2) Design a student-centric TLA for those key ILO's (relating today's material to your teaching) 3) Pair up with another group and explain your TLA (just explain the main points and the student-centricity) 5' 30' 2x5' Exercise on "Teaching/Learning Activities" (From ILO's to TLA's) ITU, Nov 20, 2009

BONUS SLIDES - = -

[ 36 ] Key References ”Teaching for Quality Learning at University” John Biggs & Catherine Tang Society for Research into Higher Education, McGraw-Hill. ”Evaluating the Quality of Learning: The SOLO Taxonomy” John Biggs & Kevin F. Collis London: Academic Press, 1982 ”Teaching Teaching & Understanding Understanding” Claus Brabrand & Jacob Andersen 19 minute award-winning short-film (DVD) Aarhus University Press, Aarhus University, 2006 "Constructive Alignment & The SOLO Taxonomy: a Comparative Study of University Competencies in Computer Science vs. Mathematics" Claus Brabrand & Bettina Dahl CRPIT, Vol. 88, ACS 3-17, R. Lister & Simon, Eds., 2007

[ 37 ] Common SOLO Competences From a study of 632x courses at NAT/(AU+SDU) ie. science context

[ 38 ] The BLOOM Taxonomy (1956) The BLOOM Taxonomy: Knowledge Comprehension Application AnalysisEvaluationSynthesis Qualitative Quantitative SOLO 4+5 SOLO 2+3 ”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item” -- (Biggs & Collis, 1982) ”

[ 39 ] Constructive Alignment ILO's (Intended Learning Outcomes) Explicitly defined and communicated TLA's (Teaching/Learning Activities) Designed to meet ILO's AT's (Assessment Tasks) Designed to assess ILO's learning support learning incentive