1. OBTL at CityU in the Hong Kong Context 7th December 2006 Professor John Biggs Principal Consultant OBTL Project

2. Outcomes-based Education (OBE) There are three main forms of “OBE”. All focus on educational outcomes but each is based on a different philosophy. It’s easy to confuse them. 1. Outcome-based Education at school level. Originally for disadvantaged children, but later used generally e.g. Target Oriented Curriculum (TOC) for individualising teaching. 2. Outcomes-based Education. Outcomes at institutional level, used for benchmarking, credit-transfer. 3. Outcomes-based Teaching and Learning (OBTL). Defining learning outcomes at course and subject level, to enhance teaching and learning.

3. Institutional, Progamme and Course Outcomes OBE OBTL Outcomes at the institutional level : Graduate attributes, General competencies for use within and across institutions. Outcomes at the programme level : Academic and professional expertise and values Outcomes at the course level: where the teaching, learning and assessment actually occur

4. What the UGC said “ The UGC’s goal in promoting outcome-based approaches is simple and straightforward— improvement and enhancement in student learning and teaching quality.” (Alice Lam, May 06) OBTL, in other words. “We think that the curriculum revision under '3 + 3 + 4' will be a good opportunity to weave 'outcomes' into the new curriculum.” The UGC is providing extra funding to promote early adoption of OBTL and a supportive Task Force.

5. Outcomes-based Teaching and Learning (OBTL): Classroom Based Intended Learning Outcomes: What the student has to do Teaching: To facilitate attaining the ILOs Assessment: How well has the student has attained the ILOs

6. Implementing OBTL using Constructive Alignment ILO: What the student has to learn, e.g. 'to analyse X' Teaching/Learning Activities: Engaging the Student to perform verb in the ILO Assessment Task: How well the student has met the ILO

7. Outcomes-based Teaching and Learning through Constructive Alignment There are four steps in designing such teaching: 1. describe intended learning outcomes in the form of verbs in a context, indicating what is to be attained and verbs in a context, indicating what is to be attained and the desired quality of attainment. the desired quality of attainment. 2. create a learning environment likely to bring about the intended outcomes. intended outcomes. 3. use assessment tasks enabling you to judge if and how well students’ performances meet the criteria. well students’ performances meet the criteria. 4. transform these judgments into standard grading criteria. criteria.

8. Constructive Alignment is a Framework for Reflective Practice Constructive alignment is not a prescriptive “method”: each step provides teachers a focus for reflective practice. each step provides teachers a focus for reflective practice. Teachers can then design ILOs, select TLAs and assessment and grading procedures that are the most effective and practicable given the particular context and subject matter of each. Several teachers have told us that this simply sharpens what they have been doing already. Absolutely! Good teachers naturally think in terms of what their students have to learn, how they might best learn it, and how they might best confirm that students have indeed learned what was intended. Absolutely! Good teachers naturally think in terms of what their students have to learn, how they might best learn it, and how they might best confirm that students have indeed learned what was intended.

9. Constructive Alignment Intended Learning Outcomes (ILOs) expressed as verbs students have to enact expressed as verbs students have to enact The very best understanding that could be The very best understanding that could be reasonably expected: verbs such as reasonably expected: verbs such as hypothesise, apply to “far” domains, hypothesise, apply to “far” domains, generate, relate to principle, etc. generate, relate to principle, etc. Highly satisfactory understanding: verbs Highly satisfactory understanding: verbs such as explain, solve, understand main such as explain, solve, understand main ideas, analyze, compare, etc. ideas, analyze, compare, etc. Quite satisfactory learning, with under- Quite satisfactory learning, with under- standing at a declarative level: verbs such standing at a declarative level: verbs such as elaborate, classify, cover topics a to n, as elaborate, classify, cover topics a to n, Understanding at a level that would Understanding at a level that would warrant a Pass: low level verbs, also warrant a Pass: low level verbs, also inadequate but salvageable higher level inadequate but salvageable higher level attempts. attempts. Teaching / Learning Activities Activities Designed to Designed to elicit desired elicit desired verbs verbs May be: May be: Large class activities Small class activities Teacher-managedPeer-managedSelf-managed as best suits context Assessment Tasks Format such that the target verbs are elicited and deployed in context. Criteria clearly allow judgement as to the quality of the student'sperformance

10. Intended Learning Outcomes (ILOs). Statements of what students are expected to be able to do as a result of studying a course or programme. able to do as a result of studying a course or programme.. Expressed in the form of learning verbs needed to achieve the outcome. The teaching/learning activities and assessment tasks also address these verbs. achieve the outcome. The teaching/learning activities and assessment tasks also address these verbs.

11. The Verbs in the ILOs Generic high level verbs include: reflect, hypothesise, solve unseen complex problems, generate new alternatives. Lower level verbs include: describe, identify, memorize. Often the these lower level ILOs can be subsumed under higher level. Some ILOs will contain ‘large’ verbs like ‘write’ with focus on a particular task (write a lab report with specifications X, Y, Z …)

12. The SOLO Taxonomy with sample verbs indicating levels of understanding Competence Prestructural Unistructural Multistructural Relational Extended Abstract Prestructural Unistructural Multistructural Relational Extended Abstract one relevant several relevant integrated into generalized to one relevant several relevant integrated into generalized to aspect independent aspects a structure new domain IdentifyName Follow simple procedure procedure CombineDescribeEnumerate Perform serial skills List AnalyzeApplyArgueCompare/ contrast contrastCriticize Explain causes RelateJustify CreateFormulateGenerateHypothesizeReflectTheorizeIncompetence FailIncompetent Misses point....

13. Distinguish the kind of knowledge you want Distinguish the kind of knowledge you want Declarative knowledge:. Knowing about things. Knowledge we can declare to someone in writing or telling writing or telling e.g. ‘Distinguish between OBE and OBTL’ Functioning knowledge:. Knowledge we put to work in solving a physics problem, analysing a case study, designing a building, making an argument e.g. ‘Write an ILO for a subject you are currently teaching’ currently teaching’

14. Designing Teaching/Learning Activities to Align with Intended Learning Outcomes Having designed the ILOs, we now need to activate the verbs or learning activities embedded in the ILOs by designing suitable Teaching/Learning Activities that will facilitate students achieving the ILOs.

15. Typical ILO Possible TLAs Typical ILO Possible TLAs Describeset reading, lecture, field trip Explaintutorial, written essay Integrateproject, assignment Applyproject, case study Solve problemPBL, case study Design, createproject, creative writing Hypothesiseexperiment, project Reflectreflective diary The point is not how you are going to teach but how and what you want your students to learn. NOTE! Many of these TLAs can be assessments tasks as well. Then you have excellent alignment.

16. Four common teaching situations and associated teaching and learning activities Situation Teaching activitiesLearning activities LECTURE talk, explain, clarify listen, take notes, accept, query, discuss with peers, one-minute paper TUTORIAL set/answer questions pre-read, prepare questions, provide feedback learn from peers, critique, analyse PROJECT set brief, ongoing apply, create, self-monitor feedback communicate, teamwork PBL set problemsset learning goals, design, apply, accessing desired integrate, solve problems content, skills What teaching /learning activities will best facilitate your ILOs?

17. Designing Assessment Tasks (ATs) Steps: 1.Selecting a practicable task that embodies the target ILO verb. (Try the TLA first). 2.Making a judgment on how well the ILO has been met by the students' performance on the ATs – developing grading criteria.

18. Common ILOs Possible Assessment Tasks Describeassignment, essay question exam Explainassignment, essay question exam, oral, letter-to-a-friend Integrateproject, assignment Analysecase study, assignment Applyproject, case study, experiment Solve problemcase study, project, experiment Design, createproject, experiment Reflectreflective diary, portfolio, self-assessment Communicatea range of oral, writing or listening tasks addressing the ILOs, e.g. presentation, debate, role play, reporting, assignment, precis, paraphasing, answering questions etc.

19. Examination is a very commonly used assessment task especially for large classes. We need to consider if 1. examinations involving answering essay type of questions under invigilated conditions is able to assess students' performance in some high level ILOs, e.g. apply, reflect, create etc. 2.there are other alternative assessments tasks which will more appropriately addess those high level ILOs.

20. Assessments Tasks for Large Classes Task-formatUseful for Final examEnsuring work is student’s own, overview of whole course Final examEnsuring work is student’s own, overview of whole course Multiple choiceRecognition, strategy, coverage Multiple choiceRecognition, strategy, coverage Ordered outcomeHierarchies of understanding Ordered outcomeHierarchies of understanding Posterintegration, application, creativity Posterintegration, application, creativity Concept maps, Coverage, relationships Concept maps, Coverage, relationships Flow charts Flow charts Three minute essayDifferent levels of understanding, sense of relevance Three minute essayDifferent levels of understanding, sense of relevance GobbetsInterpret significant detail, explain GobbetsInterpret significant detail, explain Short answerRecall units of information, coverage Short answerRecall units of information, coverage Letter to a friendIntegration, application, reflection Letter to a friendIntegration, application, reflection

21. Assessment Issues 1.Assess the ILO or the Assessment Task itself? 2.Assess the task quantitatively (marking) or qualitatively (holistically)? 3.How to combine AT performances to obtain a final grade: quantitatively or holistically? The most important thing is that the assessment task is aligned to the ILO. The methods of assessing the task and of combining results into a final grade grading are important but secondary issues.

22. Example of Grading Criteria for an ILO Marginal Pass Satisfactory Good Excellent Marginal Pass Satisfactory Good Excellent D C- C C+ B- B B+ A- A A+ D C- C C+ B- B B+ A- A A+ Grade 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 4.30 Grade 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 4.30 point/unit point/unit ILO Reflect ILO Reflect Able to use availableAble to use available Able to use available As in “Good”. Able information to self- information to self- information to self- to generalize self- evaluate and identify evaluate and identify evaluate and identify evaluation to beyond evaluate and identify evaluate and identify evaluate and identify evaluation to beyond limited aspects of own more aspects of own the full range of owm existing context. limited aspects of own more aspects of own the full range of owm existing context. strengths and weaknesses strengths and weaknesses strengthes and weak- Suggest ways of strengths and weaknesses strengths and weaknesses strengthes and weak- Suggest ways of in a general sense. No in a general sense. Little nesses. Self-evaluation improving perform- in a general sense. No in a general sense. Little nesses. Self-evaluation improving perform- evidence of suggestions application of theory in is based on theory. ance to real-life evidence of suggestions application of theory in is based on theory. ance to real-life of ways to improve self-evaluation and limited Increasingly able to professional of ways to improve self-evaluation and limited Increasingly able to professional performance. No evidence suggestions of ways to suggest ways tocontest. performance. No evidence suggestions of ways to suggest ways tocontest. of theory being used in improve performance. improve performance of theory being used in improve performance. improve performance self-evaluation. in a specific context.

23. Grading the AT In other courses it may be more relevant to grade the task, which is in effect an ILO. For example, in language courses the ILO is that students can write an effective argument in English/Chinese, or in Science, that they can write a lab report.

24. Assessing quantitatively For:.Used to it..Seems to be the logical way to assess in certain courses..Logistically easy. Against:.Defines quality in terms of accumulating small quantities.. Measurement error also accumulates, thus invalidating fine discriminations. E.g. there is no valid difference between 74and 75, yet to the student it can make a BIG difference - an A or a B! Or worse, the difference between pass or fail..Sends undesirable messages to students (backwash).

25. Assessing qualitatively For: For: Student’s performance is appropriately assessed against what they are intended to learn – criterion-referenced.Student’s performance is appropriately assessed against what they are intended to learn – criterion-referenced. Backwash is positive.Backwash is positive. The final grade tells students what they have achieved and what they need for a better grade.The final grade tells students what they have achieved and what they need for a better grade.Against: Requires a different mind set for some teachers.Requires a different mind set for some teachers. Initially more work in designing ILOs and suitable assessment tasks, but once established is not extra work.Initially more work in designing ILOs and suitable assessment tasks, but once established is not extra work.

26. Qualitative assessment involves making judgments against criteria (rubrics), not by counting ‘marks ’ If ILOs are to reflect workplace or ‘real world’ standards it is more appropriate to assess them qualitatively. ‘Real world’ outcomes are not stated in terms of marks obtained.

27. An example of a holistic way of deriving f i nal grade Curriculum and Instruction: A subject in a course for Ed. Psychlsts. ILOs: 1. Apply the principles of good teaching and assessment to chosen contexts. 2. Relate selected aspects of curriculum design and management to the educational system in Hong Kong. 3. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist. 4. Show examples of your reflective decision-making as an educational psychologist. Final grade: A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice. B Awarded when all ILOs have been met very well and effectively. C Awarded when the ILOs have been addressed satisfactorily, or where the evidence is strong in some ILOs, weaker but acceptable in others. F Less than C, work plagiarised, not submitted.

28. Assessment: Curriculum and Instruction Show evidence that you have learned according to the criteria in the ILOs. Keep a reflective journal to record useful insights as a data base. A paper explaining how you would like to see the Hong Kong educational system implement a major educational reform. You should have ILO (2) in mind. A report specifically addressing ILOs (3) and (4), a review of those aspects of the course that you think will probably enhance your work as an EP. Your own rationale of your group presentation. You should have ILO (1) in mind. A self-evaluation showing how you have addressed each of the ILOs. Place in a portfolio of about 5,000 words.

29. Use of OBTL in Hong Kong based on the Constructive Alignment Model At City U, 240 degree courses are committed and on the way: Faculty of Business: 28 Faculty of Humanities and Soc Sci 86 Faculty of Science and Engineering 105 School of Creative Media 14 School of Law 4 Dept. Building Science and Technology 3 At PolyU, well on the way Baptist U starting Hong Kong U, 4 faculties in PBL (a form of constructive alignment) and now moving more generally

30. Matching OBE and OBTL It's important that the taxonomy used at the institutional level is not only compatible with that used in the classroom but can actually guide the design of programmes and courses. Too many fragmented knowledge/ skill/ value/ societal outcomes make aligning TLAs and ATs to those outcomes difficult if not impossible.

31. A Suggested Taxonomy for Institutional level outcomes Declarative Outcomes : e.g. content knowledge expert. e.g. content knowledge expert. Functioning Outcomes: e.g. creative/innovative professional, life-long learner, professional e.g. creative/innovative professional, life-long learner, professional expert, good communicator, team-worker. expert, good communicator, team-worker. Value outcomes : e.g. ethical professional. e.g. ethical professional. Graduate outcomes need to be stated at an appropriate level (extended abstract in SOLO terms): e.g. ‘able to solve unseen problems‘. These graduate outcomes need to be teachable and assessable, usually in the Programme/Course ILOs.

32. Institutional, Progamme and Course Outcomes OBE OBTL Institutional, Progamme and Course Outcomes OBE OBTL Outcomes at the institutional level : Declarative, Functioning, Value Outcomes at the programme level: Above as appropriate to the programme. Outcomes at the course level: Programme ILOs embedded where appropriate as Course ILOs, to which TLAs and ATs aligned.

33. Reference Biggs, J.B. Teaching for Quality Learning at University. Buckingham: Open University Press/McGraw Hill, 2003. New edition by Biggs and Tang explicitly addressing OBTL is in process, due late 2007.