1. Outcome-based Curriculum Development Maiki Udam maiki.udam@archimedes.ee

2. Aim of the workshop To introduce the participants to the key concepts associated with the development of an outcome-based curriculum Develop an ability to write learning outcomes, linking them with qualifications and assessment criteria

3. Learning outcomes of the workshop On successful completion of this workshop, participants should: be familiar with the basic principles how to develop an outcome-based curriculum/program; be able to write learning outcomes for a specific subject/module; be able to link assessment methods and assessment criteria with learning outcomes.

4. Definitions The aim – a broad general statement of the teaching intention, e.g. “to introduce students to the basic principles of atomic structure” Learning outcomes – statements of what a student is expected to know, understand and/or be able to demonstrate after completion of a process of learning.

5. Definitions  “A curriculum is a plan for learning; therefore, what is known about the learning process and the development of the individual has bearing on the shaping of a curriculum. Such knowledge should determine which objectives are achievable under which conditions, and what variations and flexibilities in content and its organization are needed to provide for optimum effectiveness of learning.” (Taba, 1962)

6. Lehrplan vs Curriculum Lehrplan = input → outcome Curriculum = intended outcome→ input

7. The contribution of learning outcomes to the Bologna action lines Adoption of a system of easily readable and comparable degrees Promotion of mobility Establishment of a system of credits Promotion of co-operation in quality assurance Promotion of the European dimension in HE Lifelong learning

8. Advantages of learning outcomes Help teachers to tell students more precisely what is expected of them. Help students to learn more effectively: students know where they stand. Help teachers select the appropriate teaching strategy matched to the intended learning outcomes (lectures, seminars etc). Help teachers to tell their colleagues more precisely what a particular activity is designed to achieve.

9. Before writing... ECTS Handbook NQF, EQF, QF-EHEA Higher education standards, state curricula The mission/vision of the university Feedback from employers/alumni/students Suggestions of accreditation teams

10. „Backward design“ QF-EHEANQF Learning outcomes (LO) for a study programme Choosing modules/subjects LOs for modules/subjects Assessment methods and criteria (incl RPL) Content and teaching methods Quality Assurance 10

11. Writing learning outcomes (LO) Each LO should contain an active verb, an object, and a phrase that gives the context, e.g. Operate the range of instrumentation specified in the module safely and efficiently in the chemistry laboratory. Use only one verb per LO. Try to avoid vague terms like know, understand etc... Avoid complicated sentences. Ensure that the LO of the subject relate to the overall outcomes of the programme.

12. ... to be continued... The LO must be observable and mesurable. Ensure that the LO-s are capable of being assessed. Bear in mind the timescale within which the outcomes are to be achieved. Use more the top categories of Bloom’s Taxonomy. Include generic competencesgeneric competences Avoid writing more than 8 LO-s per subject! Ask your colleagues and former students if the LO-s make sense to them.

13. Bloom’s Taxonomy (1956) 6. Evaluation 5. Synthesis 4. Analysis 3. Application 2. Comprehension 1. Knowledge

14. Arrange, collect, define, describe, duplicate, enumerate, examine, find, identify, label, list, memorise, name, order, outline, present, quote, recall, recognise, recollect, record, recount, relate, repeat, reproduce, show, state, tabulate, tell.

15. 2. Comprehension Associate, change, clarify, classify, construct, contrast, convert, decode, defend, describe, differentiate, discriminate, discuss, distinguish, estimate, explain, express, extend, generalise, identify, illustrate, indicate, infer, interpret, locate, paraphrase, predict, recognise, report, restate, rewrite, review, select, solve, translate.

16. 3. Application Apply, assess, calculate, change, choose, complete, compute, construct, demonstrate, develop, discover, dramatise, employ, examine, experiment, find, illustrate, interpret, manipulate, modify, operate, organise, practice, predict, prepare, produce, relate, schedule, select, show, sketch, solve, transfer, use.

17. 4. Analysis Analyse, appraise, arrange, break down, calculate, categorise, classify, compare, connect, contrast, criticise, debate, deduce, determine, differentiate, discriminate, distinguish, divide, examine, experiment, identify, illustrate, infer, inspect, investigate, order, outline, point out, question, relate, separate, sub-divide, test.

18. 5. Synthesis Argue, arrange, assemble, categorise, collect, combine, compile, compose, construct, create, design, develop, devise, establish, explain, formulate, generalise, generate, integrate, invent, make, manage, modify, organise, originate, plan, prepare, propose, rearrange, reconstruct, relate, reorganise, revise, rewrite, set up, summarise.

19. 6. Evaluation Appraise, ascertain, argue, assess, attach, choose, compare, conclude, contrast, convince, criticise, decide, defend, discriminate, explain, evaluate, grade, interpret, judge, justify, measure, predict, rate, recommend, relate, resolve, revise, score, summarise, support, validate, value.

20. Levels of defining the outcomes Minimum /threshold Medium Maximum

21. Breath and depth of defining the outcomes Core principles Essential to know Could have been heard

22. Excercise Write aims and learning outcomes for the subject “Academic writing” – 3 ECTS (appr 80 hrs) – Target group: 1st year BA students

23. Why to assess students’ work? Necessity for students: Necessity for teachers: Necessity for institution:

24. Assessment The biggest impact for student’s learning has assessment!

25. Meaning of Assessment Teacher’s perspective: objectives Learner’s perspective: assessment Teaching Learning Assessment Learning Outcomes

26. Options for Assessment Assessment of learner’s development Normative assessment (“ECTS assessment”) Assessment against fixed standards – outcome-based assessment

27. Summative assessment Summative assessment is used to evaluate student learning, skill acquisition, and academic achievement at the conclusion of a defined instructional period. Summative-assessment results are often recorded as scores or grades.

28. Formative assessment The general goal of formative assessment is to collect detailed information that can be used to improve instruction and student learning while it’s happening to inform in- process teaching and learning modifications. Grading should always be more about what’s ahead than what’s behind!

29. Summative vs formative Formative assessments are for learning, while summative assessments are of learning

30. Assessment Methods Essay Open-book examination Multiple choice test Presentation Case study Project Practicum Portfolio …

31. Assessment Criteria Threshold assessment criterion – a standard of performance that a learner must reach in order to demonstrate the achievemnt of a specified element of learning. Grade assessment criterion – a specified standard of performance that the learner must reach in order to be allocated a particular grade. In this case there is likely to be a series of grade assessment criteria related to the different grades. (Moon, 2002)

32. Threshold criterion (TC) - Example LO: At the end of this module, the student will be expected to be able to explain and demonstrate the main features of effective academic essay at level 1. TC: The essay will be word processed and between 1500 and 2000 words on a given topic. The essay will relate to its title, will be clearly written and structured, will demonstrate the contribution of further reading, and thinking. The student will be able to explain how the essay demonstrates these features and how they contribute to its overall effectiveness. (Moon, 2006)

33. Grade assessment criteria Start from threshold Leave space for unexpected/extraordinary results

34. Excercise Choose methods and write grade assessment criteria for one method for the subject “Academic writing”

35. Learning outcomes Assessmen t methods Learning activities Student’s estimated workload (hours)

36. Checklist Study programme Learning outcomes A B C D E F Subject 1X X Subject 2X X X Subject 3 X X Subject 4X XX

37. References Bechtel, L. J. et al. (2005). An Objectives-Based Approach to Assessment of General Education. Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition. Biggs, J.B. (1995). Assessing for learning: some dimensions underlying new aproaches to eduvational assessment. Alberta Journal of Educational Research. 41, 1-17 Biggs, J. (1999). Teaching for quality learning at university. Society for Reasearch into Higher Education & Open University Press. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook I – Cognitive domain. New York: McKay. ECTS Users’ Guide (2015). http://ec.europa.eu/education/library/publications/2015/ects-users- guide_en.pdf http://ec.europa.eu/education/library/publications/2015/ects-users- guide_en.pdf Kenndey, D. (2007). Writing and Using Learning Outcomes. A Practical Guide. University College Cork, Ireland.

38. Moon, J. (2002). The Module and Programme Development Handbook. Routledge Falmer, London. Moon, J. (2006). Linking levels, Learning Outcomes and Assessment Criteria – EHEA version. Stiehl, R, & Lewchuk, L. (2008). The Assessment Primer. OR: The Learning Organization. The Glossary for Education Reform (2014). edglossary.org Wiggins, G., & McTighe, J. (2005). Understanding by Design (expanded 2nd ed.). Association for Supervision and Curriculum Development, Alexandria, Virginia USA.