Learning Outcomes: Keeping it Real Liz Warr

Eduspeak  Dialect particularly spoken in Educational Development Units, including favoured phrases such as intended learning outcomes, assessment criteria, constructive alignment, cognitive dissonance and performance indicators

Aim of the Session  To familiarise participants with the pedagogic principles underpinning the construction of Intended Learning Outcomes in the context of their impact on assessment practice

Learning Outcomes At the end of the session participants will be able to:  Apply the principle of ‘constructive alignment’ to module design  Devise module specifications that meet the needs of key stakeholders  Select assessment criteria that measure achievement of learning outcomes  Write learning outcomes that reflect the academic level of student learning

Assessment Criteria  Survive Scrutiny of Validation and/or Accreditation Panel  Gain Approval from ASC  Receive positive Feedback from QAA Auditors

Learning Outcomes  Statements of what a learner is expected to know, understand or be able to demonstrate after completion of a process of learning  Represent a major achievement (a significant capability, process or task)

Assessment Assessment is a generic term for a set of processes that measure the outcomes of students’ learning in terms of knowledge acquired, understanding developed and skills gained. (QAA, 2000)

Assessment Criteria  Descriptions of what the learner is expected to do, in order to demonstrate that a learning outcome has been achieved  Set at a threshold level of achievement: performance in excess of this can be further differentiated by grading criteria

Credit Level  An indicator of relative demand, complexity and depth of learning, and of learner autonomy  Generic level descriptors are available for the full spectrum of Further and Higher Education.  (Credit levels are not intrinsically linked to years of full-time study)

Assessment Design Is based on the principles of “Constructive Alignment” where teaching methods and assessment processes are aligned with the learning outcomes so that all aspects of the teaching system support appropriate student learning (Biggs, 1999)

Pedagogic arguments for Outcome-based approach  Promotes active learning  Student-centred rather than tutor focussed  Provides simple mechanism for setting and reviewing academic standards  Increases transparency  Increases flexibility for Accreditation of Prior Learning (APCL and APEL)  Facilitates progression and student mobility

Module A self-contained, formally structured learning experience with a coherent and explicit set of learning outcomes and assessment criteria which determine its credit level. It is the smallest entity for which credit may be awarded and will be assigned a volume of credit which defines the nominal size of the module (estimated notional learning hours for its completion) A self-contained, formally structured learning experience with a coherent and explicit set of learning outcomes and assessment criteria which determine its credit level. It is the smallest entity for which credit may be awarded and will be assigned a volume of credit which defines the nominal size of the module (estimated notional learning hours for its completion)

Notional Learning Time  The number of hours which it is expected a learner will spend, on average, to achieve the specified learning outcomes  The most widely used credit : learning ratio is 1 credit :10 hours.  1 full time undergraduate academic year = 120 HE credits or 1200 notional learning hours  1 Calendar Year = 180 HE credits or 1800 notional learning hours

Module Development Aims of Module Define Learning Outcomes Design Assessment Methods and Tasks Define Assessment Criteria/ Grading Criteria Develop Learning and Teaching Strategies Make Assessment Decisions Provide Feedback for Students Evaluate and Revise

Bloom’s Taxonomy of Learning Objectives Evaluation Synthesis Analysis Application Comprehension Knowledge (Acquisition) (Bloom, 1956 cited in Brown, 2001)

Hierarchy of External Influences  Sector: - Framework for Higher Education Qualifications (FHEQ) - Qualification descriptors - Qualification descriptors  Discipline: - Subject Benchmark Statements (QAA)  Programme: - Programme Specification (Institution)  Module: - Generic level descriptors (HE Credit Framework for England) - Grading Criteria (Institution) - Grading Criteria (Institution)

Threshold vs Higher Level Achievement  Assessment Criteria test ILO’s at threshold level  If achieved - Module Pass - Award of Academic Credit - Award of Academic Credit  Grading criteria measure achievement above threshold (eg Pass, Merit, Distinction, 3 rd Class, 2 nd Class, 1 St Class  Higher level achievement does not result in award of additional Credit

Building in Higher Level Skills Either Specify in the Learning Outcomes (must be assessed, must be passed) Or Specify in the rationale and context section and build into grading criteria (should be assessed, may not need to be passed) Often used to justify the use of 40% as threshold pass (remaining 60% can reflect achievement of higher level skills / graduate attributes set out in generic institutional assessment criteria

Assessment Practice Assessment should be an integral part of the curriculum delivery on an ongoing basis and must be timed carefully so that students obtain maximum benefit and so that both students and staff have manageable workloads It is good practice to map out the assessment for a whole programme:   to ensure comprehensive coverage of the Programme Learning Outcomes   to ensure students experience a balanced diet of assessment methods   to ensure realistic time-scales are set; for completion by students, marking by assessors and provision of feedback to the learners

Assesperanto  the dialect which needs to be learned by students if they are to succeed in assessments - particularly those requiring advanced competences with academic English

Feedback Is like fish – only good when its fresh

Assessment Practice “Institutions should ensure that the amount and timing of assessment enables effective and appropriate measurement of students’ achievement of intended learning outcomes” “Institutions should ensure that the amount and timing of assessment enables effective and appropriate measurement of students’ achievement of intended learning outcomes” (QAA Code of Practice on Assessment)

When Writing Learning Outcomes Avoid words like:  know, understand  be familiar with  become acquainted with  have a good grasp of  obtain a working knowledge of  be aware of  believe  be interested in

When Writing Learning Outcomes Consider using words like:  Describe  Evaluate  Explain  Identify  Distinguish  Design  construct  solve

References Biggs, J (1999) Teaching for Quality Learning at University. Buckingham UK: Open University Press Brown, G (2001) Assessment: A Guide for Lecturers. York: LTSN Generic Centre Assessment Series Guides 2001 Brown, S and Glasner, A (2003) Assessment Matters in Higher Education, 3 rd Edition. Buckingham UK: Open University Press Gosling, D and Moon, J. How to use Learning Outcomes and Assessment Criteria. London:SEEC QAA (2006) Code of Practice for the assurance of academic quality and standards in Higher Education. Section 6: Assessment of students QAA (2008) The Framework for higher education qualifications in England, Wales & Northern Ireland QAA (2008) Higher Education credit framework for England Rust, C., O’Donovan, B & Price, M. (2005) Assessment and Evaluation in Higher Education. Vol 30, No

Facilitese  Used by some trainers, often including phrases such as ‘now let’s go into groups’ and ‘go to the breakout rooms’ and ‘I’ll just unpack that for a minute’

Doing it for Real  Working in small groups, produce an outline module specification for an assigned topic -Aim -Aim -L.O’s -L.O’s -Assessment Strategy -Assessment Strategy -Teaching and Learning Strategy -Teaching and Learning Strategy -Assessment Criteria -Assessment Criteria

Get Real WHAT do you want the Students to be able to do? WHAT do you want the Students to be able to do? HOW do you want them to demonstrate that they can do it? HOW do you want them to demonstrate that they can do it? HOW WELL do they have to perform? HOW WELL do they have to perform?

What  Swim 50 meters

How  In a swimming pool wearing outdoor clothes

How Well  In 5 minutes