Learning Gain: Evaluation, Evidence and Enhancement Dr Camille B. Kandiko Howson @cbkandiko King’s College London Learning and Teaching Conference 21 March 2018 Newcastle University #ncllt18

Learning gain metrics and enhancement Overview Policy context Learning gain project Learning gain metrics and enhancement

Policy Context I

Students <Bored students in lecture theatres>

Graduates <Non-graduate jobs>

<David Willetts, Two Brains> <Browne Review of Higher Education> <Academically Adrift>

‘Academically Adrift’

‘lamentable’ <Jo Johnson> ‘patchy’ ‘value for money’

Teaching Excellence Framework Not really about ‘teaching’ per se ‘Teaching Excellence and Student Outcomes Framework’ Focus on impact and evaluation

<Sam Gyimah> <moneysupermarket. com> <moneysupermarket <Sam Gyimah> <moneysupermarket.com> <moneysupermarket.com/travel_insurance>

HEFCE  Office for Students students from all backgrounds are able to enter and succeed in higher education. Students should be able to make informed choices about their studies, with access to a diverse range of provision which meets their needs. students have access to high quality teaching and learning – whatever, wherever and however they are studying. students’ interests are protected, and that they receive value for money. students are well prepared for life after graduation, gaining qualifications which offer rewarding opportunities for postgraduate study or employment, aligned with the needs of employers. https://www.officeforstudents.org.uk/

Learning gain project II

Learning Gain: Challenges Student engagement with tests and surveys Motivating students to invest in tests that don’t contribute to assessment Context of English higher education Discipline bias in standardised tests Comparability of some entry and exit measures Reliability of student self-reports Data protection, data sharing, research ethics But closest current proxies for learning are satisfaction surveys and graduate salaries…

HEFCE Learning Gain HEFCE funding 13 mixed method projects involving 70 institutions over three years, using: Learner analytics/Grades Self-reported surveys Standardised tests Multiple measures of a specific theme National Mixed Methodology Learning Gain Project (NMMLGP) Higher Education Learning Gain Analysis (HELGA)

What is the purpose of higher education? What to measure What is the purpose of higher education? What is valued? What are student expectations of higher education? What should graduates know and be able to do?

Input and entry measures Entry qualifications and background characteristics Differences across the sector <Lack of level playing field>

Measures I Affective Transition experience Self efficacy Well-being Disposition to learning Confidence Resilience Satisfaction

Measures II Behavioural Student engagement Placements/ work-based learning Employability experiences Co-curricular activities Skills self-assessment VLE engagement Learner analytics

Measures III Cognitive General cognitive gain Disciplinary cognitive gain Critical Reasoning Skills Situational judgement Research methods

Output and outcome measures What have your students gained? Grades, progression Employability skills Affective Behavioural Cognitive Other outcome measures Back to purposes… Is it critical thinking, generic graduate skills, disciplinary mastery, developing employability or something more holistic?

Learning gain metrics and enhancement III

Uses of learning gain metrics Improve teaching and learning Personalised approaches to learning Learning analytics Pedagogy and curriculum design Contribute to student success Design interventions & support Evidence-based ‘nudge’ behaviours Predict student outcomes Marketing, admissions

Learning analytics and enhancement Shift from teaching to learning Culture of evidence and impact How is data (qualitative and quantitative) used in enhancement Gain credibility Raise awareness Target support Link individual practice with teaching teams and courses From ‘module-mania’ to larger curriculum focus How do you access, understand, interpret and use data? Who do your new partners in the institution need to be?

Learning gain and student voice Opportunities for students to be part of: Defining ‘value’ Data gathering Data analysis Data sharing ‘Actioning’ Enhancement Feedback and follow-up

© 2015 King’s College London. All rights reserved Dr Camille B. Kandiko Howson @cbkandiko camille.kandiko_howson@kcl.ac.uk http://www.hefce.ac.uk/lt/lg/ © 2015 King’s College London. All rights reserved