1. Technology Enhanced Learning at University - How can learning enhancement be demonstrated? Adrian Kirkwood & Linda Price IET, The Open University

2. What does ‘enhancement’ mean? Expression Technology Enhanced Learning (TEL) used frequently - but without a clear, shared meaning ‘Taken for granted’ – few explicit statements ‘Enhanced’ implies a value judgement What is being ‘enhanced’?  Teaching? Learning? Operational arrangements? How is ‘enhancement’ understood by teachers, students, policy-makers, senior managers?

3. HEFCE e-Learning Strategy, 2009 Efficiency Existing processes carried out in more cost-effective, time-efficient, sustainable or scalable manner Enhancement Improving existing processes and the outcomes Transformation Radical, positive change in existing processes or introducing new processes Benefits might be felt at three different levels, depending on the type of intervention:

4. Demonstrating ‘enhancement’? What must be done to demonstrate an enhancement in learning? Much depends upon the type of enhancement being sought Interpreted in many ways - a contested issue Often difficult to discern from published accounts of innovations or interventions  Need to reveal the underlying assumptions.

5. Technology as Agent The potential agent for change is often assumed to be the technology itself (i.e. device and/or software) If teachers use and/or get their students to use a particular technology, this will in and of itself improve student learning. 5

6. Teacher as Agent Less frequently, the main agent for change is seen as the teacher, through what they are trying to achieve by using a technological tool. What matters is how teachers – as agents – have chosen to design teaching & learning with technology, using appropriate tools to achieve significant outcomes. 6

7. Literature Review A review of published articles (2005-2010) of evidence-based learning and teaching with technology The Abstracts were scrutinised to ensure that they fulfilled the following criteria:  Use of technology for specific teaching and/or learning purposes in higher education;  Use of technology for specific teaching and/or learning purposes associated with one or more particular courses/modules or groups of students;  Provided some form of evaluative evidence of the impact of the technology use described. 7

8. Literature Review These criteria enabled us to eliminate articles that were wholly or primarily about  technology innovations in schools  students’ attitudes to and use of technologies in general  plans for uses of technology that were yet to be introduced with students  the generalised or idealised potential or affordances of technologies in education 8

9. Literature Review After applying the criteria, 47 unique articles and papers were selected for review. Variations found in terms of ‘enhancement’:  Differing conceptions of ‘enhancement’ being used by teachers / researchers  Different methods employed to demonstrate enhancement  Differing forms of evidence considered necessary to demonstrate achievement of enhancement  Problems with the appropriateness and adequacy of evidence in relation to conception of enhancement. 9

10. Literature Review Various research approaches had been used, from quasi-experimental to case studies. The approaches and methods used to collect data provide some indications of the ways in which the practitioners/researchers involved conceptualise learning and teaching in higher education and the types of evidence that they consider to be salient. 10

11. Categories of Intervention - 1 Replicating existing teaching practices a. An element of conventional teaching replicated and delivered to students via some form of technology b. A comparison of different technologies for delivering the same teaching

12. Categories of Intervention - 2 Supplementing existing teaching a. Making available versions of existing teaching that students can access and use whenever they want b. Developing additional learning resources or tools

13. Categories of Intervention - 3 Transforming the learning experience a. Re-design of learning activities to promote active learning b. Effective use of TEL activities for richer learning

14. 1. Replicating existing teaching practices Nature of ‘enhancement’ sought often imprecise ‘Comparative study’ method often used (i.e. only the means of delivering teaching changed) Learning gains usually confined to test or assessment scores – the acquisition and retention of knowledge (i.e. quantitative view of learning) Positive student (and teacher) attitudes sought

15. 2. Supplementing existing teaching Nature of any ‘enhancement’ often imprecise – often linked to flexibility for students Self-report student attitudes & experiences  Problems of interpretation  The 4-stage model of evaluation (Kirkpatrick, 1976) Some comparative studies (how appropriate?) ‘Enhancement’ often taken as a quantitative change rather than a qualitative improvement (e.g. Säljö, Perry) Type of test and assessment used not always sensitive to nature of enhancement sought

16. The 4-stage model of evaluation Reaction  to what degree participants react favourably Learning  to what degree participants acquire the intended knowledge, skills, attitudes, confidence and commitment Behaviour  to what degree participants apply what they learned to their situation Results  to what degree targeted outcomes are achieved as a result Effectiveness is best evaluated at four progressively challenging levels (Kirkpatrick, 1976)

17. 3. Transforming the learning experience Interventions often responded to identified learning / teaching issues Often associated with achieving qualitative changes in student outcomes Qualitative analysis of individual / group interviews about processes and products of learning activities Scrutiny of student-generated materials / presentations for richer outcomes

18. But … technology or pedagogy? When the student learning experience had been changed, what was the main driver  use of technology?  redesign of the learning and teaching approach and goals?  both? Teachers’ differing conceptions of teaching and learning with technology are an important influence (Kirkwood & Price, 2012)

19. Differing conceptions of teaching Perceptions of Academic Context Conceptions of Teaching Approaches to Teaching Conceptions of Teaching & Learning with Technology Perceptions of Technological Context Approaches to Teaching & Learning with Technology

20. Review of Findings There remains a considerable lack of clarity – and little consensus – about what constitutes an enhancement in student learning. Considerable differences are expressed about the form(s) of evidence that are necessary to demonstrate ‘enhancement’. A clearer articulation of what is meant by ‘enhanced learning’ in higher education might help advance the effectiveness of TEL. 20

21. 21