1. The student experience of blended learning Dr. Rhona Sharpe rsharpe@brookes.ac.uk Oxford Centre for Staff and Learning Development Team: Greg Benfield George Roberts Richard Francis

2. Oxford Centre for Staff and Learning Development Besides all the complexity created by marked differences across subject areas and myriad individual differences among both staff and students which prevent simple patterns emerging, there are additional crucial differences between the idealized world described by research and the actual world experienced by the participants. Entwistle, N., McCune, V. and Hounsell, J. (2002). 'Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings.' online http://www.ed.ac.uk/etl/docs/ETLreport1.pdf. http://www.ed.ac.uk/etl/docs/ETLreport1.pdf

3. Oxford Centre for Staff and Learning Development 1.How is the term blended learning being used in higher education? 2.What are the underlying rationales being used for promoting blended learning? 3.What monitoring and evaluation strategies are being adopted for ensuring and enhancing the quality of blended learning? 4.What impact is blended learning having on the student experience? 5.What are the success factors for blended learning? HEA review of undergraduate experiences of blended e-learning

4. Oxford Centre for Staff and Learning Development HEA review methodology field work (interviews, document analysis) using the teams connections with practitioners and managers – to give access to private/grey literature highlight the small number of transformative and thorough studies, making visible the existing research, policy and practice which is transforming the student experience. review is based on real world practice of institutions and courses and actual world of students. review will bring together the literature on student experience and institutional good practice to make recommendations for practitioners and managers.

5. Oxford Centre for Staff and Learning Development Two basic streams Institutional good practice Student experience strategies/ rationales QA processes support staff dev technologies pedagogies evaluationsholistic student exp learning designs student attributes

6. Oxford Centre for Staff and Learning Development Institutional visits Institution Attribute 1Attribute 2Attribute 3 Blackwater Regional/local focus Related to flexibilityEnhancing learning Deepshire Old U/Research intensive Enhancing learningRegional/local focus Eastonhall Regional/local focus Old U/Research intensive Widening participation Kilderhill Enhancing learning Regional/local focusRelated to flexibility Longside Prominence in sector Enhancing learningRegional/local focus Metroville Widening participation CAARegional/local focus Westington Widening participation Regional/local focus Prominence in sector

7. Oxford Centre for Staff and Learning Development How is the term blended learning being used? 1.Wide scale use of virtual learning environments to provide supplementary course resources 2.Radical, transformative course (re)designs to improve learning 3.A holistic view of technology, including use of learners own technologies to support learning

8. Oxford Centre for Staff and Learning Development What are the underlying rationales being used for promoting blended learning? Institutional rationales were highly contextualised and specific to each institution: Flexibility of provision Supporting diversity Enhancing the campus experience Operating in a global context Efficiency

9. Oxford Centre for Staff and Learning Development What are the underlying rationales being used for promoting blended learning? Course level rationales mostly in response to practical challenges: Loss of staff-student contact Engaging students in out of class activities Large classes Inconsistency in quality and quantity of feedback Developing professional skills

10. Oxford Centre for Staff and Learning Development What monitoring and evaluation strategies are being used? The institutional VLE survey Range from an item in the National Student Survey like did you find the VLE useful to your studies? to (less frequently) large, representative student (and staff) surveys involving questionnaires and focus groups that gather data on usage patterns and satisfaction ratings and discriminate between user groups (e.g., department, year, gender, etc.)

11. Oxford Centre for Staff and Learning Development Institutional surveys of VLE use find: Weve never done any surveys, ever, that have given other than the students want more of it, wider and deeper (Longside 2) Students value flexible access to course resources: The one stop shop Access to lecture notes Support for students with disabilities Maintaining a connection with the institution Students are concerned about Inconsistency in use between modules Time and expense associated with downloading and printing

12. Oxford Centre for Staff and Learning Development Some problems with surveys Do not reveal how or with what technology helps students to learn Do not differentiate between students with diverse backgrounds and needs Implication of withdrawal of service Lack of triangulation of data

13. Oxford Centre for Staff and Learning Development Six of the seven institutions explicitly identified institutional level evaluation of blended learning as problematic, e.g.: One of the things thats been seriously lacking over the last five years at least, has been a lack of interest in evaluation, proper evaluation, not lip service evaluation like the student satisfaction surveys (Blackwater 2). We don't have any systematic, institution- wide, sufficiently detailed research into the student experience in my view (Longside 1).

14. Oxford Centre for Staff and Learning Development What monitoring and evaluation strategies are being used? The institutional VLE survey Business audits of VLE systems Systematic module level feedback Case studies linked to project funding

15. Oxford Centre for Staff and Learning Development Business audits of VLE systems E.g. KPMG auditors visit for a few days, produce a report I think it is that notion of mission critical. People now regard it in terms of things like the risk assessment…. The VLE is now well up the list, because of the amount of stick we get if things fall over (Eastonhall 1) Know very little about this, including whether and/or what impact they have on blended learning development

16. Oxford Centre for Staff and Learning Development What monitoring and evaluation strategies are being used? The institutional VLE survey Business audits of VLE systems Systematic module level feedback Case studies linked to project funding

17. Oxford Centre for Staff and Learning Development Systematic module level feedback Existing module feedback items need revising: We just thought … well just use our ordinary module evaluation. Well it wasnt going to work was it? Its not asking the right sort of questions. (Deepshire 1) Systematic implies Standardised module evaluation forms that include items on VLE use Central analysis Synthesis and reporting to departments, who incorporate findings into action plans

18. Oxford Centre for Staff and Learning Development What monitoring and evaluation strategies are being used? The institutional VLE survey Business audits of VLE systems Systematic module level feedback Case studies linked to project funding

19. Oxford Centre for Staff and Learning Development Case studies The dominant method Relies for effectiveness on variety of well-established internal dissemination avenues, including informal practitioner networks Social repositories Issues of accessibility to practitioners Necessity for incentives to create and disseminate E.g. Teaching Fellows, Ed Development unit, internal and external project funding, internal T&L conference, PGCert, institutional research strategy Such elements are a package at Longside

20. Oxford Centre for Staff and Learning Development Mixed methods (triangulation) Many evaluations cannot report improved student outcomes despite intentions to do so because of flawed methodology (Alexander 1999) Mixed methods help evaluators learn not only what works (or doesnt) but how and why it does (or doesnt) E.g. Boyle & Nicol (2003) evaluated introduction of PRS and used 5 focus groups Critical incident questionnaire 365 item Likert scale survey Staff focus group

21. Oxford Centre for Staff and Learning Development Recommendations Less frequent surveys aligned to institutional strategic planning Purposive sampling Mixed method data gathering Audits of pedagogical approaches Meaningful data about the impact on the student experience Institutional systems for the dissemination of good practice

22. Oxford Centre for Staff and Learning Development Critical success factors There may be advantages to using a poorly defined term in institutional change Institutional rationales which are contextualised and specific Course designs or redesigns undertaken as a team, developed iteratively over a number of years in response to student feedback Implementations which tackle real and relevant problems at the course level Students understanding their own learning and the role of resources and technology in learning

23. Oxford Centre for Staff and Learning Development References Alexander, S. (1999) An evaluation of innovative projects involving communication and information technology in higher education. Higher Education Research & Development. 18 (2), 173-183 Boyle, J. T. & Nicol, D. J. (2003) Using classroom communication systems to support interaction and discussion in large class settings. ALT-J Research in Learning Technology. 11(3), 43-57 Entwistle, N., McCune, V. and Hounsell, J. (2002). 'Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings.' online http://www.ed.ac.uk/etl/docs/ETLreport1.pdf http://www.ed.ac.uk/etl/docs/ETLreport1.pdf