Effecting institutional change through the evaluation of e-learning Richard Walker & Rose Papworth E-Learning Development Team, University of York eLearning in HE: A Symposium for FE/HE Strategists & Practitioners 26th February 2008

Key Questions  What are the drivers for e-learning in your institution?  How is e-learning activity managed?  What role does evaluation play in: –informing academic practice on e-learning course design and delivery? –shaping organisational learning on e-learning best practices?

Starting late  Late, phased roll-out of VLE –1 st round ca. 20 projects –2 nd round ca. 40 projects –3 rd round 50 projects Project Timeline

Drivers for e-learning  Enhancement of student learning experience through blended learning  Promotion of active student learning –skills development, personalised learning pathways, collaboration  Transformation of academic practice –student-focused course design –sharing & collaborative development of learning and teaching resources

Strategy  Institutional e- learning policy  Centrally managed platform  Close control over pilot projects  Departments develop local strategies (mature/developing/pilot)  Delegated training, admin and quality assurance  Departmental champions oversee long-term development in collaboration with central services Top Down Bottom Up

Pilot project cycle Training (Getting Started, Library Accessibility, Roll- Out + specialist workshops) Application procedure (project scope; aims & objectives; support; delivery & funding requirements; consultation with central services) Evaluation (case study research) Project review & dissemination of results (feedback to course team; dissemination via lunchtime 'show & tell' meetings, internal conference, departmental teaching committees) Delivery (staff / student support)

Rationale for case study research evaluation  Interpretive research agenda: focusing on student perceptions of their learning experience  Experiences of actors are important, the context of action is critical  Holistic & multi-dimensional picture of a real life context “to understand phenomena through accessing the meanings participants assign to them” (Orlikowski & Baroudi, 1991)

Key research questions  What are the students’ affective and attitudinal responses to the blended course experience?  How are the e-learning tools used by students to support their learning in formal/informal study activities?  How did the lecturer/tutor perceive students’ learning relative to previous performance? (What actions to be taken for future course development?)

Data collection methods (Informal progress checks) Entry survey Staff interview Focus group interviews Exit survey Activity logs

Case study template  Case Study Overview –Background –Description of approach –Learning activities & tools –Student profile –Experience with computers –Experience with computers for learning –Expectations towards the VLE –Outcomes of the pilot –Activity statistics –Focus group feedback –Exit survey feedback –Instructor and tutors’ feedback –Student skills required & developed –Staff skills required & developed –Actions for further development

Communicating the research  Target an audience of practitioners: dissemination and application a priority, reinforced in f2f gatherings  Are presented in a ‘ whole course ’ context with course artefacts such as module site and resources available to explore  Inform training activities, informally and formally in a virtuous cycle The case studies:

Impact on organisational learning  Case study research helps us to identify generalisable features of learning experience (adoption trends; characteristics of the student population)  Lessons learned contribute to evolving framework of instructional responsibilities (course design & delivery methods)  Outcomes inform departmental strategy & support activities  Outcomes inform staff training and support services and influence institutional policy (e.g. accessibility training)

DISCUSSION  What are the drivers for e-learning in your institution?  How is e-learning activity managed?  What role does evaluation play in: –informing academic practice on e-learning course design and delivery? –shaping organisational learning on e-learning best practices?

End