Embedding eLearning Critical success factors for institutional change (CSFIC) ECDL 2006 Sarah McNicol Evidence Base UCE Birmingham

Aims of the evaluation and support study: Support the four UK and US projects to help develop and share evaluation methodologies Synthesise key achievements and lessons learned from the projects Explore with the projects how the forms of learning developed can best be embedded and sustained within the organisational structure by devising metrics to measure key impacts.

Evaluation activities in depth interviews with project staff interviews with other departmental staff interviews with staff in other departments (e.g. IT support staff, learning support staff); interviews with senior institutional managers informal discussions observation or demonstrations of items developed through the DLIC programme question and answer sessions discussions with students

Success factors identified 1.The composition of curriculum planning, development and delivery teams 2.The development process (how the teams worked) 3.Institutional and department support (how the teams were embedded)

1. Curriculum planning, development & delivery teams Joint input from learning technologists and academics Learning technology and lecturing staff work closely together …each one of [the team members] has to have an understanding of all the elements…I had to understand totally the pedagogy behind what was happening. I had to be prepared to teach and be a teacher…they [other team members] had to have an understanding of the information side and support and the literacy skills and the tools. So you’ve got a very different model because they people traditionally have their own little area and they’re very safe and secure in their area…they’re the expert in the area. What happened in this model required people to blur the boundaries…

Learning technologists who have a basic understanding of the subject area, and of pedagogy [The learning technologist] is exceptionally good, not just technically, but in grasping the kind of pedagogical and intellectual basis of what we’ve been trying to do. Without someone of that calibre on board who can translate those ideas into technical digital tools, you’re lost…so the technical ability and the ability to appreciate where we’re coming from in terms of ideas is crucial…we couldn’t have done it otherwise.

A member of the team with both a technology and pedagogy background who can ‘interpret’ between academics and IT specialists I’ve acted as a sort of translator, if you like, between the technical staff and [project leader] because I understand…I’ve worked with technical staff…[and I also understand] the pedagogical reason for what we’re doing…I’ve been able to answer questions from both ends and explain to each end what the other person means, and I’ve quite enjoyed doing that. It’s not something that I’ve worked on before, so it’s been an interesting challenge and a new thing to focus on. If you take somebody like xxx, you know, he’s a geographer, he’s also a learning technologist. He’s got a background in both areas. So they can talk to the geographers as a geographer and that makes a huge difference, but they can also talk to the technical people…So you’ve got this kind of bridging role…

The involvement of individuals who really want to examine existing practices and introduce new ideas I mean, it’s been very, very rewarding…I’ve been lucky because I’ve been working with academics who have a real enthusiasm for teaching and a real desire to try new ways of teaching and an appreciation that the standard way of, you know, standing up and blurting out what you know to a class is not really enough. I was very happy and excited to take the opportunity of the project to try and jazz my own teaching up a little bit, I guess was my initial motivation…as soon as the whole idea of e- learning and digital libraries was presented to me, I saw quite a lot of potential…to embed into my teaching practices and help me out, so I was very interested and excited by the project from the word go.

…it’s been so fantastic this early in my career to have this opportunity to be somewhat involved in the whole process of it [e-learning]…I think it has improved my teaching quite a lot…I think in order to deliver the materials we’ve delivered, I think you have to do a lot of thinking about teaching and a lot of thinking about how to get the most out of the resources that you have and get the most out of the students that you have…at the start I don’t think I saw that; I can see it much more clearly now. …been very interesting and an education for me personally to see how these people think in different ways…how they manage their own work and how they manage their own topic areas…I definitely think that’s been a positive aspect of the project.

Opportunities for individuals to play to their strengths (i.e. a flexible approach to roles) Student input

2. The development process Tools/resources which complement teaching, rather than interfering with the core activity Taking an iterative approach to the development of tools and resources Occasionally someone has designed something and some one else will say, ‘That’s not quite how I imagined it’, but actually in this learning environment, that’s exactly what happens. Until you do it and show it to someone then they’re not capable of saying, ‘Well, that’s not quite what I had in mind’.

Communication and discussion (within departments, within teams, between project partners and with the external world)..I’ve never actually sat down with a group of people who are geographers to think about geography teaching and I could certainly see that as being one of the benefits… …it forced us all to sit down at the table and discuss things, to think about things that we never maybe in the past paid as much attention to… …this project made them [academic and support staff] talk to each other, made them try to understand each other…I think that’s an important outcome.

Sufficient time to embed projects within the curriculum and working of the department …it took a year and a half to get the team to develop a common language and understanding of the issues, and if it had been a year, a two-year, project, we wouldn’t have been able to have really progressed…because of the nature of the project…it raises questions for JISC, I think, about the length of project…

An element of external funding …you could do it, but it would be much, much harder for a pool of interested people to keep creating new content and collaboration if you had no funding for it. …critical mass of success in the front part of the project…the snowball is not a bad analogy because then, you know, it becomes so large, it just keeps rolling. Whereas I suspect if this was a relatively small resourced project with, you know, one or two items here and there, it doesn’t really fundamentally change the way in which a significant number of people work…

3. Institutional and departmental support Support and active involvement from senior staff in the department A high level of interest and involvement in e-learning in general within the department You have to get a project that you really want and you really want to do because whatever the money is, you’re going to spend three times as much time on it. So, you can’t afford to waste time on things you don’t want to do. Therefore, you have to make sure the project connects with things you’re interested in; that in your institution is interested in, that aligns with its strategy and you have to make connections between projects.

I think Leeds geography has been quite technology driven for some time…I think there’s a higher bar of IT-savviness when it comes to trying new things. In different departments there’s going to be a different culture of trying new methods of teaching, especially if the teaching is involving online methods, mixed methods… DialogPlus and the other [e-learning] projects are consistent with what we do…that’s why we applied for them, that’s why we’ve gone through it…what we try and do, is if it’s an area that we want to continue to be in, and obviously development of e- learning generally is an area we want to be in because we can see all the benefits of it, then we’ll continue seeking external support for that. There is the fact that progression from things we’d been doing anyway… the video project with Clyde Virtual University…So we’d done quite a lot of experiments with interactive teaching and trying to access distant resources in our teaching programme…

Leads, or champions, in academic departments We’ve always tried to identify an academic champion who will come from the academic community and, hopefully, we’ll be able to say, you know, ‘I did this, it worked for us, you think about it’, and I think that’s been, for this university, a very successful approach

Support at a senior level within the institution If you start off with a university which doesn’t really want these things in the classroom, then as soon as the funding finishes, they’re going to finish. Clearly, we don’t embark on these projects if we don’t think we have methods of sustaining them after the external funding has expired … we are acutely aware that, you know, it’s not just a case of having a project for two or three years and you know, that’s it…We have to sustain it.

Embedding e-learning Senior level support A coherent strategy A multi-disciplinary/multi-professional team Effective communication Time Some external funding Enthusiasm

Sarah McNicol Evidence Base Research and Evaluation UCE Birmingham