1. The Academy Research Observatory – Surveying the Landscape & update on developments Martin Oliver London Knowledge Lab & Higher Education Academy

2. Emails - m.oliver@ioe.ac.ukm.oliver@ioe.ac.uk -martin.oliver@heacademy.ac.ukmartin.oliver@heacademy.ac.uk -observatory@heacademy.ac.ukobservatory@heacademy.ac.uk More information at: http://www.heacademy.ac.uk/ourwork/research/observatory Piloting site: http://academy-research-observatory.pbwiki.com/

3. The Observatory A service promoting and exploring the use of practice- and research-based evidence to influence policy and practice in teaching and learning in Higher Education Tools to enable access to evidence and syntheses of this evidence (e.g. prototype repository and syndicated search, wiki) Spaces (real and virtual) to help communities explore evidence-based practice and its implications for students' learning

4. A work in progress Initial proposal to HEFCE for an e-Learning Research Observatory Landscaping study https://mw.brookes.ac.uk/display/hearoc Judged to have wider relevance A research observatory for Higher Education e-Learning, Widening Participation, Employer Engagement Other strands may be added - although the strands may not be explicit represented in the final structure

5. Overview of development August 07 – July 08 Scoping the Observatory Landscaping report for e-learning (exemplar area) Generation of pilot resources and services Proof of concept piloted at Academy conference July 08 – July 09 Development phase Pilots focusing on community engagement Wider consultation Promoted at Academy conference July 09 Phase 0 Phase 1 Phase 2, 3…

6. Landscaping evidence use in e-learning Series of exploratory studies within e-learning Beetham, Sharpe & Benfield Landscaping consultation Interviews with key informants Survey (116 responses) Follow-up interviews https://mw.brookes.ac.uk/display/hearoc/

7. Yes, but… Keen on single point of access; research reviewed, evaluated and synthesised Questions – e.g. how does this relate to other Observatories…? After that, great variability The impossibility of categorising respondents as users, producers, policy makers or intermediaries for research is in itself is an important outcome.

8. HB: What kind of research or evidence should a research observatory focus on? I: Evidence that e-learning really works. HB: What would that evidence look like? I: It would need to show real improvements to learning outcomes, across a large number of students. It would have to have credibility and rigour. HB: Can you think of an example of research evidence of that kind? I: Not off the top of my head, no. HB: So does this research really exist?

9. I: No, the observatory would have to fund it. OR Yes, it is out there, the observatory will have to work really hard to find it. HB: Can you think of a situation when evidence like this has really changed peoples practice or understanding, in your experience? I: Well, the cynics always ask for evidence that e-learning really works. HB: Do you think that evidence, if you had it, would lead them to change their minds? I: No, they would find arguments against it from their own discipline perspective.

10. HB: So what about people who are actually open to change? I: They never ask for evidence. They ask for examples, especially from their own subject area, and practical ideas. They are really responsive to other people in their discipline who have tried something and made it work.

11. Identified strategic choices, approaches and risks Strong persuader or neutral observer? (Setting agendas) Funder? Push or pull communications? Quality assured, selective research or evidence and examples? Audience: researchers, intermediaries, practitioners or policy makers? Building authority or democratic knowledge? (Who can write?) Central or local knowledge management? Face-to-face or technology supported networks?

12. Building from this: choices and risks Internal discussions about priorities, style, focus Something that has to be revisited. Regularly. Oakleigh evaluation report – the need for engagement with the community; perceived as remote, authoritarian Personal interests in people and how they make sense of technology in their work Risk assessment Scoped scenarios; risk document based on report Focus for discussion within project team meetings However… Impending launch at Academy conference – determined initial priorities

13. Pilot site for preliminary consultation

14. Building from this: e-Learning pilot How do communities produce, share and use evidence? QA/QE in e-learning SIG Expert review about e-portfolios Discussion about professional role (M25 Learning Technologists group) Open peer commentary on the national development programmes Review processes, to identify approaches that may have wider value Feed these back to inform the development of the whole observatory

15. Overview of lessons: Resources provided for people were not necessarily (ever?) taken up Communities rise and fall Expert review uninviting to others (useful, but not an invitation to contribute) Open peer commentary engaged invitees then stopped

16. Building from this pt2: wider consultation Inviting contributions from wider groups Employee learning and widening participation communities TLRP conference workshops ELESIG meeting (Today!) Case studies with communities (Heads of e-Learning Forum) Input into specifying the Observatory; documented cases of evidence generation and use Phases – question of whether this is Phase 1, 2 or 3…

17. Building from this pt3: e-Learning projects Small grants for research Projects awarded, visited once, conclude by generating reports New model Projects awarded, brought together, given technical infrastructure (wiki, Ning) Will be visited, encouraged to use Web2.0, brought together mid-project and again at end

18. Wiki for Academy-funded projects

19. And what next? New round of projects (provisionally) Smaller-scale reviews Special Interest Groups Possibly projects about technology to support evidence- informed practice Emphasis on engagement (I hope) Wiki-based reviews for open engagement Social networks – finding people, not just research outputs

20. However… Continued visibility of central resource Yes, its a repository… but its not just a repository If its just a website itll be pointless Invitation to engage does not guarantee engagement

21. So… The hidden agenda for this meeting: ELESIG as a community of research/evidence producers (and potentially, users too…) An opportunity to support and encourage evidence use Documenting the process Identification of some outcomes Available via the RO wiki (so you can edit it til youre happy!)

22. What now? Working from your experiences to influencing policy and practice Morning: generating messages from current practice Afternoon: working with key messages Task: Claims youd like to make based on what youre currently up to

23. The format In tables… 1. Chat about what youve been up to 2. Jot down claims youd like to make 3. Note, under each, evidence to support them Then… 4. Swap tables! Critique! 5. Return and repair to make claims robust