1. Clare Madge - Department of Geography Henrietta O’Connor - Centre for Labour Market Studies University of Leicester, University Road, Leicester, LE1 7RH, UK

2. Structure of presentation 1.Background to project 2.The project process 3.Evaluation 4.Problems and prospects 5.Tour of the training resource

3. 1. Project background Programme: Research Methods Programme (Phase 2) Period of project: 01.04.04 to 31.07.2006 Total amount of award from ESRC: Prior to indexation: £94,242.26; After indexation: £100,360.59 (1st round of funding) The project team: Clare Madge (Geog) Henrietta O Connor (CLMS), Jane Wellens (Educational Developer), Rob Shaw (Learning Technologist), Tristram Hooley (PG Training Coordinator), Julia Meek (Evaluation Consultant)

4. Project aims 1.To produce and evaluate a high-quality online portal to provide training in online research methods; 2.To act as a self-supporting online resource to enhance understanding of both theoretical and practical aspects of online research methods including web-based questionnaires, virtual synchronous and asynchronous interviews; 3.To draw on a wide range of successful good practice case studies, cover associated ethical issues of online research, provide important resource links and technical guidance.

5. Achieved…? http://www.geog.le.ac.uk/orm/index.htm

6. 2. Project process The process or methodology made apparent; The social production of this technical artefact made transparent; A reflexive process to team working; A variety of FTF and virtual means of group communication used; http://www.geog.le.ac.uk/orm/background/projb ackground.htmhttp://www.geog.le.ac.uk/orm/background/projb ackground.htm

7. 3. Evaluation: Problems with on-line resources Expensive; Rarely used; Difficult to use; Inflexible – rarely meet our exact needs; Frequently out of date; Anxiety about the reliability of the information; Frequently poorly referenced.

8. Addressing These Problems Through Evaluation Graphic design/lookGroup feedback Usability feedback UsabilityHeuristic (expert) evaluation Watching sample users (cognitive walkthrough) FlexibilityCase studies and classroom evaluation Academic contentAcademic peer review

9. Development and Evaluation Project idea Project group evaluation Expert evaluation Usability studies User feedback Design Classroom testing Academic Peer Review Complete Project

10. The evaluation process http://www.geog.le.ac.uk/orm/background/eval2.h tm

11. Designing Evaluation Ask the experts – speak to an evaluation consultant; Evaluate early and often; Use a range of evaluation techniques; Try to evaluate with a range of different users who are drawn from your target audiences; Act on your evaluation – redraft, redesign and re-evaluate.

12. 4. Problems and prospects Overall achieved aims. Quality. Peer evaluation; Team- working, diversity, full-time RA; Great response from research community (case studies, evaluations, links, training etc); Importance of on-going evaluation; Commitment to longevity, good investment; Build-up dissemination networks; Foundation for training in ORM.

13. But…. Labour intensive- new skills required (writing/training/proof-reading); Site expansion (other methods/interactive tool) RAE submission (is training a research output?); Publication/copyright/citation http://www.geog.le.ac.uk/orm/site/home.htm Staff retention (temporary RA); Future maintenance and updating; Technological advances; Expanding training (RDI).