1. + QUALITY ASSURANCE FOR E-LEARNING AN ESSENTIAL EXTENSION FOR LIFELONG LEARNING Oulu 19-20.5.2009 Keith Williams Leo Wagemans Jo Boon Pekka Kess European Association of Distance Teaching Universities

2. EADTU EADTU is Europe’s leading representative association for Lifelong Open and Flexible (LOF) learning in distance HE. Members are: 12 Open and distance teaching Universities 15 Consortia and Associations of conventional universities operating with e-learning and DE 3 Associate members Representing over 200 Universities and 2 million students.

3. Added value of e-learning QA in HE is receiving much attention at institutional, national and European level. Still missing: QA specific to e-learning -criteria based on ease of access -new forms of interaction (students and staff) -flexibility, personalisation and other pedagogical aspects that are more relevant to e-learning These express the added value of e-learning to HE.

4. E-xcellence: Benchmarking Quality in e-learning Main Contributors: EADTU OULU-University (Finland) OUNL (Netherlands) OUUK (United Kingdom) with support from: CNED (France) UNED (Spain) UOC (Spain) EITSA (Estonia) NETTUNO (Italy) APERTUS (Hungary) NVAO (Netherlands/Belgium) European University Association (EUA)

5. APPROACH Analysis of existing quality processes of partners and/or requirements of national quality organisations Making inventory of missing criteria (by 12 European e-learning experts) Stakeholders consultation procedure Translation of these parameters into a quality assessment tool based on benchmarks Testing in 3 pilots the benchmarks, the manual and approach: -the institutions benefited from the process and interactive discussions. -they received useful information in further developing the course/ programme. The exercise was relevant for the course but also for the university as a whole. -the process was however time-consuming and there is a need for improved instructions beforehand.

6. The E-xcellence manual Mindmap QUALITY manual as backbone of web-tool

7. Products to work with  manual –reference tool for the design and assessment of e- learning programmes –benchmarks, quality criteria and notes for guidance against which e-learning programmes and their support systems may be judged  assessors notes –provide a more detailed account of the issues and the approaches. Good practices for various situations. web-based instrument  Quick scan  Full assessment + http://www.eadtu.nl/e-xcellenceQS / E-xcellence Products

8. Quickscan Simplified version of the full e-xcellence assessment tool Quick self-assessment of your institution’s e-learning performance Identifies areas in your e-learning programme/course that need more attention Score yourself on the most relevant aspects Results will show after completing quick-scan. These help you: To identify elements to be improved To guide the internal discussion To learn if a full quality assessment procedure is useful http://www.eadtu.nl/e-xcellenceQS/

10. Full e-learning assessment Evaluate the institution/program/course using the E-xcellence method and tool. Manual; Benchmark statements, in relation to criteria and indicators. Questionnaire: comments on the specific issue: in the box you can refer to documents or other references which can be used as evidence on that specific aspect of e-learning. The on-line questionnaire needs to be filled out by different disciplines in your organisation coming from management, course designers, tutors and students On-site visit of e-learning expert

11. Full assessment REVIEW TOOLAGAINSTINSTITUTIONALSTRUCTURE+ PROCESS PREPARESELFASSESSMENTUSING THETOOL SITE VISIT- SELF ASSESSMENT ANALYSIS- INTERVIEWS- CONCLUSIONSACTIONPLANNING Support: manual, proposed agenda, assessors’ notes, glossary.

12. E-xcellence + Goal: From project to mainstream implementation of the E-xcellence instrument European wide at the local level. E-XCELLENCE+ brings together the expertise and experience of universities in lifelong learning from 13 countries as well as the expertise of quality assurance and accreditation processes from several QA agencies as a framework for educational improvement and innovation. E-XCELLENCE+ promotes the use of E-xcellence European wide and envisages increased performance and innovation in e- learning by integration of the instrument in the institutional and national policy frameworks. How to reach out and influence current bussiness models? +

13. Local introduction Steps of local introduction 1. Sensibilisation on including QA for e-learning 2. Information on the E-xcellence instrument 3. Organising cooperation universities and QA-agencies 4. Finetuning instrument and existing systems 5. Integration 6. Implementation

14. What to achieve in the local seminars Mapping experiences Determining the local impact (dialogue, shift of attention, roadmaps to improvement, etc) Finetuning Establishing a sustainable use of the instrument (internal + external QA) Receive feedback on the instrument

15. Local introduction 13 Local seminars OUNL (Heerlen) Oulu University (Oulu) OUUK (Milton Keynes) EITF (Tartu) Hungarian Virtual University Network (Budapest) Uninettuno (Rome) Lund University (Lund) Czech Association of the Distance Teaching Universities + University of Hradec Kr á lov é (Brno/ Hradec Kr á lov é ) UNED (Madrid) KU Leuven (Leuven) Slovak university of technology (Bratislava) MESI (Moscow) FernUni Schweiz (Bern) +

16. 2 full Assessments: -Open Universiteit Nederland -Lund university Universities External QA agencies E-learning experts

17. Yearly update of the instrument, available as open source Improving accessibility, interactiveness and flexibility for the LLL-student. E-xcellence Associates Label Promote continuous educational improvement cycles for LLL. 13 and more countries working with the E-xcellence instrument (institutional/QA agency) Mainstreaming by integration. Reaching out by networking ENQA-EADTU UNESCO E-xcellence contributing to LLL + 2009

18. Experience Rome Uninettuno A student questionnaire is developed based on E-xcellence framework to analyse the perception of students about e-learning with regard to: -integration and interaction of multimedia technologies -flexibility and adaptability -instruments of collaborative learning

19. Experience Finland ‘The evaluation with the E-xcellence instrument created a lively discussion inside the university. In all, the experience was very useful to most people participating internally in the evaluation: - new ideas were created to the course design - new foundations were found to justify decisions - experience exchange between the evaluators and staff was extremely valuable - new working methods were discovered’ (Pekka Kess: OULU University) Peer review with other universities

20. Remarks about the Manual, Tool and Audit process, Finland Using the tool: individually vs. team (from management to students) Good exercise for the institution and the members to go through the stages of the process of benchmarking; people become aware of choices and implementations. It brings the institution really to a formulation of the policy of e-learning. E-xcellence Project

21. Remarks about the Manual, Tool and Audit process, Italia Two kind of benchmarks: benchmarks which describe products and benchmarks which describe processes E-xcellence Project

22. Remarks about the Manual, Tool and Audit process, Estonia The tool is too general and too much focused on management level There must be an instruction for the use of the tool: a manual. E-xcellence Project

23. Remarks about the Manual, Tool and Audit process, Leuven Function of the QS: too much as an assessment tool; better to use it as an improvement tool Possible scenario’s to use QS Too much from the perspective of DL, less focus on blended learning. E-xcellence Project