E-XCELLENCE Benchmarking quality in European HE e-learning EATING Seminar Feb 19 th 2009

Added value of e-learning QA in HE is receiving much attention at institutional, national and European level. Still missing: QA specific to e-learning -criteria based on ease of access -new forms of interaction (students and staff) -flexibility, personalisation and other pedagogical aspects that are now deliverable by e-learning These express the added value of e-learning in HE.

E-xcellence: Benchmarking Quality in e-learning Main Contributors: EADTU OULU-University (Finland) OUNL (Netherlands) OUUK (United Kingdom) with support from: CNED (France) UNED (Spain) UOC (Spain) EITSA (Estonia) NETTUNO (Italy) APERTUS (Hungary) + NVAO (Netherlands/Belgium) European University Association (EUA)

General objective QA in e-learning that: Optimises the learning process and offering assurance to stakeholders that e-learning provision is of high quality. Is complementary to existing national quality assurance systems related more to content, staff and infrastructure. Our main aim was to establish: a framework of quality criteria for the development, operation and evaluation of e-learning programmes an appropriate set of performance indicators, parameters and guidelines by which the quality of e- learning programmes can be measured by assessment a definition of excellence

APPROACH Analysis of existing quality processes of partners and/or requirements of national quality organisations Development of a suite of criteria relevant to e-learning (using EADTU members’ experience) Stakeholder consultation procedure Translation of these parameters into a flexible quality assessment tool based on benchmarks Testing in 3 pilots the benchmarks, guidance manual and approach: Univ Oulu Finland EITSA Estonia UNINETTUNO Italy

Results Set of (33) European benchmarks These cover institutional, pedagogical, technical, ethical and management aspects of e-learning organised into the three main categories of :  Management (institutional and programme levels)  Products (curriculum, course design and delivery)  Services (student and staff support) An important aspect of E-xcellence is that it offers a European-wide set of benchmarks, independent of particular institutional or national systems, and with guidance for educational improvement

Structure of Resources Benchmark Statements –Define desired features Indicators –Source of evidence Indicators of excellence –Exemplars of evidence Assessors Notes –Provide background guidance for assessors

The E-xcellence manual Mindmap QUALITY manual as backbone of web-tool

Web:

Modes of use Informal Self Assessment via Quickscan Identify hot spots (and cold spots) Internal Quality Assurance/Improvement embed selected benchmarks in internal process adopt and formalise indicators External Quality Assurance - informal embed use of benchmarks in Self Assessment Adapt to meet QA agency requirements External Quality Assurance - formal external QA agencies embed benchmarks within their processes

Quickscan Simplified version of the full e-xcellence assessment tool Quick self-assessment of your institution’s e-learning performance Identifies areas in your e-learning programme/course that need more attention Score yourself on the most relevant aspects Results will show after completing quick-scan. These help you: To identify elements to be improved To guide the internal discussion To learn if a full quality assessment procedure is useful

Full e-learning assessment Evaluate the institution/program/course using the E-xcellence method and tool. Manual; Benchmark statements, in relation to criteria and indicators. Questionnaire: comments on the specific issue: in the box you can refer to documents or other references which can be used as evidence on that specific aspect of e-learning. The on-line questionnaire needs to be filled out by different disciplines in your organisation coming from management, course designers, tutors and students On-site visit of e-learning expert

Full assessment REVIEW TOOLAGAINSTINSTITUTIONALSTRUCTURE+ PROCESS PREPARESELFASSESSMENTUSING THETOOL SITE VISIT- SELF ASSESSMENT ANALYSIS- INTERVIEWS- CONCLUSIONSACTIONPLANNING Support: manual, proposed agenda, assessors’ notes, glossary.

Pilot Experience 3 Internal Quality Assurance reviews with E-xcellence team members participating Some difficulties with language and interpretation of benchmarks Suggestions for additional factors and regrouping More needed on the process of the exercise More on best practice

Pilot Experience Conclusions an intensive and productive exercise relevant information for the project: materials, process, procedure relevant for the institutions relevant for the piloting teams work in progress base document/material for different products

E-xcellence + EU funding for dissemination and implementation Strengthened links with ENQA Additional project partners

E-xcellence + Partners Core Contributors: EADTU OULU-University (Finland) OUNL (Netherlands) OUUK (United Kingdom) with support from: CADUV + Uni Hradec Kralove ( CZ Rep) UNED (Spain) EITSA (Estonia) NETTUNO (Italy) Kath Univ Leuven ( Belgium) APERTUS (Hungary) NVAO (Netherlands/Belgium) NHSU + Univ Lund ( Sweden) Slovak Univ Technology ( Slovakia) MESI ( Russia) IFEL ( Switzerland)

E-xcellence + Activities European Seminar May 2008 Madrid –Jointly organised with ENQA –~ 50 participants National Seminars –One in each partner country –Participation of institutions and Accreditation agencies/ministries –Trialling of Quickscan and Full Assessment

E-xcellence + Activities Quickscan trials –Use of Quickscan by each participating institution Full Assessment trial –Two online PG programmes at Univ Lund in association with NHSU –OUNL in association with NVAO

E-xcellence + Activities Current outcomes: –Quickscan analysis at Univ Hradec Kralove –Internal staff training programme in use of Web CT –Identified discrepancies between departments in implementation of central policy –Interesting exemplars of learner support provision and staff reward structures

E-xcellence + Sustainability E-xcellence Associates label –Commitment to continuous improvement in e- learning activity –Use of E-xcellence tools to identify and review improvement Virtual Benchmarking community –Peer review by community members –Input to update and revision of E-xcellence tools

Web: Thank you for your attention