1. Next Steps for Excellence in the Quality of e-Learning
Jon Rosewell, Karen Kear, Keith Williams
Dept of Communications and Systems, Faculty of Maths Computing and Technology, The Open University, UK
CALRG Conference, 19-20th June 2012

2. QA for e-learning What is the problem?
There are established HE QA procedures in Europe
These were designed for conventional universities
They don’t necessarily fit e-learning
Solution:
Provide resources and processes for QA of e-learning
These can be adapted for local/national purposes

3. E-xcellence project 2005–present
Funded by EU Lifelong Learning programme
Managed by EADTU
E-xcellence
Development and trialling of criteria, handbooks and methodology
E-xcellence plus
Dissemination to institutions and to QA agencies in 9 European countries
E-xcellence NEXT
Continuing dissemination and updating of criteria and resources

4. E-xcellence: structure and resources
Strategic Management
Curriculum Design
Course Design
Course Delivery
Student Support
Staff Support
Resources
33 Benchmark statements
Manual (includes indicators)
Assessors’ notes
Online Quickscan tool
Framework for action roadmap
See:

7. Quickscan Simplified version of the full E-xcellence assessment tool
Quick self-assessment of e-learning performance
Rate programme/course on the most relevant aspects
Identifies hot / cold spots of e-learning programme/course
Online version provides feedback:
To identify elements to be improved
To guide the internal discussion
To learn if a full quality assessment procedure is useful

8. E-xcellence: modes of use
Informal self-evaluation
Use Quickscan
Local seminar
Local use of Quickscan with justification for rating
Meetings with project team and local QA Agency
Improvement roadmap
Full assessment
As above but part of formal accreditation
Evidence provided for benchmarks

9. Local seminars – purpose
To discuss with HE institutions the quality of e-learning on the basis of the benchmarks
To explore with QA agencies how to incorporate e-learning into their frameworks
To exchange ideas during an on-site visit
To improve process:
Exchange experience on the E-xcellence framework and the Quickscan
Collect feedback on tools

10. Local seminars – format
Preparation
Participants: managers, staff members, course designers, tutors, students
Decide programme to be assessed
Select some or all benchmarks
Team meets to complete QuickScan self-evaluation
Seminar
First day: local team meet with assessors
Second day: local team, assessors and national QA agency
Report
From assessors
Roadmap for improvement from institution

11. E-xcellence NEXT: updating
General updating of manual
Clarifying language / terminology
Deal with emerging trends
Convergence between distance and F2F  blended modes
Social networking in HE
Use of Open Educational Resources
Process:
Quickscan comments from partners
Participatory workshops
Feedback from local seminars

12. E-xcellence NEXT: social networking
How might social networking contribute to high quality in e-learning?
What risks to quality might arise?
Which of the existing E-xcellence quality benchmarks might apply in this context?
Are any new benchmarks needed to cover this scenario?

13. What do we mean by ‘social networking’?
‘Social networking’ can be interpreted broadly to cover a range of online communication processes
e.g. via forums, blogs, wikis
It can also be interpreted more narrowly to focus on social network sites that provide accessible tools
e.g. Facebook, Twitter, LinkedIn
Note categories of users 13

14. Why use social networking?
Social networking has two primary purposes in education:
facilitating learning
social learning theories
focused pedagogic function such as group work, peer assessment
building communities
motivation and progress
informal and social

15. Social networking tools
Forums discussion and debate
Wikis co-creation of resources
Blogs reflection, sharing and feedback
Social network sites sense of community
Public (Facebook etc) or walled-garden (VLE)?
Boundaries and invasion of student space?

16. Social network sites Benefits many students already use them regularly
seen as more social, informal and flexible
Challenges
privacy issues
lack of control
blurring of boundaries between social and academic life.

17. Revised benchmarks – social networking
Curricula are designed to enable participation in academic communities via online social networking tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities.

18. E-xcellence NEXT: OERs
How might OERs contribute to high quality in e-learning?
What risks to quality might arise?
Which of the existing E-xcellence quality benchmarks might apply in this context?
Are any new benchmarks needed to cover this scenario?

19. OER use-cases
Individual life-long learner finding material for own use
Individual teacher obtains assets and uses in own material
Course uses podcasts from iTunes U
Course uses a 10-hour unit
Entire 100-hour module reused, with new assessment
Course and assignments in OER; tutorial / marking / accreditation offered for fee
Consortium develops material for own use and ‘frees’ it
Issue: OER use is very varied in scale – from single assets to whole courses – so QA procedures could be very different in different contexts

20. Quality points Provenance Reputation Brand OER repository checking use
creation
Is it possible to evaluate quality of components in isolation, or only in the context of their use?
Quality process
Checking
Peer review
Feedback
Rating / voting / recommendation
Branding / provenance / reputation
peer review
user recommendation 20

21. Quality Dimensions Pedagogic Effectiveness Learning objectives
Prerequisites
Learning design
Learning styles
Assessment
Content
Accuracy
Currency
Relevance
Ease of use
Clarity
Visual attractiveness, engaging
Clear navigation
Functional!
Reusability
Format
Localisation
Discoverability: metadata

22. Trends toward Open Educational Practice?
use  create
teacher centred  learner centred
transmission  constructivism
(sage on stage)  (guide on side)
focus on outcome  focus on process
standardised  personalised learning
individual  social/ peer learning
Capability maturity model:
Use OERs  Adapt OER material  Create OER material
See, for example, OPAL OEP Guide

23. Revised benchmarks – OERs
OER material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. OER materials are subject to the same review processes as other course materials.

24. Local seminars 2011-12 Russia MESI University, Moscow
Lithuania Kaunas University of Technology
Poland Akademia Górniczo-Hutnicza, Krakow
Cyprus Open University of Cyprus, Nicosia
Latvia Riga Technical University
Portugal Universidade Aberta, Lisbon
Greece Hellenic Open University, Patras

25. Local seminar feedback (1)
Framework
Quickscan is valuable to structure discussion
Completeness of the framework is appreciated
Team working
People exchange perspectives with other departments
External perspective
Exchange of experience between the evaluators and staff was valuable
New ideas surfaced for course design

26. Local seminar feedback (2)
Reflection
A valued ‘moment of reflection’ on quality
People become aware of choices and implementations
Gives insight into strengths and weaknesses
Analysis
Opportunity to formulate e-learning policy
Provides foundations for decision making

27. Updating E-xcellence resources
Work done:
Revised benchmarks and Quickscan
Manual
Edited for language, relevance to blended learning
Including social networking and OER
Work still to be completed:
Finalise benchmarks, Quickscan and manual
Update assessors’ notes

28. Comments and feedback? Web: http://www.eadtu.nl/e-xcellencelabel/
Thank you for your attention