Next Steps for Excellence in the Quality of e-Learning Jon Rosewell, Karen Kear, Keith Williams Dept of Communications and Systems, Faculty of Maths Computing and Technology, The Open University, UK CALRG Conference, 19-20th June 2012

QA for e-learning What is the problem? There are established HE QA procedures in Europe These were designed for conventional universities They don’t necessarily fit e-learning Solution: Provide resources and processes for QA of e-learning These can be adapted for local/national purposes

E-xcellence project 2005–present Funded by EU Lifelong Learning programme Managed by EADTU E-xcellence 2005-06 Development and trialling of criteria, handbooks and methodology E-xcellence plus 2008-09 Dissemination to institutions and to QA agencies in 9 European countries E-xcellence NEXT 2011-12 Continuing dissemination and updating of criteria and resources

E-xcellence: structure and resources Strategic Management Curriculum Design Course Design Course Delivery Student Support Staff Support Resources 33 Benchmark statements Manual (includes indicators) Assessors’ notes Online Quickscan tool Framework for action roadmap See: http://www.eadtu.nl/e-xcellencelabel/

Quickscan Simplified version of the full E-xcellence assessment tool Quick self-assessment of e-learning performance Rate programme/course on the most relevant aspects Identifies hot / cold spots of e-learning programme/course Online version provides feedback: To identify elements to be improved To guide the internal discussion To learn if a full quality assessment procedure is useful http://www.eadtu.nl/e-xcellenceQS/

E-xcellence: modes of use Informal self-evaluation Use Quickscan Local seminar Local use of Quickscan with justification for rating Meetings with project team and local QA Agency Improvement roadmap Full assessment As above but part of formal accreditation Evidence provided for benchmarks

Local seminars – purpose To discuss with HE institutions the quality of e-learning on the basis of the benchmarks To explore with QA agencies how to incorporate e-learning into their frameworks To exchange ideas during an on-site visit To improve process: Exchange experience on the E-xcellence framework and the Quickscan Collect feedback on tools

Local seminars – format Preparation Participants: managers, staff members, course designers, tutors, students Decide programme to be assessed Select some or all benchmarks Team meets to complete QuickScan self-evaluation Seminar First day: local team meet with assessors Second day: local team, assessors and national QA agency Report From assessors Roadmap for improvement from institution

E-xcellence NEXT: updating General updating of manual Clarifying language / terminology Deal with emerging trends Convergence between distance and F2F  blended modes Social networking in HE Use of Open Educational Resources Process: Quickscan comments from partners Participatory workshops Feedback from local seminars

E-xcellence NEXT: social networking How might social networking contribute to high quality in e-learning? What risks to quality might arise? Which of the existing E-xcellence quality benchmarks might apply in this context? Are any new benchmarks needed to cover this scenario?

What do we mean by ‘social networking’? ‘Social networking’ can be interpreted broadly to cover a range of online communication processes e.g. via forums, blogs, wikis It can also be interpreted more narrowly to focus on social network sites that provide accessible tools e.g. Facebook, Twitter, LinkedIn Note categories of users 13

Why use social networking? Social networking has two primary purposes in education: facilitating learning social learning theories focused pedagogic function such as group work, peer assessment building communities motivation and progress informal and social

Social networking tools Forums discussion and debate Wikis co-creation of resources Blogs reflection, sharing and feedback Social network sites sense of community Public (Facebook etc) or walled-garden (VLE)? Boundaries and invasion of student space?

Social network sites Benefits many students already use them regularly seen as more social, informal and flexible Challenges privacy issues lack of control blurring of boundaries between social and academic life.

Revised benchmarks – social networking Curricula are designed to enable participation in academic communities via online social networking tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities.

E-xcellence NEXT: OERs How might OERs contribute to high quality in e-learning? What risks to quality might arise? Which of the existing E-xcellence quality benchmarks might apply in this context? Are any new benchmarks needed to cover this scenario?

OER use-cases Individual life-long learner finding material for own use Individual teacher obtains assets and uses in own material Course uses podcasts from iTunes U Course uses a 10-hour unit Entire 100-hour module reused, with new assessment Course and assignments in OER; tutorial / marking / accreditation offered for fee Consortium develops material for own use and ‘frees’ it Issue: OER use is very varied in scale – from single assets to whole courses – so QA procedures could be very different in different contexts

Quality points Provenance Reputation Brand OER repository checking use creation Is it possible to evaluate quality of components in isolation, or only in the context of their use? Quality process Checking Peer review Feedback Rating / voting / recommendation Branding / provenance / reputation peer review user recommendation 20

Quality Dimensions Pedagogic Effectiveness Learning objectives Prerequisites Learning design Learning styles Assessment Content Accuracy Currency Relevance Ease of use Clarity Visual attractiveness, engaging Clear navigation Functional! Reusability Format Localisation Discoverability: metadata

Trends toward Open Educational Practice? use  create teacher centred  learner centred transmission  constructivism (sage on stage)  (guide on side) focus on outcome  focus on process standardised  personalised learning individual  social/ peer learning Capability maturity model: Use OERs  Adapt OER material  Create OER material See, for example, OPAL OEP Guide http://opal.innovationpros.net/publications/guide/

Revised benchmarks – OERs OER material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. OER materials are subject to the same review processes as other course materials.

Local seminars 2011-12 Russia MESI University, Moscow Lithuania Kaunas University of Technology Poland Akademia Górniczo-Hutnicza, Krakow Cyprus Open University of Cyprus, Nicosia Latvia Riga Technical University Portugal Universidade Aberta, Lisbon Greece Hellenic Open University, Patras

Local seminar feedback (1) Framework Quickscan is valuable to structure discussion Completeness of the framework is appreciated Team working People exchange perspectives with other departments External perspective Exchange of experience between the evaluators and staff was valuable New ideas surfaced for course design

Local seminar feedback (2) Reflection A valued ‘moment of reflection’ on quality People become aware of choices and implementations Gives insight into strengths and weaknesses Analysis Opportunity to formulate e-learning policy Provides foundations for decision making

Updating E-xcellence resources Work done: Revised benchmarks and Quickscan Manual Edited for language, relevance to blended learning Including social networking and OER Work still to be completed: Finalise benchmarks, Quickscan and manual Update assessors’ notes

Comments and feedback? Web: http://www.eadtu.nl/e-xcellencelabel/ Thank you for your attention