Evaluating Learning Technologies at University of Worcester Review Meeting October 4th, 2006 John Colvin

HEA Pathway Pilot Initial results publication co-incided with start of project Trialled e-Learning benchmarking in a number of HEI’s Feb 2007 Pathfinder bid!! eLearning cannot be assessed in isolation from L&T ICT HR Estates e-Learning data often not recorded Results confidential Blogging review culture is considered appropriate

HEA Pathway Pilot (Cont…) The 5 benchmark tools piloted all “fit for purpose”.  All Available for Feb 2007 Bid Customisation appropriate ELTI “Pick ‘n Mix” eMM OBHE MIT90 JC Decision: Adopt one of benchmarking tools - avoid re-inventing the wheel As the report says the “the precise nature of the method is much less important than starting …. to ask penetrating questions”

Choice of Benchmarking Tool Adapted version ELTI Pathfinder summary did not favour any tool deemed them all fit-for-purpose All available Feb 2007 bid Why ELTI Pilot supported by external consultants Availability of associated support documentation was considered crucial.   Materials for elti and ‘Pick and Mix’ were appropriate and readily available Accessing materials for other tools was less successful Extent and relevance of ELTI audit questions are, IMO, impressive Pathfinder summary suggests that participation is the crucial issue

ELTI audits (Evidence + Assessment) Profile of learning and teaching Profile of learning technologies Recognition and reward e.g. There are specific professional rewards for LT innovation (e.g. additional funding, progression points, teaching fellowships) Research and/or development ICT infrastructure Learning technology support (Cont…)

ELTI audits (Cont….) Learning Technology funding Administrative infrastructure Staff ICT skills All teaching staff have basic WebCT skills (putting lectures and discussion forums online) Student ICT skills Digital/ multimedia resources Networks and collaborations

Some Jobs Completed / Underway Selection of Benchmarking Tool Data Collection / Analysis L&T: e.g. Strategies, quality enhancement, support, innovations LT Profile e.g. University initiatives, integration at course level LT Recognition & Reward e.g. Innovation support, Progression

Some Jobs Completed / Underway R&D e.g. RAE, specialist units Staff Skills e.g. ICT skills, WebCT skills, skills support Student Skills e.g. audit procedures, support LT Support Effectiveness of h/ware & s/ware support

Selection of Major ‘Jobs’ Outstanding Audit of WebCT Usage Audit use of Learning Technologies in classroom Audit of Learning Technologies at UW Review Dept AMR’s & L&T strategies Benchmarking with other HEI’s

Audit of WebCT usage After Learning week - Fair snapshot Inspection of WebCT courses ‘Health Warning’ Everyone email circulated Access Strategy under discussion Level of Audit No WebCT course set up WebCT course set up, not used General course data Course Materials Discussion Groups Other use e.g. Assessment Tool

Use of Learning Technologies in Classroom Clip Board Survey?? Learning week Contingency Plan: email survey Ask questions about last classroom lesson taught LT in class IWB, projector, MP3… LT to support learning outside classroom WebCT, blogs, eBooks Ask about their own most innovative LT use How, if at all has LT changed the way you teach? Ask: Single most important support that UW could offer to encourage/improve your use of LT?

Other HEI’s Benchmark data from 2000/1 Selection of new and some pre-92 HEI’s Contacts? Particularly Pathfinder Trial participants December / January Large body of our data available Some analysis completed Organise late October