1. Evaluating Learning Technologies at University of Worcester
Review Meeting
October 4th, 2006
John Colvin

2. HEA Pathway Pilot
Initial results publication co-incided with start of project
Trialled e-Learning benchmarking in a number of HEI’s
Feb 2007 Pathfinder bid!!
eLearning cannot be assessed in isolation from
L&T
ICT HR
Estates
e-Learning data often not recorded
Results confidential
Blogging review culture is considered appropriate

3. HEA Pathway Pilot (Cont…)
The 5 benchmark tools piloted
all “fit for purpose”. 
All Available for Feb 2007 Bid
Customisation appropriate
ELTI
“Pick ‘n Mix”
eMM
OBHE
MIT90
JC Decision: Adopt one of benchmarking tools - avoid re-inventing the wheel
As the report says the “the precise nature of the method is much less important than starting …. to ask penetrating questions”

4. Choice of Benchmarking Tool
Adapted version ELTI
Pathfinder summary
did not favour any tool
deemed them all fit-for-purpose
All available Feb 2007 bid
Why ELTI
Pilot supported by external consultants
Availability of associated support documentation was considered crucial.  
Materials for elti and ‘Pick and Mix’ were appropriate and readily available
Accessing materials for other tools was less successful
Extent and relevance of ELTI audit questions are, IMO, impressive
Pathfinder summary suggests that participation is the crucial issue

5. ELTI audits (Evidence + Assessment)
Profile of learning and teaching
Profile of learning technologies
Recognition and reward
e.g. There are specific professional rewards for LT innovation (e.g. additional funding, progression points, teaching fellowships)
Research and/or development
ICT infrastructure
Learning technology support
(Cont…)

6. ELTI audits (Cont….) Learning Technology funding
Administrative infrastructure
Staff ICT skills
All teaching staff have basic WebCT skills (putting lectures and discussion forums online)
Student ICT skills
Digital/ multimedia resources
Networks and collaborations

7. Some Jobs Completed / Underway
Selection of Benchmarking Tool
Data Collection / Analysis
L&T:
e.g. Strategies, quality enhancement, support, innovations
LT Profile
e.g. University initiatives, integration at course level
LT Recognition & Reward
e.g. Innovation support, Progression

8. Some Jobs Completed / Underway
R&D
e.g. RAE, specialist units
Staff Skills
e.g. ICT skills, WebCT skills, skills support
Student Skills
e.g. audit procedures, support
LT Support
Effectiveness of h/ware & s/ware support

9. Selection of Major ‘Jobs’ Outstanding
Audit of WebCT Usage
Audit use of Learning Technologies in classroom
Audit of Learning Technologies at UW
Review Dept AMR’s & L&T strategies
Benchmarking with other HEI’s

10. Audit of WebCT usage After Learning week - Fair snapshot
Inspection of WebCT courses
‘Health Warning’ Everyone circulated
Access Strategy under discussion
Level of Audit
No WebCT course set up
WebCT course set up, not used
General course data
Course Materials
Discussion Groups
Other use e.g. Assessment Tool

11. Use of Learning Technologies in Classroom
Clip Board Survey??
Learning week
Contingency Plan: survey
Ask questions about last classroom lesson taught
LT in class
IWB, projector, MP3…
LT to support learning outside classroom
WebCT, blogs, eBooks
Ask about their own most innovative LT use
How, if at all has LT changed the way you teach?
Ask: Single most important support that UW could offer to encourage/improve your use of LT?

12. Other HEI’s Benchmark data from 2000/1
Selection of new and some pre-92 HEI’s
Contacts?
Particularly Pathfinder Trial participants
December / January
Large body of our data available
Some analysis completed
Organise late October