Costs of e-learning in HE - and the benefits Professor Paul Bacsich Sheffield Hallam University, UK

2 Methodological foundation “Suitably interpreted and broadened, and leavened with pedagogy, the rather dry methods of business costing and planning can provide a sound basis for analysing e-learning developments and their impact.”

3 Summary  CNL Phase 1 [JISC JCALT]  CNL Phase 2 [JISC JCALT]  Effectiveness: impact of other work including analysis of the training sector [EU Telelearn etc] and the NLN Evaluation work [FEDA]  Some impact from related work including RCIT (CNL methods applied to admin IT systems) [JCALT] and soon perhaps from “Managing Change” (formerly TLTR) on change management in HEIs [JCIEL]

4 Main issues What learners want - must be first! Costs Effectiveness

5 What learners want...  From our own work on learners (CNL):  students believe that e-learning increases costs to them  but they behave as though it saves costs  and saves time!

6 What learners want  From our own work on learners (CNL):  students believe that e-learning increases costs to them  but they behave as though it saves costs  and saves time!  From other work (including USA):  they want flexibility (but not too much)  some want overclocking (ie faster throughput)

7 Hidden Costs - examples  Increased home telephone call bills for students due to Internet usage  Entertainment expenses incurred by lecturers but not reimbursed by the HEI  The usage of teaching or research budgets to bolster IT budgets - or vice versa

CNL1

9  In order to accurately record the Costs of Networked Learning you must have a universally accepted method of what costs to record and how.  Much of the literature formed interesting background reading but failed to travel far enough towards operational conclusions to be taken (individually) as a basis.  The method must include a lifecycle model, stakeholders, activity based costing and a “cost- aware” approach to planning. Conclusions - CNL1

10 Course Lifecycle Model Planning and Development Production and Delivery Maintenance and Evaluation Three-phase model of course development

11 Breakdown of three- phase model

12 The main Stakeholders  The HEI  and the staff of that HEI  The learners

13 Stakeholders - two more special cases  The HEI  and the staff of that HEI  The learners  and their organisation and in some cases their parents/partners

14 The Financial Schema - cf Transparency Review  The traditional financial model underpinning work in most HEIs is defective in four ways: No Stakeholders Crude Overhead Allocation No agreed set of activities No time recording

15 Planning Document and Financial Schema Multi-level Activity-Based Costing Flexible Overheads Three-phase Course Lifecycle Model Based on previous works Multi-stakeholder Time Division Time recording

Activity-Based Costing

Activity-based costing (ABC) ABC was developed as an alternative costing methodology by Robin Cooper and Robert Kaplan of the Harvard Business School (1988) in their research into product costing in the manufacturing industry.

18 Cooper and Kaplan (1988)  They argued that traditional costing distorted product costs (“peanut butter spread approach”)  There is a cost to all activities carried out within an organisation  Activity costs should be distributed to products in relation to their use

ACTIVITY DRIVERS Hold Tutorials No. Students Business Intelligence Resources Cost for each Customer/product/channel group method Activities Staff time Cost driver I.T. & Management FT DL

Activity A series of related tasks carried out repeatedly Chase a student for late payment Process A series of activities required to achieve an outcome Invoicing Process Task One action by one person at one point in time Telephone student Definitions

21 Advantages of ABC  Gives more than just financial information  Fairer system of overhead allocation  Accountability of central services  Highlights cross-subsidisation  Recognises the changing cost behaviour of different activities as they grow and mature  Can provide data for other initiatives

22 Disadvantages of ABC  Data collection is costly and time consuming  Initially it can be difficult to collect the data you want  Determining appropriate cost drivers  More complex system

23 Implementing ABC takes time “Its important in a service organisation to get started with ABM and not worry about total accuracy from the start. Accuracy will improve with time” Antos (1992)

24 CNL2  This was a trial of ABC in one faculty of Sheffield Hallam University, using industrial-strength ABC software and consultancy (Armstrong-Laing)  The trial was funded by JISC JCALT  The report of the trial (plus workbook) will be available in August 2001  The full set of costs are not reported!

Tiptoeing into effectiveness issues

26 History  Relatively early in the history of CNL1, effectiveness issues were looked at (Ash, November 1999)  Main conclusion was that a uniformly accepted methodology was needed there also; and some suggestions were given, based to a certain extent on the US Flashlight work.  This work led into the NLN evaluation studies (English FE sector)

27 Effectiveness: a big “border” between E & T  The NSD effect in academia (No Significant Difference)  versus learning gains and large Return on Investment in training world:  700% ROI at Royal Bank of Scotland (Shepherd)  routine % learning gains  and situations with 60% learning gains

28 NSD in academia...  It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students  The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work.

29 NSD in academia  It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students.  The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work.  But we contend that the NSD work has several major flaws - in particular many key variables were not controlled.

30 Self-incrimination?  “There were no significant pre- and post- course differences on the items related to students’ expectations and perceived benefits...  There were no significant differences between students in the [online] compared to students in the [face to face] courses for course retention or course grade...  but there were increases in the amount of time and the purposes for which students used computers." (Green et al 2001)

31 So is a key difference to do with time?  And what is time?

32 What is time?  “Time is the new distance” (Mason, inaugural lecture 2001)

33 Can e-learning save time?  travel time

34 Can e-learning save time?  travel time  reduce “time on task”  by “better” training (25-40%)  by individualised training...

35 Can e-learning save time?  travel time  reduce “time on task”  by “better” training (25-40%)  by individualised training...  by using “time of the 3rd kind”  by consolidating time fragments

36 Time of the 3rd kind  Time 1 : on duty - used to be called “at work”  Time 2 : off duty (not at work)

37 Time of the 3rd kind  Time 1 : on duty - used to be called “at work”  Time 2 : off duty (not at work)  Time 3 : in-between  on duty but less productive  eg travelling  eg slots in between other tasks  off-duty but somewhat productive  eg in bar with colleagues  at dinner with customers

38 One key conclusion  To understand effectiveness issues we need a much better understanding of time  and its monetary and other value to students  Only when one understands time should one move on to other measures of effectiveness

Details about our work at: Paul Bacsich Professor of Telematics School of Computing and Management Sciences Sheffield Hallam University