Evaluation meets costing: Fight or Flight? Charlotte Ash Sheffield Hallam University 21 st HE Cost-effectiveness framework 21 st HE Cost-effectiveness framework C

The Costs of Networked Learning b Six month study funded by the JISC b Additional support provided by Sheffield Hallam University b Project team - director, manager and additional researchers  For further information, visit the Costs of Networked Learning Web page:

21 st HE Cost-effectiveness framework Cost b problems b solutions b What has been done already Effectiveness b problems b solutions C

The Future of Higher Education Universities must: b Change infrastructure b Increase access b Reduce expenditure b Become transparent

Assessing Cost-Effectiveness Stage 1: Identify problems with costs Stage 2: Find solutions Stage 3: Testing and refinement Stage 4: Identify effectiveness problems Stage 5: Find solutions Stage 6: Testing and refinement Stage 7: Joint solution to Cost-effectiveness

Assessing Costs Problems 1 Lack of literature 2 What to cost? 3 How to cost? 4 Whom to cost?

Assessing Costs: Problem 1 Problem b Lack of literature b Lack of studies b Lack of framework Solution b Eclectic and Holistic approach

Assessing Costs: Problem 2 Problem b What to cost? Solution b Identify hidden costs b Formulate list for each stakeholder: staff, student and institution b Structure list into working model

Lifecycle models Planning the learning experience/ environment Maintaining the learning experience/ environment Developing the learning experience/ environment Managing the learning experience/ environment Providing the learning experience/ environment Academic Staff Support Staff Students Planning and Development Production and Delivery Maintenance and Evaluation Final three stage model Previous five stage model

Assessing Costs: Problem 3 Problem b How to cost? Solution b Planning framework Planning document Financial schema Financial Schema

Assessing Costs: Problem 4 Problem b Whom to cost? Whom to include?Whom to include? Who to do?Who to do? Solution Stakeholders Translator/Negotiator

Solutions 1 Eclectic/Holistic approach 2 Hidden, List Model 3 Planning framework planning document financial schema 4 Whom to cost: Stakeholders Translator/Negotiator Assessing Costs Problems 1 Lack of literature 2 What to cost? 3 How to cost? 4 Whom to cost? Whom to include?Whom to include? Who to do it?Who to do it?

Assessing Effectiveness Reflections on a Model for Evaluating Learning Technologies by Jones, Scanlon and Blake CIAO! evaluation framework developed by the Computers and Learning Research Group at the Open University developed by the Computers and Learning Research Group at the Open University “Do my students learn better, faster or more using this technology?”

Assessing Effectiveness Evaluating Technology-Based Learning by Dr Robin Mason Holistic evaluation Good evaluations demand skill, perception and resources Good evaluations demand skill, perception and resources

Flashlight b Current Student Inventory ( 1994 ) effectivenesseffectiveness b Costs Analysis Handbook ( 1999 ) costscosts b Problems: 1 Costs and effectiveness kept separate 2 Individualistic approach prohibits comparisons

Assessing Cost-Effectiveness Stage 1: Identify problems with costs Stage 2: Find solutions Stage 3: Testing and refinement Stage 4: Identify effectiveness problems Stage 5: Find solutions Stage 6: Testing and refinement Stage 7: Joint solution to problem of evaluating the cost-effectiveness of HE

Evaluation meets costing: Fight or Flight? Charlotte Ash Sheffield Hallam University Thank you for listening