Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Evaluating programs

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Problem: You have been given responsibility for the quality of a three year undergraduate program involving some 30 courses. What evidence do you have about the program and its outcomes that informs decisions about quality and quality improvement?

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Feeling powerless???? GCEQ – but……………… Commission an evaluation by some-one – but what data can be collected, what standards apply, who interprets the data against the standards and how will evaluation be used in decision making?

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Answer????? Depends on what you consider to be relevant data and how it might be used either ‘by itself’ or in conjunction with other information

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA The issues for you are: Getting data that connects to academic work – eg how do students regard my course? Having tools that allow you to generate data about programs Being able to connect data and tools with professional judgments to decide about quality improvement? Getting others to understand the system and its limitations in assisting decisions about quality improvement

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Aggregated evaluation items courses Program characteristic 1 Program characteristic 2 Program characteristic n Model 1 All students in all courses in the program Exit students Program evaluation Program characteristics Model 2

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Course evaluation aggregated to provide program and sub program data Policy framework identifies Program Directors to manage evaluation through management of course evaluation Online delivery that uses data from student information system Formal Program Evaluation Reports include benchmarking based on data

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA System Components A workable view about quality Identified standards A means of measurement Judgment against standards Linkages with quality improvement Linkages with change agents

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA A workable view about quality Resolved through policy directions Shaped by national ‘tests’ and performance tables Privileges student data within evaluation process

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Identified standards GCEQ scales related to teaching and generic program outcomes

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA A means of measurement Course data that is aggregated rather than specific program instrument Core items make up standardised survey – items sample scales from the GCEQ and include overall satisfaction Constructed and delivered online through course home pages

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Judgment against standards Intra program benchmarking according to other high performance courses in relation to items of the core External benchmarking – overall satisfaction and GCEQ scales

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Linkages with quality improvement Required reportage of evaluation of program Requirements regarding identification of changes for quality enhancement

Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Linkages with change agents Reward systems use evaluation data