1. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Evaluating programs

2. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Problem: You have been given responsibility for the quality of a three year undergraduate program involving some 30 courses. What evidence do you have about the program and its outcomes that informs decisions about quality and quality improvement?

3. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Feeling powerless???? GCEQ – but……………… Commission an evaluation by some-one – but what data can be collected, what standards apply, who interprets the data against the standards and how will evaluation be used in decision making?

4. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Answer????? Depends on what you consider to be relevant data and how it might be used either ‘by itself’ or in conjunction with other information

5. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA The issues for you are: Getting data that connects to academic work – eg how do students regard my course? Having tools that allow you to generate data about programs Being able to connect data and tools with professional judgments to decide about quality improvement? Getting others to understand the system and its limitations in assisting decisions about quality improvement

6. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Aggregated evaluation items courses Program characteristic 1 Program characteristic 2 Program characteristic n Model 1 All students in all courses in the program Exit students Program evaluation Program characteristics Model 2

7. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Course evaluation aggregated to provide program and sub program data Policy framework identifies Program Directors to manage evaluation through management of course evaluation Online delivery that uses data from student information system Formal Program Evaluation Reports include benchmarking based on data

8. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

9. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

10. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

11. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

12. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

13. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

14. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

15. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA

16. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA System Components A workable view about quality Identified standards A means of measurement Judgment against standards Linkages with quality improvement Linkages with change agents

17. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA A workable view about quality Resolved through policy directions Shaped by national ‘tests’ and performance tables Privileges student data within evaluation process

18. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Identified standards GCEQ scales related to teaching and generic program outcomes

19. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA A means of measurement Course data that is aggregated rather than specific program instrument Core items make up standardised survey – items sample scales from the GCEQ and include overall satisfaction Constructed and delivered online through course home pages

20. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Judgment against standards Intra program benchmarking according to other high performance courses in relation to items of the core External benchmarking – overall satisfaction and GCEQ scales

21. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Linkages with quality improvement Required reportage of evaluation of program Requirements regarding identification of changes for quality enhancement

22. Educating Professionals - Creating and Applying Knowledge - Serving the Community UNIVERSITY OF SOUTH AUSTRALIA Program Evaluation at UniSA Linkages with change agents Reward systems use evaluation data