Teaching Effectiveness Matters: A survey of teaching effectiveness indicators, quality criteria and standards across Australian and Malaysian Universities 1. Introduction – introduce the project 2. Describe the survey 3. The seven criteria 4. Go over the seven slides of most important standards in each criterion 5. Group activity –give sheet all criteria and standards and ask small groups to pick one standard and discuss what kinds of evidence they might gather 6. Feedback on issues 7. Issues arising 8. Finish – where next and thanks. Dr. Craig Whitsed Professor Richard Cummings Centre for University Teaching and Learning Professor Heather Smigiel Flinders University

Aim of the collaboration: To survey senior academic and professional staff across the IRU and MRUN networks about their perceptions of importance and application of the AUTCAS criterion and standards AUTCAS Framework Perceived importance Probation & promotion IRU and a group of Malaysian universities agreed to collaborate and funded a range of research projects including in teaching and learning One project was a two year study to explore the use of the AUTCAS framework in the two groups of universities. One of the activities was to survey staff on three key questions: Which of the 51 standards in the 7 criteria in the framework are currently being used in the institutions in annual reviews of academic staff? Which standards are currently being used in the institutions in probation or promotion processes for academic staff? How important is each standard when assessing teaching effectiveness? Annual review of performance Universiti Malaya (UM), Universiti Sains Malaysia (USM), Universiti Kebangsaan Malaysia (UKM), Universiti Putra Malaysia (UPM), Universiti Teknologi Malaysia (UTM) Murdoch University, Flinders University, Charles Darwin University (CDU), James Cook University (JCU)

What we did Which AUTCAS standards do university staff consider most importance? Which standards are currently used in academic annual performance reviews? Which standards are currently used in academic promotion and probation processes? Developed a common survey with the MRUN institutions to find out from university staff, specifically senior academic staff who generally are involved in making judgments about effective teaching, the following three key questions: Which AUTCAS standards do they consider most importance? Which standards are currently used in academic annual performance reviews? Which standards are currently used in academic promotion processes? Received nearly 1000 responses across the 9 unis and have

Evaluation of practice and continuing professional development http://uniteachingcriteria.edu.au/framework/indicative-criteria/ Integration of scholarship, research and professional activities with teaching and in support of student learning Evaluation of practice and continuing professional development Professional and personal effectiveness Design and planning of learning activities Teaching and supporting student learning Assessment and giving feedback to students on their learning Developing effective learning environments, student support and guidance Many of you will know of the AUTCAS Framework developed by the 5 WA universities in one of the final OLT strategic projects. Many universities have used the framework in reviewing and developing processes for academic staff annual reviews, probation and promotion processes, and professional development frameworks. A number of IRU institutions were interested in exploring the potential use of the framework and the MRUN universities were very interested because they generally were only using scores on student feedback surveys. They were very interested in exploring a more broadly based approach to measuring teaching effectiveness.

DESIGN AND PLANNING OF LEARNING ACTIVITIES MOST IMPORTANT STANDARDS Criterion 1 DESIGN AND PLANNING OF LEARNING ACTIVITIES Teaching plan that aligns to teaching and learning activities and assessment Knowledge of the discipline area In Criteria 1, these three standards were the top three in both countries Preparation of teaching materials

Teaching and Supporting Student Learning Criterion 2 Teaching and Supporting Student Learning Effective supervision of student research activity Engage and stimulate student participation In Criteria 2, two standards were agreed as being in the top three in both countries IRU – also focused on student centred approach MRUN – focused on Innovation which is a national priority in Malaysia Use of innovative teaching approaches Use of student centred approaches

Assessment and Giving Feedback to Students Criterion 3 Assessment and Giving Feedback to Students Constructive and timely feedback is provided to students The assessment tasks are closely linked to the learning outcomes Criterion 3 – agreed on the need for feedback to students MRUN staff concerned with linking assessment to Los IRU concerned with this but also with the clarity of assessment requirements Assessment constructively aligned to learning outcomes Assess intended learning outcomes The assessment requirements are clearly stated

DEVELOPING EFFECTIVE ENVIRONMENTS, STUDENT SUPPORT AND GUIDANCE Criterion 4 DEVELOPING EFFECTIVE ENVIRONMENTS, STUDENT SUPPORT AND GUIDANCE Initiative and innovation in supporting students and the creation of engaging learning environment Availability for consultation Criterion 4 agree on initiative and innovation IRU – support student diversity and availability MRUN – Support to students with specific needs and demonstrating collaborative learning Demonstrate effective practice in developing collaborative learning Support and guidance of students with specific academic needs Creating supportive learning environment to support student diversity

INTEGRATION OF SoTL AND PROFESSIONAL ACTIVITIES SUPPORT OF LEARNING Criterion 5 INTEGRATION OF SoTL AND PROFESSIONAL ACTIVITIES SUPPORT OF LEARNING Use of current disciplinary research in curriculum through teaching activities Use of authentic case studies, industry experience and/or partnerships in teaching Agree on use of disciplinary research in teaching and use of authentic case studies and industry experience Share teaching and learning research within your discipline Effective preparation of students prior to their work-based experience

EVALUATION OF PRACTICE AND CONTINUING PROFESSIONAL DEVELOPMENT Criterion 6 EVALUATION OF PRACTICE AND CONTINUING PROFESSIONAL DEVELOPMENT Leadership and contribution in the provision of professional development of others Self-evaluation leading to changes in teaching practice and student outcomes Criterion 6 - Agree on on 2 Participation in teaching and learning conference/forums Successful completion of Foundation of University Teaching program or equivalent

PROFESSIONAL AND PERSONAL EFFECTIVENESS Criterion 7 PROFESSIONAL AND PERSONAL EFFECTIVENESS Application of professional ethical practices in work and teaching context Approaching teaching with enthusiasm, passion and confidence Agree on 2 Overall there are 13 agreed standards in the top three out of a possible 21 (62%) Responding positively to opportunities and new teaching approaches Demonstrating commitment and interest in students and their learning

to

Issues arising from the research Criterion/Standard Importance Use in Annual Reviews Use in Probation/Promotion C2 - Scores in student evaluation survey Low High C2 - Effective supervision of student research activity C6 - Self-evaluation leading to changes in teaching practice and student outcomes Criterion 5 - Integration of Scholarship (SoTL), Classroom Research and Professional Activities with Teaching and in Support of Learning

What next? Dr. Craig Whitsed: c.whitsed@murdoch.edu.au It might be useful to refer to the HEA framework and to point out that one possible direction for the framework is in the context of the IRU and the conversations with them having Dr. Craig Whitsed: c.whitsed@murdoch.edu.au Professor Rick Cummings: r.cummings@murdoch.edu.au