1. Professor Sabiha Essack B. Pharm., M. Pharm., PhD Dean and Head of School Quality Promotion and Assurance in Teaching and Learning A Pilot Study in the Faculty of Health Sciences, UKZN

2. Overview Global Call for Quality Teaching 2 nd Cycle of Quality Audits QPA of Teaching and Learning in the Health Sciences – Quantitative and Qualitative Indicators/Measures – Data Analysis OECD Report on Quality Teaching in Higher Education AACU - High Impact Educational Practices South African Survey of Student Engagement Challenges and Solutions for Quality Audits

3. Global Call for Quality Teaching 2009 UNESCO Conference on Higher Education: participation and access to be translated into completion and success. President Obama’s concerned about college attrition - $12 billion plan to restore graduation rates. 2009 budget speech, Minister Blade Nzimande: – Improve access, success, throughput rates and the quality of the HE education experience – Increase graduates as a percentage of total enrolments to 22% by 2010. 2011 budget speech, Minister Blade Nzimande: – teaching development grants (R421 million) to improve graduate outputs – funding for foundation programmes (R174 million) to improve the success rates of students from disadvantaged educational backgrounds.

4. 2 nd Cycle of Quality Audits Identify obstacles to the improvement of teaching and learning based on: – Throughput analysis of student cohorts – Institutional self-evaluation report on the quality management of teaching and learning. Accreditation panels will interrogate institutional teaching and learning practices in relation to research and community engagement and as aligned with the context, mission and strategic focus of the institution. 3-tiered methodology: 1)University-wide throughput analysis 2)List of disciplines/Schools for detailed focus 3)Selected modules/courses as the ultimate unit of analysis.

5. QA of Teaching & Learning in the Health Sciences To investigate the use of: Pass rates, Student and peer evaluations of teaching, Moderator and external examiner reports, Reports from student support personnel, Feedback from experiential learning supervisors, and, Institutional research on teaching and learning to inform quality improvements in teaching and learning on an Individual level while delineating the quality-related roles and responsibilities of the various academic portfolio holders and/or Faculty units with respect to teaching.

6. Quantitative Indicators/ Measures Programme Data Graduation rate Cohort completion rates Academic exclusions Drop outs Module Data Pass rates per module Average mark per module Mean mark Maximum mark Minimum mark Proxies English language: previous disadvantage Matric points/NSC scores: mainstream/alternative access

7. Qualitative Indicators/ Measures Student evaluations - questionnaires designed by the Faculty with QPA. Students additionally highlighted 3 positive aspects/ strengths and/or 3 negative aspects/weaknesses of teachers and/or modules. Peer evaluations - team approach with an internal academic, an academic external to UKZN and a healthcare professional in public/private practice. The peer evaluation questionnaire designed by the Faculty with QPA and a consolidated team report submitted.

8. Confounding Factors The high admissions and stringent selection criteria in Health Sciences programmes allows the recruitment of academically proficient students that show consistent year-on-year performance trends.

9. Graduation Rate No. of graduates/headcount x 100 Compared with the national benchmark of 22% Qualification2004200520062007200820092010Average B. Nursing19%26%28%22%25%24% B. Optometry18%19%21% 18%24%19%20% B. Pharmacy24%21%25%22%20% 21%22%

10. Stratified Graduation Rate No. of graduates/headcount x 100 Compared with the national benchmark of 22% B. PharmacyLanguage2004200520062007200820092010Average L127%22%24%28%18%16%17%22% L220%21%26%15%21%25%28%22% Total 24%21%25%22%20% 21%22% B. PharmacyMatric Pts2004200520062007200820092010Average AA23%26%28%21%25%76%38%34% M22%18%23%20%17%14%19% Total 24%21%25%22%20% 21%22% B. PharmacyRACE2004200520062007200820092010Average A19%21%25%15%21%25%28%22% C0%100% 0% 14% I28%21%24%28%18%16%17%22% W0% 67%0%33%0%50%21% Total 24%21%25%22%20% 21%22%

11. Cohort Completion Rate QualificationYearMin timeMin+1yrMin+2yrsTotal B. Nursing200478.57%21.43%0.00%100.00% 200593.10%6.90%0.00%100.00% 2006100.00%0.00% 100.00% Total 90.80%9.20%0.00%100.00% B. Pharmacy200483.05%15.25%1.69%100.00% 200570.31%29.69%0.00%100.00% 2006100.00%0.00% 100.00% Total 82.21%17.18%0.61%100.00% B. Optometry200477.42%12.90%9.68%100.00% 200565.52%34.48%0.00%100.00% 2006100.00%0.00% 100.00% Total 79.52%16.87%3.61%100.00% Requires interrogation of bottleneck modules over the full course of the degree

12. Stratified Cohort Completion Rates Requires interrogation of bottleneck modules over the full course of the degree B. PharmacyRACE012Grand Total A64.71%33.82%1.47%100.00% I94.68%5.32%0.00%100.00% W 0.00% 100.00% Total 82.21%17.18%0.61%100.00% B. PharmacyLanguage012Grand Total L194.68%5.32%0.00%100.00% L265.22%33.33%1.45%100.00% Total 82.21%17.18%0.61%100.00% B. PharmacyMatric Pts012 Grand Total AA65.85%31.71%2.44%100.00% M87.72%12.28%0.00%100.00% No points87.50%12.50%0.00%100.00% Total 82.21%17.18%0.61%100.00%

13. Exclusions & Drop-outs Qualification200520062007200820092010Average B. Nursing3 1 1 B. Optometry1143 22 B. Pharmacy2312122 Qualification200520062007200820092010Average B. Nursing5913161412 B. Optometry6454238 B. Pharmacy81171871511 Drop-outs Academic Exclusions Drop-outs exceed exclusions and are largely finance-related

14. Pass Rates RaceLangMatPts200520062007200820092010 AfricanL2<30 0%75% <3686%100%67%100% 30+ 64%94% 36+79%100%76%93% L1<36 100% 30+ 100% WhiteL130+ 100% 36+ 100% IndianL2<36100% L1<36100%80%100% 30+ 90%98% 36+98%100%87%100% ColouredL1<36 100% 92%99%80%98%76%96% PHRM 202 – Introduction to Pharmacology 85% pass rate for modules

15. Average Mark RaceLangMatPts200520062007200820092010 AfricanL2<30 33.361.8 <3652.262.448.362.5 30+ 52.163.0 36+53.467.151.366.3 L1<36 63.0 30+ 59.0 WhiteL130+ 70.0 36+ 86.0 77.0 IndianL2<3667.069.0 L1<3668.566.050.0 30+ 62.366.4 36+62.970.158.075.2 ColouredL1<36 57.0 59.067.253.769.957.464.8 PHRM 202 – Introduction to Pharmacology

16. Quantitative Data Analysis Differences in performance trends would most commonly be attributed to changes in teachers, pedagogy or assessment. – Teacher-related changes verified by student and peer evaluations – Curriculum-related changes verified with module and programme portfolios. Stratified quantitative data identified student cohorts who collectively required teaching and learning interventions. Consolidated quantitative data was found to have limited use in improving individual teaching practice. Quantitative data could thus be used for benchmarking in relation to internal and national performance and equity benchmarks and targets

17. Qualitative Data Analysis Qualitative data from students and peers: – Best highlighted strengths and weaknesses – Assessed standard of content at national professional/peer level – Provided the most useful data to inform changes in teaching practice. Student evaluations informed teaching practice and relevance of content to the practice settings Peer evaluation holistically addressed curriculum content (e.g. currency of cases), pedagogy and assessment practice. The qualitative data engendered and enhanced reflective practice and was found of greatest use for teaching portfolios. Academics tended to concentrate on and address the negative aspects while cursorily acknowledging strengths Line managers found that qualitative data best allowed the identification of best practice or the need for teaching-related interventions.

18. OECD Report on Quality Teaching in HE OECD review of quality teaching in higher education undertaken in 29 institutions in 20 countries Evaluation of quality teaching is challenging Most evaluation instruments relate to input indicators: – Resource allocation for teaching e.g., number of teaching posts or staff:student ratios; – Suitable learning conditions e.g., square meters for the library – Monitoring and evaluation of policies that support quality teaching e.g. number of teachers attending training courses Dearth of instruments to evaluate the impact of quality teaching, i.e. there is no explicit evaluation criteria linking teaching input to learning outcome.

19. OECD Report on Quality Teaching in HE (2) More and more institutions are including more qualitative indicators, some of which are: – Student satisfaction rates regarding the quality of education – includes courses, teacher attitude, understanding of the discipline, quality of the learning environment – Teacher opinion surveys related to the quality of institutional support for improving teaching quality. – Use of qualitative indicators using descriptors based on a rating scale – Triangulation of information sources from open-ended questions to various stakeholders such as students, academics, external partners etc.

20. AACU: High Impact Educational Practices Learning Outcomes University education should foster deep learning and practical skills for economic, societal, civic & personal success in the 21 st century: Development of intellectual abilities and competencies Global and inter-cultural learning Technological sophistication Collaborative problem-solving Ethical and social responsibilities Transferable skills Real-world applications Personal growth and self-efficacy High Impact Practices First year seminars and experiences Common intellectual experiences Learning communities Writing-intensive courses Collaborative assignments and projects Undergraduate research Diversity/global learning Service-learning and/or community- based learning Internships Capstone Course and projects

21. South African Survey of Student Engagement What students do - the time and energy they devote to educationally purposive activities What institutions do - the extent to which they employ effective educational practices to induce students to do the right things The survey conducted amongst staff and student ascertained : – Level of academic challenge – Active and collaborative learning – Student-staff interaction – Enriching educational experiences/adjunct learning opportunities – Supportive campus environment The results: – Were stratified by first year and senior students – Areas of agreement and discrepancy between staff and students were highlighted – Staff time spent on teaching-related activities and participation in teaching development/improvement activities was described.

22. Challenges for the 2 nd Cycle of Quality Audits The impact of teaching and/or student engagement with learning is critical to student retention and throughput but is yet to be effectively measured. Striking the correct balance between quantitative and qualitative teaching quality indicators/ measures Identifying suitable qualitative indicators/ measures for quality teaching Ensuring that such indicators (quantitative/ qualitative) address both teaching inputs as well as teaching impact/learning outcomes.

23. References Henard, F. Learning our Lesson Review of Quality Teaching in Higher Education. Paris: Organization for Economic Co-operation and Development; 2012. Kuh, G.D. High Impact Educational Practices What Are They, Who Has Access To Them And Why Do They Matter. Washington: Association of American College and Universities; 2008. http://www.unesco.org/fileadmin/MULTIMEDIA/HQ/ED/ED/pdf/WCHE_2 009/FINAL%20 COMMUNIQUE%20WCHE%202009.pdf http://www.unesco.org/fileadmin/MULTIMEDIA/HQ/ED/ED/pdf/WCHE_2 009/FINAL%20 COMMUNIQUE%20WCHE%202009.pdf http://www.universityworldnews.com/article.php?story=2009091123370 3416 http://www.universityworldnews.com/article.php?story=2009091123370 3416 http://www.dhet.gov.za/LinkClick.aspx?fileticket=zhQIjhq1jSc%3d&tabid= 36&mid=1071 http://www.dhet.gov.za/LinkClick.aspx?fileticket=zhQIjhq1jSc%3d&tabid= 36&mid=1071 http://www.che.ac.za/documents/d000208/SASSE_2010.pdf

24. Acknowledgements Nursing: Professor Petra Brysiewicz Ms Charlotte Engelbrecht Optometry: Dr Rekha Hansraj Ms Diane Wallace Pharmacy: Professor Fatima Suleman Dr Johannes Bodenstein Dr Frasia Oosthuizen

25. Small Group Activity 1.What quantitative and qualitative indicators/measures does your institution use for teaching inputs? Why? 2.What quantitative and qualitative indicators/measures does your institution use to monitor teaching impact and outputs and learning outcomes? How does this link to student surveys? 3.How do these findings (from questions 1 and 2 above) improve the quality of teaching and student engagement?