Exploring the contractions and tensions in student evaluation in Australian higher education Stephen Darwin

Origins of the study Experiences in teacher education and reform debates around the elevating role of student feedback in mediating conceptions of quality teaching in higher education Rising level of institutional and sectoral significance placed on the outcomes of student opinion as an accountability (as opposed to improvement) mechanism Limited scholarly research around the powerful epistemological assumptions and critical tensions of orthodox forms of (quantitative) student feedback 2

Research Questions 3  Which factors have shaped the development of student feedback-based evaluation in Australian higher education?  What functions does student feedback-based evaluation perform in the contemporary Australian higher education?  Does student feedback-based evaluation have a further developmental potential beyond that derived from its conventional quantitative form?

Conceptual framing  Qualitative methodology grounded in CHAT  CHAT used to understand and influence the nature of complex social practices through the contextual analysis of the historical layers, mediating artefacts and objects of local activity  Focus on multi-layered and multi-voiced collective forms of activity, centred on identification of tensions and contradictions 4

Methodology (continued)  Use of CHAT and action research method to investigate: – local practices around role and function of orthodox quantitative student feedback –whether student feedback could generate collective professional dialogue and program development –expansive learning potential of elevated use of qualitative student feedback  Researcher as participant-observer 5

6 Sources of data Historical analysis of evolution of student feedback in Australian higher education and localised context Two case studies conducted in the ANU College of Law (Migration Law and GradDip in Legal Practice) Data derived from document analysis, observational analysis, generated artefacts and semi- structured interviews Data progressively thematically coded and itself became part of the emergent design of the study

Outcomes: current function 7  Strong tensions between the drives of quality improvement, quality assurance and performance management creates ambiguities around its contemporary role and function  Simultaneously an academic fringe dweller and privileged institutional citizen (as a powerful proxy for assessments of teaching quality)  Loosening relationship with academic development and strengthening with internal quality assurance and performance management

Key identified tensions: case studies  Maintaining learning quality versus need for ‘above average’ student satisfaction levels  Drive for pedagogical innovation versus the need to meet student expectations-demands  Accountability to the discipline for ‘standards’ versus individual accountability as an ‘effective’ educator to institution and students  Quantitative assessment versus need for broader qualitative insight and support pedagogical improvement  Pressure to sustain student enrolments versus need to challenge/broaden legal knowledge and disrupt ingrained assumptions about effective legal practices 8

9 Volunteering academics All teaching academics Academics seeking tenure/promotion or identified for poor performance Local ‘idiosyncratic’ student feedback systems, academic development tradition Sectoral accountability demands, CEQ, public league tables Promotional/tenure/performance management processes requiring quantitative evidence of teaching quality Heightening professional demands for improved teaching practice Selection and performance demands Sectoral performance standards and related funding Academics and academic developers Academics and university administration Students, institutions and regulators Individual academics Academics and panels Institutions and regulators Improved student learning Improved individual outcomes Improved student evaluation outcomes Shared Object Quality improvement of individual or course teaching (Activity One) Quality assurance of teaching and learning practices (Activity Two) Individual performance measurement (Activity Three) Potential for expansive learning Use of student feedback as a tool for individual and collective improvement of teaching and learning quality at a course or program level Make visible the broad range and implications of student feedback to highlight areas of potential improvement to encourage professional dialogue Frame situated academic development around the outcomes of this professional dialogue toward substantial and demonstrable improvement Provide more substantial and valid qualitative data of individual and course change and improvement as a means of assuring later quantitative outcomes Provide basis for more sophisticated evidence of performance and/or improvement in teaching

Outcomes: the expansive potential 10  The sociocultural foundations in programs and the institution played a critical shaping function  Case studies demonstrated that elevated use of student feedback can drive collective professional dialogue and expansive learning  Largely unresolved tensions with broader quality assurance and performance management discourses  Evidence of encouraging boundary crossing (legal profession, educational design)

Outcomes: the expansive potential (cont.)  Given scale of potential development and time/resource limitations, collective approach was prone to domination by program leaders (and alienation by peripheral staff)  Unresolved tension with broader quality assurance and performance management discourses  CHAT-informed action research model was resource intensive and difficult to sustain once a ‘quality threshold’ was perceived to have been reached (suggesting limitations in its potentiality as a methodology) 11