Quality - what are we measuring? Jim Murdoch, University of Glasgow European Law Faculties Association

I keep six honest serving men... What? - measuring-rods Why? - rationale When? - change through time How? - methodology Where? Who? - appointment of reviewers –Rudyard Kipling ( )

Why not ?

What is quality assurance? ‘inspection, compliance, bureaucracy, report writing, form-filling, self- justification, standardisation, piles of unread [paper], and procedures of such baroque (even, on occasion, rococo) elaboration that they resemble the wilder architectural fantasies of the 18th century Tsars’ –Chief Executive, QAA [UK], October 2004

What is quality assurance? Stratification - competition between law schools Academic life as ‘nasty, brutish and (for the non-productive) short’ - competition between individuals Wasteful of academic time - detracting from teaching and research –Brownsword, [1994] 21 JoLaw &Soc

Why?

What is quality assurance? ‘It is a way of defining and securing good learning through effective support for students. It is a chance to demonstrate… the transmission of knowledge and the development and transformation of individuals is recognised and discharged professionally’. –Chief Executive, QAA [UK], October 2004

What is quality assurance? Collective achievement Responsiveness Recognising success Accountability to ‘stakeholders’ Enhances mobility

How?

Methodology QA is a means to an end, not an end in itself. The more paper generated in the name of QA, the less is likely to be read or understood. Less should be more. There is no justification for more ‘bureaucracy’. –Chief Executive, QAA [UK], October 2004

Methodology Development of agreed framework –length / intensity (credit scheme) –content : knowledge / skills (‘benchmarking’) Self-assessment Peer review Peer assessment –accreditation / grading / consultancy / identification of ‘best practice’

What?

What are we measuring? Subject provision and overall aims Curriculum Teaching and learning Learning resources Teaching quality enhancement Assessment and achievement Student support

Subject provision and overall aims Overall aims and intended learning outcomes of the programme? Appropriateness of aims/ILOs in terms of developments in learning and teaching Do staff and students understand aims and ILOs?

Subject provision and overall aims Aims are implicit, not explicit ILOs are not stated, or fail to spell out development of knowledge / skills (generic and law-specific) / or the inculcation of attitudes and values Staff and students do not know/understand

Curriculum Effectiveness of curricular content and design in achieving ILOs Academic and intellectual progression within the curriculum Inclusion of recent developments in the subject; and reflection of best practice in pedagogy

Curriculum Input-driven curriculum & student overload No conceptualising (core; contextual; legal values; professional) [ACLEC Report] No progression through the programme; and recent developments ignored/avoided

Teaching and learning Effectiveness of teaching and learning in relation to aims and curriculum Range and appropriateness of teaching methods employed Student participation & workload

Teaching and learning Traditional teaching methods (didactic) & not ‘student-centred learning’ No proper link between T&L and assessment Student workload not properly addressed

Learning resources Teaching and support resources (physical, human and material) Effectiveness of utilisation (including induction, mentoring and staff development).

Learning resources Unfavourable staff-student ratios Inadequate accommodation / support resources (library/IT/audio-visual) & support staff Inappropriate academic staff deployment / lack of staff development

Teaching quality enhancement Effectiveness of evaluation and use of quantitative data and qualitative feedback Commitment to enhancement and continuous improvement Constant review / remedial action in respect of: curriculum, teaching and learning, resources and assessment.

Teaching quality enhancement No evidence of quality control and enhancement mechanisms such as: –course review through course questionnaires –staff-student liaison committees –procedure for scrutiny of new course procedures

Assessment and achievement Effectiveness of assessment in measuring achievement of ILOs Discrimination between different categories of performance Effectiveness of assessment in promoting student learning (& use of diagnostic and formative assessment)

Assessment and achievement ‘One size fits all’ assessment Assessment driving surface-learning Reliability and validity questionable

Student support Academic support strategies Recruitment and induction Overall academic guidance and supervision

Student support Poor drop-out / completion rates Poor recruitment and induction processes ‘One size fits all’ support

BUT… does it work?

Methodology Fairness - consistency? Reliability - methodology? Utility - competing purposes (vfm / encouraging improvement / public info)? –Brownsword, [1994] 21 JoLaw &Soc

What should be measured? - the next phase Institutional audit ‘Enhancement’ strategies ‘Employability’ strategies