Summary of the discussion sessions at the Sixth Quality in Higher Education Seminar The End of Quality? 25–26 May 2001 Birmingham

The End of Quality? Has external quality review had its day? Has control of quality been usurped by the market and by information technology? Does the development of mass education necessarily mean the end of quality? Three discussion themes:

External Quality Monitoring (EQM) accreditation (of institutions and programmes); institutional quality audit or assessment; programme assessment or review; external evaluation and comparison of standards; research reviews; internal-external. EQM includes :

Theme 1 Has external quality review had its day?

External quality monitoring leads to bureaucratisation and inflexibility? is amateurish? leads to game playing and performance? short-term response not cultural changes? has a superficial impact on standards? has no real impact on student learning? is obsessed with accountability?

Bureaucracy Any form of EQM would involve some level of bureaucracy. Key issue is not the existence of a bureaucracy or of bureaucratic processes but the nature of the bureaucracy and its processes. Bureaucracy must meets needs of external and internal stakeholders, not be self-perpetuating.

Quality monitoring bureaucracies Three main roles: ensure integrity –initial –private; act as a catalyst –improvement; act as a conduit –qualitative –research –participatory not consumerist.

legitimacy burden engagement self-assessment type student feedback impact longevity improvement Quality process

Type and approach Programme, subject or institutional level? Less important than the purpose of the activity. Dominant approach: self-assessment, peer review, statistical data. Not necessarily seen as the best approach.

Self-assessment Main value of EQM is the internal self- reflection. But two sets of books. Fear of revealing weaknesses.

Engagement Engagement mediated by the perceived, short-term affect. Game playing and compliance. Performance to ensure maximum return. Obscures the reality. No surprise: natural outcome of accountability-oriented processes. Game playing taking up resources for very little real return.

Burden However benign, imposed an unnecessary bureaucratic burden. Time taken to prepare specific event- related documentation. Evaluate on the basis of what institutions already produce.

Legitimacy How far it is supported by academics. Collaborative not something being done to an institution. Help to improve outputs. Some internal-external processes have legitimacy (e.g., external examining and departmental reviews), which external monitoring should not displace.

Improvement Temporary impact. EQM must interact with internal quality systems. Changes in culture –slow –commitment. Event or process: performance and game playing.

Longevity Improvement potential decreases as process becomes more elaborate and routinised? Emphasis shifts to procedural elements rather than innovative process. Need constant reflection on and change in EQM, more trust and collaboration. Periodic change in purposes and in the agencies themselves.

Longevity Without periodic change, there is the danger of ending up with a British-style, QAA-type, system: a rolling juggernaut, that is not sure what it is looking for, but which ensures compliance and minimizes innovation and risk- taking. British institutions continue to comply, even if the return on the investment is derisory, because of the fear of loss of income.

Efficiency Doubts about the efficiency of most EQM. Cost (external and internal) in no way reflect the value gained from the process. Little engagement with the internal improvement process. Periodic events do not help inform change management. EQM inhibits innovation through its conservative or rigid evaluation criteria.

Impact on learning Extreme sceptical that EQM had any impact on programme quality or student learning. No evidence of clear impact on learning –available research suggests that other factors outweigh the impact of EQM.. Structure and organisation of EQM is not compatible with empowering staff and students to enhance the learning situation.

Student feedback Important element of quality monitoring. Link feedback to action and empowerment. Over-evaluation of students, unlinked to action, leads to questionnaire fatigue and cynicism by both students and staff. Too much emphasis on a consumerist approach to feedback rather than a participative approach?

Theme 2 Has control of quality been usurped by the market and by information technology?

Market and IT HE business: infinite Internet providers? boundaried location for learning is obsolete? the market will arbitrate on quality? monitoring procedures stifle creativity? EQM is about restricting the number of providers?

Quality monitoring and gold standard The market will not adequately monitor quality –at the very least need accreditation (e.g., US) –the market is not self-regulating –competition will not ensure the integrity of HE. Underlying gold standard notion of HE (international) HE encapsulates a certain type of learning experience; a degree has a universal structure and integrity.

Impact of the market The market has had limited impact despite predictions (US apart) There will not be a higher education market that will determine quality. Some aspects of a market will impact on HE either to encourage improvement or inhibit it. Despite their interconnections, distinguish between the impact of IT and of the market.

Public versus private Important issue in some countries. (Negative) impact on standards of fast growth in private sector. Cherry-picking –public universities:basic theoretical learning –private universities: lucrative (post- graduation) vocational skill development.

Pseudo-market conditions Competition and efficiency –contingent funding, e.g., performance –lowering standards. Diverse income streams. Compliance –government agendas –distortion. Diversity (and equivalence) –transformative potential for more students –possibly drives down standards.

Consumer choice Internet –market challenge (but not distance) –international QM –integrate and enhance existing provision. Student choice in learning style. Marketable skills development.

Theme 3 Does the development of mass education necessarily mean the end of quality?

Mass higher education standards threatened by resource cuts and students doing more paid work? student outcomes are different not worse? employer demands have changed? grade inflation (especially at the bottom end) ? external pressures to increase pass rates? EQM is concealing the drop in standards and the reduction in the unit of resource?

Mass higher education increase the general level of education, skills and abilities; recruitment of less well qualified students; necessitating greater value added to avoid decline in standards; most governments are not prepared to fully fund expansion of higher education; resulting in access and equal opportunity issues.

Value-added It is unfortunate that so little advance has been made in developing value-added approaches or measures. Although it might be difficult to measure student progress in any quantifiable way one suspects that there was no political will to develop value-added, not least because it might turn conventional league tables upside-down.

Student experience Massification gives rise to concerns about the quality of the student experience. Lowering of service standards to students Fewer resources, less staff time. Less formative assessments. More paid work, less peer interaction. Different rather than worse experience. More emphasis on explicit attribute development.

Standards Little support for the notion that standards in HE are declining. HE has changed and that current and past standards cannot be compared. Value-added is improving. New era in HE, which needs new methods of evaluating standards that provide for continuous assessment and allow for assessment of skill development

Quality as agent for change EQM as agent of change or vehicle for government policy. Teaching ability is now expected: staff development, to enhance teaching ability. Quality processes as a means to change curricula and priorities: –responsiveness to external factors. Biggest problem of quality reviews is legitimation of declining resources.

A Quality Manifesto Academics and students of the world unite and reclaim the quality agenda…..

Thank you