Principles for the regulation of e-assessment An update on developments Andrew Boyle Regulations and Standards division QCA

Statutory body in England, sponsored by DCSF Functions: –Maintains and develops the national curriculum and associated assessments –Regulates qualifications offered in schools, colleges and workplaces Regulatory role covers all qualifications except those awarded by HEIs Role is restricted to England –Regulates quals jointly with partners in Wales and NI –Works closely with counterpart in Scotland Role of QCA

QCA approach to regulation Proportionality (interventions are related to risk) Accountability (the public has a right to see what QCA does) Consistency (in judgements made; in data requested; in criteria used) Targeting (measures taken related to purpose) Transparency (openness and visibility)

Regulatory principles for e-assessment Published by QCA & regulators in Wales, NI & Scotland –Consultation Oct 06 – Feb 07 –Enforceable Oct 07 via monitoring activities –Fully enforceable April 08 via self-assessment

Structure of presentation Aims of principles Background issues that impact on e-regulation –Few papers addressing regulation of e-assessment specifically –Standards docs, etc. –Literature from diverse fields Discussion –Understanding nature of regulation for e-assessment –Describing further work to be done

Aims of principles Robust e-assessment strategy and operations Guide operations and innovation in consistent way Extend access to e-assessment for benefit of learners Identify and address parameters for success and areas at risk for innovative e-assessment strategy Ensure all regulation: –allows for flexibility –promotes and guides innovation –maintains integrity, reliability and validity Principles designed to: –maintain public confidence –support ABs who wish to innovate and add value through use of technology

Literature directly describing regulation of e-assessment Previous QCA reports –Code of Practice inappropriate for e-assessment  ‘Tends to stifle innovation’ –Develop ‘higher level principles’ –Facilitate dissemination of good practice to ABs Exams on Demand predictions – 2004  2014 –Drivers of e-assessment uptake  Regular review of regulation to maintain relevance and flexibility  Good quality e-learning materials support personalised learning  Technical, physical and human resources in schools and colleges  Organisational structures in schools and colleges to support personalised learning and on-demand assessment  Greater adoption of MCQs as a valid assessment method  Agreed standards for data held in e-portfolios

e-assessment standards, Codes of Practice, etc. Sets of documents –ISO23988: International code of practice for use of IT in delivery of assessments –ITC International Guidelines on CB- and Internet-Delivered Testing –ATP Guidelines for CB Testing Issues around standards –Scope –Audience –Enforceability –Wording –Granularity of detail –Relationship with other sets of standards –Vision of education quality

Guidance documents Include: –QCA, DELLS and CCEA’s: Guide to effective practice in e-assessment –SQA’s guidelines on e-assessment for schools –SQA’s guidelines for FE –NAA’s effective guide for staff implementing the KS3 ICT test Tend to: –Be aimed at learning providers –Give advice about how to run e-assessment sessions

Other relevant topics History and current trends in regulation UK government approach to regulation QCA approach to regulation New-style regulation as a transformation of law and decision- making Regulation of transformed industries

Discussion Need better factual info in several areas Light regulation might not lead to widespread adoption –Ways in which pre-existing conditions are non-optimal –Regulator helps create circumstances conducive for innovation No ‘one-size-fits-all’ solution that will facilitate e- assessment uptake –Lack of agreed opinions suggest niche solutions –High-stakes, high-volume qualifications present particular issues

Further work e-strategy project –Contractor working to define e-strategy for QCA Research projects –International approaches to (e-)regulation –Indices for uptake of e-assessment  Continuing to develop the most representative indices  Seeking context for indices – e.g. from adoption of other ICT innovations  Using theory to try to understand adoption (e.g. diffusion of innovations) –Technical group re e-assessment in general quals