Towards a diversified review model for FE colleges First and second thoughts with no formal status! Dr Iain MacRobert and other HMI thinking out loud!

Aims and objectives: carried forward from the current model Support and develop quality improvement in colleges to provide the best possible experience for the learner Support and develop quality improvement in colleges to provide the best possible experience for the learner inform the council's funding allocations and policy development inform the council's funding allocations and policy development provide assurances of value for money for public funders provide assurances of value for money for public funders provide information to the public on the quality of the colleges and educational programmes provide information to the public on the quality of the colleges and educational programmes

Aims and objectives: in addition Inform prospective students of the quality of the college's educational programmes and services Inform prospective students of the quality of the college's educational programmes and services provide fast and detailed information for college managers and staff to improve the quality of the programmes and services they provide provide fast and detailed information for college managers and staff to improve the quality of the programmes and services they provide provide a breadth and depth of review that is proportionate to the level of confidence that the college has effective quality management in place for all significant aspects that impact on the quality of the students' experience provide a breadth and depth of review that is proportionate to the level of confidence that the college has effective quality management in place for all significant aspects that impact on the quality of the students' experience ease the external review and audit burden on colleges. ease the external review and audit burden on colleges.

Three key assumptions Not all Elements QIs are of equal value - some (eg ELD, Guidance and support, QA and QI) are key independent variables that determine the quality of other elements. Not all Elements QIs are of equal value - some (eg ELD, Guidance and support, QA and QI) are key independent variables that determine the quality of other elements. A key output is student achievement for which retention and attainment data often provide a reliable proxy. A key output is student achievement for which retention and attainment data often provide a reliable proxy. That the key process is learning (and all that facilitates it including guidance, teaching, materials, support etc.) That the key process is learning (and all that facilitates it including guidance, teaching, materials, support etc.)

Two phases: three levels of review First phase (December to March) First phase (December to March) –Core review (all colleges) Second phase (April to June) Second phase (April to June) –proportionate to the strengths and weaknesses identified during the Core Review –proportionate to the size and complexity of the college

First phase: Core (diagnostic) review Obtain documentation from college - similar to and no more than that currently requested: Obtain documentation from college - similar to and no more than that currently requested: –Context statement –Strategic plan –Operational plans for all teaching departments and student support sections –Recent reports by other bodies carrying out external evaluations of relevant provision –Self-evaluations and/or reviews for all educational provision and student support functions produces routinely by the college

First phase: Core (diagnostic) review Obtain documentation from college (continued): Obtain documentation from college (continued): –Targets and KPIs (including PIs) for all areas –College procedures for planning, line management, internal communication, programme reviews, student guidance and support, SD&CR, accommodation and resources etc.. –Records of programme reviews, reviews of services for students, student guidance and support, staff development etc.. –and completed questionnaires from a sample of taught, flexible, distance and online learning students

First phase: Core (diagnostic) review Evaluate documentary evidence (college review team) Evaluate documentary evidence (college review team) –against B Elements ELD (including flexi learning strategies and resources), Access and inclusion, Guidance and support, QA and QI, and A Element Student achievement (revised and extended) –identifying potential key strengths and weaknesses –using 3 reviewers for 5 days (15 days) with a senior member of college staff (Depute / senior AP to assist with interpretation)

First phase: Core (diagnostic) review Visit the college (college review team): Visit the college (college review team): –to gather additional evidence to confirm, deny or modify key strengths (and good practice) and weaknesses provisionally identified from documentation (3 reviewers for 2 days [6 days] but up to a maximum of 3 reviewers for 5 days [15 days]) and

First phase: Core (diagnostic) review Visit the college (subject review team): Visit the college (subject review team): –to evaluate Learning and teaching process (revised to include first-line guidance QIs, learning resources - T&L materials, classrooms, workshops, and formative assessment) sampling viable subject areas with: –5 reviewers for 5 days (25 days) normally comprising an HMI MI, 3 subject specialists and one flexible learning reviewer.

First phase: Core (diagnostic) review Visit the college (subject review team): Visit the college (subject review team): –evaluating: viable subject area with highest retention and attainment rates viable subject area with highest retention and attainment rates viable subject area with lowest retention and attainment rates viable subject area with lowest retention and attainment rates subject area with largest volume of provision subject area with largest volume of provision a cross-curricular sample of flexible (open, distance, work-based, community-based, online etc.) provision a cross-curricular sample of flexible (open, distance, work-based, community-based, online etc.) provision or other subject areas broadly meeting these criteria, as identified in discussion between the MI and college senior managers or other subject areas broadly meeting these criteria, as identified in discussion between the MI and college senior managers

Issues that we need to address in relation to Learning and teaching We tend to confuse learning outcomes with learning processes We tend to confuse learning outcomes with learning processes Grades awarded at present tend to give administrative and planning QIs equal weight with learning Grades awarded at present tend to give administrative and planning QIs equal weight with learning The order of QIs and prompts does not prioritise learning The order of QIs and prompts does not prioritise learning Reviewers looking at more student work over time would help to evaluate progress in learning. Reviewers looking at more student work over time would help to evaluate progress in learning.

Issues that we need to address in relation to Learning and teaching The role of assessment in learning and teaching needs to be more fully recognised. The role of assessment in learning and teaching needs to be more fully recognised. QIs need to be applicable to all forms of learning (classroom, workshop, community- based, work-based, resource-based, flexible, distance, open, online and blended) and address learning materials accordingly. QIs need to be applicable to all forms of learning (classroom, workshop, community- based, work-based, resource-based, flexible, distance, open, online and blended) and address learning materials accordingly. The Teaching and learning element needs to be revised and extended. The Teaching and learning element needs to be revised and extended.

Perhaps a new Element A5: Learning, teaching and student outcomes A5.1Learning process A5.1Learning process A5.2 Teaching process A5.2 Teaching process A5.3 Student (or learning) outcomes A5.3 Student (or learning) outcomes With a single evaluation grade With a single evaluation grade

First phase: Core review Draft report in bullet points identifying strengths, weaknesses and MPsfA (7 days - one per reviewer) Draft report in bullet points identifying strengths, weaknesses and MPsfA (7 days - one per reviewer) Issue draft report within 15 working days to principal and senior college management, present findings and 'agree' volume and scope of the subsequent follow through review Issue draft report within 15 working days to principal and senior college management, present findings and 'agree' volume and scope of the subsequent follow through review

First phase: Core review Criteria for determining the scope of follow through reviews could be the levels of confidence that: –educational provision is well designed, planned and managed, and responsive to community, student, employer, economy and stakeholder needs –programmes and all services to students are evaluated systematically, major weaknesses are few and being addressed, and improvements being made –the quality of learning and teaching is being evaluated and improved

First phase: Core review Criteria (continued) for determining the scope of follow through reviews could be the levels of confidence that: –demanding but realistic targets (KPIs) are being set and largely achieved –student retention and attainment is high across almost all subject areas and good or very good in the subject areas reviewed –learning, teaching and related student support is good or very good in the subject areas reviewed

Second phase: Follow through review Proportionate. For example: –Exploring in more detail the causes of strengths and weaknesses without extending the scope of the Core review –Extending the evaluation of Learning and teaching to 6 subject areas (3+1+3) –Evaluating against other A and B Elements as required (based on the evidence from the Core review) –Extending the evaluation of Learning and teaching to 7 or 8 subject areas –Evaluating against all A and B elements.

Reports For college managers and staff: strengths, weaknesses, grades and MPsfA in bullet point format and FE jargon issued to the principal within 15 working days of the review For college managers and staff: strengths, weaknesses, grades and MPsfA in bullet point format and FE jargon issued to the principal within 15 working days of the review Core and follow through report from c4K to c11K words in the subject section and c 3.5K to c8K to in the college section Core and follow through report from c4K to c11K words in the subject section and c 3.5K to c8K to in the college section For public accountability: a report of up to 2K words (for a large follow through) in plain English (similar to our current summaries which answer the key questions) For public accountability: a report of up to 2K words (for a large follow through) in plain English (similar to our current summaries which answer the key questions) For prospective students of the college: 400 to 600 words in plain English to be published by the college For prospective students of the college: 400 to 600 words in plain English to be published by the college

Questions Is a Core review a suitable transparent model for deciding on the differentiated review to follow? Is a Core review a suitable transparent model for deciding on the differentiated review to follow? –Can HMIE deliver it? Will colleges buy into it? Will SFEFC be happy with it? What are the implications for a differentiated second phase? What are the implications for a differentiated second phase? –How will this impact on HMI-college relationships? –What other major issues will this throw up?