Programme Specification, Benchmarks etc. Warren Houghton School of Engineering and Computer Science, University of Exeter

Context of examples University of Exeter – mid ranking “old” university – “research lead” Department of Engineering –in School of Engineering and Computer Science –small general engineering department –26 full time academic staff –approx 400 U/G students

Context of examples 3-yr general BSc 3-yr BEng accredited ElectronicMechanicalCivil Eng. & Management 4-yr MEng accredited ElectronicMechanicalCivil Eng. & Man Common first year

Constructive Alignment Intended Learning Outcomes Learning Activities Assessment methods and criteria John Biggs: Teaching for Quality Learning at University, Society for Research into Higher Education and Open University Press 1999

How we wrote Programme Specifications 1.Put Subject Benchmark Statement to one side ! 2.Wrote aims and ILOs for existing programmes –many iterations - emergent outcomes 3.Wrote aims and ILOs for existing modules –many iterations –drawing out what staff were already doing 4.Then, checked against Benchmark Statements etc. 5.Did not try to achieve one-to-one mapping

Why not use Benchmark Statements as “blueprints”? What authority should we give the Benchmark Statements ? –What does the QAA say ? Is there a “correct answer” ? How can we obtain a set of required ILOs ? –From industry? Do we have to take responsibility, with our own ideas?

William Perry’s positions: 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

What position do we take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

What position are we encouraged to take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

NOT as some definitive “right answer” We have to take responsibility for creating/recreating the curriculum –drawing on –our own experience –others’ experience - set out in benchmarks etc. an iterative, reflective, process How should we use benchmarks?

Curriculum development Emergent outcomes Rewrite Programme Specification Implement changes Read available literature. Benchmark Statements etc. Contribute to national discussion Compare curriculum with Benchmark Statements etc. Articulate current aims, ILOs etc. Curriculum experienced by teachers and students

Threshold standards

benchmarking implies... all graduates will meet all threshold standards we need to show how we may have to change our assessment QAA(2000) Engineering Benchmark Statement.

The EPC Engineering Graduate Standard 26 “ability to” statements under 7 headings Ability to exercise Key Skills... transform existing systems into conceptual models transform conceptual models into determinable models use determinable models to obtain system specifications... select optimum specifications and create physical models apply results from physical models to create real target systems critically review real target systems and personal performance

The EPC Engineering Graduate Standard “The A2 statements set out what all engineering graduates should be able to do and the intention is that graduates should be able to provide evidence of all 25 abilities to a greater or lesser extent - there are no options or choices.”

Threshold standards Effective/easy in training ( e.g. the Royal Navy ) Different in HE – why? –“Education and training are different” –Is certification realistic / useable ? PDP offers a solution to both problems

setting and assessing threshold standards in core academic modules Example: setting and assessing threshold standards in core academic modules in Engineering at Exeter

Traditional examination 3 hr paper Choose 5 out of 8 questions Pass mark 40% Pass provides evidence of 25% of ILOs tested But which 25% ? What can we build further learning on ?

define detailed ILOs/assessment criteria... For all 1 st and 2 nd year engineering modules

Module specification – A/B structure DETAILED LEARNING OUTCOMES / ASSESSMENT CRITERIA List A comprises core outcomes that will be covered fully in lectures and must be achieved by all students to meet the minimum requirement for progression. List B comprises outcomes that are EITHER more difficult to achieve OR are to be achieved by private study (or both). A: THRESHOLD LEVELB: GOOD TO EXCELLENT. Apply nodal analysis, with step by step prompting, to 2 loop circuits.. Apply nodal analysis, without guidance or prompting, to 2 and 3 loop circuits..

Assessment - examinations Paper A: –Covering list A ILOs only –Typically, short straightforward questions –No choice –Expected mark >80% –Criteria referenced Paper B –All ILOs, and some choice –Longer, more challenging questions with no “easy” parts –Expected average < 40%

Doesn’t this approach mean that we are blatantly teaching to the examination? Yes !

Development Accepted because of PEI accreditation –evidence that students with different marks had achieved identifiably different learning outcomes Originally developed as part of a scheme to give better guidance to students

Impact on staff Staff find it hard to split ILOs this way BUT asking for differentiated ILOs seems to work better than just asking for single level ILOs

Realism Any test is demanding when the pass mark is 80% If we are genuinely going to test all ILOs they must be achievable. We have to be honest.

Deep vs. surface learning Are we encouraging surface learning? Other factors enable deep learning... Consider structure of the learning (noun) –Hierarchy of concepts What happens if students try to understand complex concepts when they haven’t grasped the components? A/B approach makes deep learning possible

a problem of success ?

Supporting students A/B split originally introduced for student guidance students asked to identify progress against ILOs on weekly basis now whole of 1 st year ILOs are a prerequisite to PDP

How do we achieve alignment of: with How academic staff think about research in their disciplines How academic staff approach curriculum development with How we want academic staff to approach curriculum development How academic staff are managed ?