Developing an effective assessment strategy Peter Hartley, Professor of Education Development University of Bradford

2 This session introduction whats the problem? introduce current projects and initial outcomes: the PASS and TESTA projects identify important issues, suggestions and implications

3 Introduction myself good and bad news!

4 Introduction myself good and bad news! –The bad news – the jury is out! –The good news – growing evidence and examples from new projects and initiatives with the opportunity to build on and extend others experience

5 Whats the problem? #1 Consider the issues buried in the title of this paper –the implications of developing –effective for whom? –clarifying the purposes of assessment –the notion of strategy

6 Whats the problem? #2 See the PASS Issues Paper –Please comment/feedback and use Would highlight: –Assessment drives and channels –What/why are we measuring: the slowly learnt –Limitations of grading systems (e.g. marks are not numbers) –Implications for course structures/regulations –

7 Introducing the projects PASS project #1 PASS project NTFS group project over 3 years –One year of development and investigation and two years of implementation Consortium –Led by Bradford –2 CETLs – ASKE and AfL –Plus Exeter, Plymouth and Leeds Met.

8 PASS project #2 Outcomes and outputs –approaches to PBA –choice and consequence guides –workshop and resources for local implementation –case studies from different disciplines

9 TESTA projectTESTA project, #1 NTFS group project with 4 partner universities aims to improve the quality of student learning through addressing programme- level assessment.

10 TESTA projectTESTA project # 2 starting from audit of current practice, so far looked at nine programmes surveyed students using focus groups and AEQ – Assessment Experience Questionnaire – Graham Gibbs et al also using tool to identify programme level assessment environments (Gibbs)

11 Issues to disentangle include: Defining assessment Assessment environments and their impact Defining programme-based assessment Student perceptions and expectations The need for a strategic approach Grading and credit

12 Defining assessment: a challenge –program outcomes need to be assessed in complex, multidimensional student performances –Multidimensional performance entails the whole dynamic nexus of the individuals intentions, thoughts, feelings, and construals in a dynamic line of action and his or her entanglement in an evolving situation and its broader context. Such a context may be within or across work, family, civic, or other settings. –(Rogers, Mentkowski, & Reisetter Hart, 2006, p. 498).

13 Assessment environment and impact Interim findings from TESTA –variety of assessments can cause problems –Issues over understanding assessment criteria, marker variation, and feedback –variation across programmes –QA myths and traditions can get in the way

14 Starting to define PBA

15 Typical student perceptions and concerns (based on PASS) perceptions of the course are variable. assessment experienced as fragmented. anxieties re move to more integrated assessment – perceived risk in terms of performance. Concerns about feedback and timing.

16 The need for strategy An example finding from Gibbs –greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer And what did make a difference?

17 The need for strategy An example finding from Gibbs –greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer And what did make a difference? Formative-only assessment More oral feedback Students came to understand standards through many cycles of practice and feedback

18 An example: Peninsula Medical School NB Case study forthcoming from PASS Includes: four assessment modules that run through the 5 year undergraduate medical programme and are not linked directly to specific areas of teaching focus on high-quality learning (Mattick and Knight, 2007)

19 Issues re grading and credit Teaching/assessment links The assessment/credit link Credit cf accreditation Threshold cf grades in performance

20 And back to effective assessment strategy

21 And back to effective assessment strategy Will it explain to staff, students and external agencies: –How the course/programme assesses the main outcomes? –How assessment and teaching are linked? –How assessment both supports high-quality learning and develops it over the course?

22 And finally … Other initiatives and events, e.g. –EARLI conference later this year –Growth of work on feedback (e.g. use of audio as in ASEL)

23 And absolutely finally … Contacts for PASS The project website Project Director: Peter Hartley Project Manager: Ruth Whitfield