1. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 1 Contextualising assessment: the lecturers’ perspective Lee Shannon, Bill Norton, Lin Norton, Frances Phillips and Katherine Harrington* Liverpool Hope University, UK. *London Metropolitan University, UK

2. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 2 Outline Context and research literature Aims of the research Research process and methodology Analysis of findings Conclusions and further research

3. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 3 Context of the research Widely held view that assessment should be for rather than of learning (Birenbaum et al, 2005) View that assessment in HE manifests many poor practices (Rust, 2007) Relationship between lecturers’ pedagogical beliefs and assessment practice remains under- researched area ( exception Samuelowicz & Bain, 2002) Weston Manor Assessment manifesto (ASKe November 2007)

4. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 4 Theoretical background ‘Ideal’ versus ‘Actual’: disjunction between beliefs and practices is one of the mysteries of HE. (Murray and MacDonald, 1997) ‘Espoused Theory’ versus ‘Theory in Use’ (Argyris and Schon,1974) Cognitive Dissonance (Festinger 1957)

5. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 5 Research aims To elicit lecturers’ perceptions of assessment within the broader context of their philosophy of learning and teaching, specifically: ideal conditions for student learning marking feedback emotional issues related to assessment relationship between lecturers’ past experiences and current practices

6. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 6 Methodology In depth semi-structured interviews 29 lecturers in 18 disciplines at four universities in the UK Assessment experience ranged from 1-22 years. Interviews were transcribed verbatim Thematic analysis using QSR NVivo 7

7. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 7 Interviewees’ disciplines Archive and Record Management Biology Business Cognitive Neuroscience Computer Studies Education English Film Studies Geography History Human Resource Management Media Studies Medicine Political Science Psychology Religious Studies Sociology Theatre Studies

8. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 8 Representation of the themes Next five slides represent our findings visually Not a model but a colour coded representation of the main themes where: –Yellow is the main focus of the interview –Blue represents themes where there is a dichotomy –Green represents the ‘ideal’ assessment practices –Red represents the ‘actual’ assessment practices

9. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 9 Relationship between pedagogical philosophy & assessment practices Autonomy to change delivery only No autonomy to change assessment Institutional requirements Changing with experience Autonomy to change assessment Complete alignment In an ideal world Personal philosophies Relationship

10. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 10 Strategic Signposting Range of students Motivational Fit for purpose Pedagogically appropriate Aids learning Focus on grades Assessment for learning Assessment for learning

11. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 11 Mental models Marking processes Marking & assessment criteria Highly regulated Students’ understanding of criteria Criteria useful for lecturers Criteria useful for students Connoisseurship Marking

12. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 12 Feedback Power relations Motivates Allows self reflection Timely feedback Time with students The unmotivated Negative aspects for lecturers

13. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 13 Emotional issues related to assessment Self doubt Non supportive colleagues Negative aspects for lecturer Positive two-way relationship Rewarding for lecturer Supportive colleagues Power relations Emotional issues

14. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 14 Development of Assessment, Marking & Feedback Skills Models of good practice Experience Informal Training A.M.F

15. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 15 Conclusions Lecturers in this study articulated a personal pedagogical philosophy of learning, teaching and assessment, but …. they gave competing accounts of the processes associated with assessment, marking and feedback. This finding supports the ‘ideal’ versus ‘actual’ dichotomy of Murray & Macdonald (1997) and the ‘espoused theory’ versus ‘theory in use’ of Argyris and Schon (1974).

16. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 16 Implications For researchers: –The interview process required lecturers to articulate what is often tacit, so may have engendered some cognitive dissonance which will be further researched. For staff developers: –Despite recognition by lecturers that assessment can be problematic, there was still a collective resistance to formal training in assessment design, marking and feedback.

17. 27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 17 References Argyris, C. and Schön, D. (1974) Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass. Assessment Standards Knowledge Exchange (ASKe) http://www.brookes.ac.uk/aske/ (accessed 14 february 2011) http://www.brookes.ac.uk/aske/ Birenbaum, M., Breuer K., Cascallar E, Dochy, F. Dori, Y. & Ridgway, J. in Wiesemes, R & Nickmans, G.(eds) (2006) position paper; A learning integrated system Educational Research Review 1 (2006) 61–67. Avaialble electronically: http://edu.technion.ac.il/chemical- education/judy/publications/no12_Assessment%20for%20learning%202006.pdf (accessed 14 February 2011)http://edu.technion.ac.il/chemical- education/judy/publications/no12_Assessment%20for%20learning%202006.pdf Festinger, L. (1957) Theory of cognitive dissonance. Stanford University Press Murray, K. & Macdonald, R. (1997) The disjunction between lecturers' conceptions of teaching and their claimed educational practice. Higher Education, 33, 331 - 349. Rust, C (2007) "Towards a scholarship of assessment" Assessment & Evaluation in Higher Education 32, 2, 229-237 Samuelowicz, K & Bain,J.D. (2002) Identifying academics’ orientations to assessment practice, Higher Education, 43,2, 173-201