Helping psychology undergraduates improve examination writing Kathy Harrington, Mercedes Freedman, Savita Bakhshi, Peter O’Neill London Metropolitan University Promoting Psychology Student Learning through Assessment 27 March 2009 London Metropolitan University HEA Psychology Network Write Now Centre for Excellence in Teaching & Learning

2 Overview of presentation  Aims and context of study  Pedagogical rationale  Description of the intervention  Method  Findings  Conclusions and implications

3 Study aims  Improve students’ examination writing and performance  In addition, we wanted to… Build on previous research on use of assessment criteria to improve student learning Develop and evaluate an intervention based on collaboration between psychology academics and writing specialists (Writing in the Disciplines model) Create re-useable teaching & learning resources

4 Context of study  2 nd -year cognitive psychology module  Assessment 100% by examination  Traditionally lower than average pass rate Identified by university as a “killer module” Plan of response required  Write Now CETL works with discipline-based academics to improve student learning through curriculum and teaching development  Write Now CETL runs university Writing Centre staffed by trained student peer mentors in academic writing Collaborative, non-directive, supportive Enable students to take responsibility for own work

5 Pedagogical rationale  Students and tutors often interpret meanings of assessment criteria differently (Harrington et al, 2006; Lea & Street, 1998; Merry et al, 1998)  Providing clear and explicit criteria is a first step in helping students understand what tutors are looking for in written work  However, research has also shown that facilitating students' active engagement with the criteria is necessary if learning and performance are to be demonstrably enhanced (Price et al, 2001)  Structured interventions focussed on understanding and demonstrating assessment criteria have been shown to lead to improvements in student learning and performance (Norton et al, 2005; Rust et al, 2003)  Other research has found that students value being able to talk to peer tutors about their writing, and that psychology students prefer peer tutors from their own discipline (Bakhshi et al, in press)

6 The intervention  4 hour-long compulsory workshops embedded as part of module teaching across the autumn semester  Immediately following two-hour lectures  Delivered in alternate weeks, with workshops run by lecturers in between (focussed more explicitly on lecture content)  Designed by team of academic writing specialists, psychology lecturer, psychology PhD student with experience of pedagogical research in area of student writing and assessment  Delivered by academic writing specialists and peer writing mentors studying psychology (3 rd -year and PhD students)

7 The intervention (continued)  Exam answers posted in VLE prior to workshops for students to read and give a grade  Set of accompanying materials developed using extracts from authentic examination answers annotated with comments in relation to assessment criteria Specific focus on cognitive psychology Use of departmental assessment criteria  In workshops, materials used to facilitate discussion about demonstrating assessment criteria at different levels of performance  Students guided in small groups to adopt role of examiner and apply criteria to whole past examination answers  Final class discussion drew out students’ insights and summarised main points

8 Data collection and analysis  Attendance registers taken at workshops 2, 3 and 4 (not at first workshop)  Questionnaire distributed at last workshop (n=63) Likert scale: students’ perceptions of helpfulness of workshops in relation to  Examination writing  Meeting assessment criteria  Understanding subject matter of cognitive psychology  Examination grades  Analysis using SPSS to produce descriptive and inferential statistics

9 Study sample  N=205 students enrolled on the module who also took the examination  40 students enrolled on module did not take the examination Study population Total students enrolled on module Conversion Diploma64 (31.2%)72 (29.4%) Single Honours93 (45.4%)113 (46.1%) Other/unidentified5 (2.4%)11 (4.5%) Joint Honours43 (21%)49 (20%) Total205 (100%)245 (100%)

10 Findings: pass rate and mean grade Number of students Pass rateFail rateMean final grade %29%47% %38%43.6%

11 Pass rates by degree course Final Grade category (pass or fail) TestValuedf Asymp. Sig. (2-sided) PassFailTotal Conversion Diploma Count57764 Pearson Chi- Square ** %89.1%10.9%100.0% Single Honours Count %58.1%41.9%100.0% Other Count505 %100.0%.0%100.0% Joint Honours Count %25.6%74.4%100.0% TotalCount

12 Attendance at workshops Attendance not recorded for first workshop Attendance highest at second workshop (37.1%)

13 Percentage who attended workshops FrequencyPercentValid Percent Cumulative Percent Valid Total (48.8%) students did not attend any workshops 105 (51.2%) students attended at least one workshop

14 Attendance and pass/fail in examination Pass or fail TestValuedf Asymp. Sig. (2-sided) PassFailTotal Attendance at at least one workshop No Count Continuity Correction ** %51.0%49.0%100.0% Yes Count %72.4%27.6%100.0% Total Count %62.0%38.0%100.0%

15 Attendance and final grade category Figure 1: Percentage of students who attended workshops by grade category

16 Attendance and course studied Figure 2: Percentage of students who attended workshops by course studied

17 Workshop attendance and grades: 1 Final grade Total number of workshops Final grade Pearson Correlation * Sig. (2-tailed).012 N63 Total number of workshops Pearson Correlation.314 * Sig. (2-tailed).012 N63 *. Correlation is significant at the 0.05 level (2-tailed). Correlation between total number of workshops attended and final grade

18 Workshop attendance and grades: 2 Figure 3: Correlation between number of workshops attended and final grade significant positive relationship (r=.314, n= 63, p<0.05) the more workshops attended, the higher the grade

19 Workshop attendance and grades: 3 Figure 4: Total number of workshops attended and mean final grade Mean final grade overall: 43.61% Conversion Diploma: 60.34% Single Honours: 38.8% Other: 58.2% Joint Honours: 27.44%

20 Similar findings in other research  Lusher (2007) Small-group workshops focused on assessment criteria with 3 rd -year health psychology students Significant correlation between attendance and mean examination scores (r=0.254, N=111, p<0.01) Multiple regression showed that performance did not independently predict attendance, so not just a matter of more able students attending workshops

21 Students’ perceptions of workshops  7-point Likert scale: strongly disagree to strongly agree  Mean scores for all items were positive Understanding what assessment criteria are Understanding subject matter of cognitive psychology Achieving a better grade Producing better writing Understanding how to demonstrate assessment criteria

22 Students’ perceptions (continued) ItemQuestionsNMinimumMaximumMean Std. Deviation 5.1 The workshops helped me understand what assessment criteria are The workshops helped me understand what makes a good examination essay in Cognitive Psychology The workshops helped me understand the subject-matter of Cognitive Psychology The workshops will help me achieve a better grade in Cognitive Psychology The workshops will help me achieve a better grade in my other psychology modules The workshops will help me produce better academic writing in general The workshops helped me understand how to critically evaluate in psychology examination writing The workshops helped me understand how to use theories and evidence in psychology examination writing

23 Students’ perceptions (continued) ItemQuestionsNMinimumMaximumMean Std. Deviation 5.9 The workshops helped me understand how to develop an argument in psychology examination writing The workshops helped me understand how to structure a psychology examination essay The workshops helped me understand how to address the question in a psychology examination essay The workshops helped me understand how to demonstrate my understanding of the subject in psychology examination writing The workshops helped me understand how to use appropriate language in psychology examination writing The workshops helped me understand how to identify strengths and weaknesses in my own psychology writing The workshops helped me feel more confident about examination writing in psychology The workshops motivated me to spend more time studying Cognitive Psychology

24 Students’ perceptions (continued)  Highest mean scores for “The workshops helped me understand what assessment criteria are” = 6.16 “The workshops helped me understand what makes a good examination essay in Cognitive Psychology” = 5.92  Lowest mean scores for “The workshops motivated me to spend more time studying Cognitive Psychology” = 4.52 “The workshops helped me understand the subject- matter of Cognitive Psychology” = 4.56

25 Conclusions  Module pass rate lower this year at 62%, compared to 71% in  However, a number of minor changes were made to content and delivery, so comparison across years problematic  In , attendance at the workshops was significantly correlated with higher examination grades  Confounding factor is that more able students are more likely to be attending in first place More tests needed, cf. Lusher (2007)  Students’ who attended perceived the workshops to be helpful for Understanding what assessment criteria are Understanding how to demonstrate the criteria in their own writing Understanding the subject matter of cognitive psychology Achieving a better grade Producing better writing  Difficulty of addressing needs of weaker students, even with “embedded” teaching

26 Implications  Students benefit from explicit focus on writing within disciplinary teaching  Students benefit from talking about their own academic writing with trained peers in same discipline  Importance of embedding teaching of writing and “writing to learn” activities within modules, rather than viewing writing as an add-on skill separate from learning subject matter  Planned changes to module for next year Less distinction between lectures and workshops by identifying 3-hour “teaching blocks” instead, with varied mix of lecture and workshop activities Time to practice writing in teaching sessions Peer review of own writing, facilitated by peer mentors

27 References Bakhshi, S., Harrington, K., and O'Neill, P. (in press). Psychology students’ experiences of academic peer mentoring at the London Metropolitan University Writing Centre, Psychology Teaching and Learning, Spring Harrington, K., Elander, J., Norton, L., Reddy, P., Aiyegbayo, O. & Pitt, E. (2006). A qualitative analysis of staff-student differences in understandings of assessment criteria, in C. Rust (Ed.), Improving Student Learning through Assessment. Oxford: Oxford Centre for Staff and Learning Development. Lea, M. R. & Street, B. (1998). Student writing in higher education: an academic literacies approach, Studies in Higher Education, 23, pp Lusher, J. (2007). How study groups can help examination performance, Psychology Update, 16, 1 & 2. Norton, L., Harrington, K., Elander, J., Sinfield, S., Lusher, J., Reddy, P., Aiyegbayo, O. & Pitt, E. (2005). Supporting students to improve their essay writing through assessment criteria focused workshops, in C. Rust (Ed.), Improving Student Learning: Inclusivity and Diversity. Oxford: Oxford Centre for Staff and Learning Development. Merry, S., Orsmond, P. & Reiling, K. (1998). Biology students’ and tutors’ understanding of a ‘good essay’, in C. Rust (Ed.), Improving Student Learning: Improving Students as Learners. Oxford: Oxford Centre for Staff and Learning Development. Price, M. and O'Donovan, B. and Rust, C. (2001). Strategies to develop students' understanding of assessment criteria and processes, in C. Rust (Ed.), Improving Student Learning - 8: Improving Student Learning Strategically. Oxford: Oxford Centre for Staff and Learning Development. Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning developing their understanding of assessment criteria and processes, Assessment and Evaluation in Higher Education, 35, pp

28