Online assessment: who collaborates with whom and what is the benefit Mike Mimirinis Faculty of Health, Social Care and Education Anglia Ruskin University, Cambridge Digital University SRHE network event University of Edinburgh, 14 June 2019

Studies on conceptions of assessment Entwistle (2000): limited evidence that contrasting conceptions of teaching, tend to hold corresponding views on assessment. Samuelowicz & Bain (2002), Postareff et al. (2013): continuum ranging from an emphasis on knowledge reproduction to an emphasis on knowledge construction and/or transformation.

Phenomenographic studies on conceptions of assessment, blended learning, online discussions & e-teaching Watkins et al. (2005): relation between teaching and assessment, and the focus of the backwash effect of assessment Students’ conceptions of, and approaches to online discussion (Ellis & Goodyear, 2010; Ellis et al., 2007). Ellis et al. (2006): conceptions of blended learning that focus on the use of technology as a means of achieving learning outcomes are associated with conceptions of blended learning that prioritise students’ construction of meaning. González (2009, 2010): teachers’ conceptions of e-learning in four categories of description- (1) to provide information to students, (2) for peer-to-peer student communication, (3) to engage students in online discussions, and (4) to support knowledge-building activities.

What are the qualitatively different ways in which academics understand and experience e-assessment?

Overview of participants 21 participants (14 female, 7 male) Years of teaching experience: 1-35 Years of e-assessment/computerised assessment experience: 1-25 3 Profs, 5 Assoc. Profs, 8 Sr Lecturers, 5 Lecturers

after Becher and Trower (2001) SOFT Academic Development Linguistics Education Marketing Criminology Politics Social Work (3) Sociology Management Nursing (2) Music (2) Microbiology APPLIED PURE Architecture Biosciences Fashion Biomechanics Engineering after Becher and Trower (2001) HARD

Interview & analysis Semi-structured interviews (40-70 minutes) Stimulated recall technique (optional) “Pool of meanings”, grouping and regrouping, labelling

Category A e-assessment as a means of managing and streamlining the assessment process “to work, to function, so that my students can access it when they need it, when they need to – which could be 24 hours a day, they work sometimes in the middle of the night. So the role of the technology is to be functional and operational in order to enable my students to complete this task required of them” [P09] “I think e-assessment should allow for things to be quicker, and certainly when we’re doing double marking with my colleagues being able to share it via the shared space means it’s much quicker. Likewise with the external examiners we can share it much more quickly, we’re not sending parcels to each other.” [P05]

Category B e-assessment as a means of promoting engagement and dialogue “The discussion, where they post something up and a small group can see what they’ve done, again means feedback and ideas go to a wider audience than just the one to one.” [P01] “so he can share his and I can share mine if he wants to show me other stuff that he’s been working on on his laptop. So it’s as if we are together and of course the system allows us to be seeing each other at the same time and talking, of course. So it’s as if it’s a face-to-face interaction” [P08] “It should be a system that enables a quick or certainly a time...an accessible time-focused way of communicating the submission and feedback. I’m trying to really articulate this. I’m trying to say that it’s about the relationship for not just summative but ongoing conversations about how the work is progressing. It should be… through e-assessment it should be quick, easy and accessible.” [P05]

Category C e-assessment as a means of enhancing learning “And I think back when we first started off, when I first started off, yeah I did look at the two in very distinct ways because it was very different and people were trying to come up with new, imaginative ways of doing that rather than fundamentally looking at what’s best for the assessment, what’s best to meet the learning outcomes and what’s best for the student. And I think happily we’re now, or certainly within my department, we’ve moved towards that medium now where we’re just talking about it as another tool. It’s not necessarily ‘e’, it’s just, “Well for that, we’ll do that. For that, we’ll do this because that’s appropriate.” [P06]

Category D e-assessment as a means of community- and (digital) identity- building “And then I guess in the wider community, e-assessment, you know it’s been successful if we have a good reputation, our students get jobs, they stay in their jobs, they contribute as citizens; they may not be doing that in acts of heroism but they are doing it as citizens so they’re voting. So there are philosophical ways to look at this in terms of a university’s success: in terms of engaging with society productively" [P07] “We were hoping to achieve a sense of community, a development of the relationship between the students who are new, who don’t necessarily know one another on the module” [P07]

Outcome space: referential and structural aspects of academics’ conceptions of e-assessment Referential (‘what’ of the conception) Structural (‘how’ of the conception) Technology perspective Learning perspective Collective perspective A Managing and streamlining of assessment Category A B As in (A) and promoting engagement/dialogue Category B C As in (B) and enhancing learning Category C D As in (C) and identity- and community- building Category D

Dimensions of variation The role of the teacher/assessor The role of the student/assessed The role of the medium The benefit of e-assessment The relationship between e-assessment and teaching-learning The level, purpose and quality of collaboration in e-assessment

Category A Category B Category C Category D   Category A Category B Category C Category D What is the benefit of e-assessment Access-flexibility-organisation-efficiencies Clarity of communication and extending engagement Student skills and development of ideas Professional careers, personal growth, civil engagement, cohort development What the teacher does Administrator- plagiarism detector- safeguard Communicator- conversation partner Facilitator of learning- Guide Mentor What the student does Recipient of results, feedback, system user Communication partner Active learner- partner in the assessment process Develop within cohort, online collective spaces/ society What is the role of the medium Transactional Communicative function Instrumental to teaching and learning Instrumental to the development of the individual within cohort, profession, society What is the Purpose (P), Quality (Q) and Level(L) of Collaboration P: Supporting administration Q: Transactional L: Low and mostly uni-directional P: Supporting dialogue Q: Relational L: High and multi-directional P: Supporting learning, development of ideas and skills P: Supporting collective processes What is the relationship to teaching-learning External Internal

Dimension of Variation: the relationship between e-assessment and teaching-learning Internal Category D Internal External Category C External Category B Category A

Discussion e-Assessment can be seen as internal or external to teaching-learning (cohesive vs fragmented conceptions of e-assessment) Conceptions represent incrementally advanced agency within (or against) e-assessment structures Teachers engaged in same e-assessment task, may think differently about it and therefore achieve qualitatively different outcomes, e.g. plagiarism detection (see also Bakkenes et al., 2010).

Reference Mimirinis, M. (2019). Qualitative differences in academics’ conceptions of e-assessment. Assessment & Evaluation in Higher Education, 44(2), 233-248. DOI: https://doi.org/10.1080/02602938.2018.1493087