Qualitative research and multimodality Inna Kotchetkova Emma Renold Bambo Soyinka Chris Taylor Bella Dicks QUALITI, School of Social Sciences Cardiff University

Presentation 1 Multimodality Inna Kotchetkova

Multimodality We deal with multimodal texts on a daily basis – every time we read a newspaper, watch television, play a video or computer game, or even read a book. All practices are always multimodal. A theory has to be developed in which that fact is central, and a methodology produced for forms of description in which all modes are described and describable together.

The roots Since early 1970s semiotics had ways of talking about gestural, bodily, animal and other nonlinguistic ways of meaning making (Eco, Sebeok, Barthes, 1977; 1974, 1977)

multimodality –people communicate through: speech, accent, gesture, facial expression, gaze- direction, shape, weight, size, body posture, position & movement, dress (colour, shape, texture, etc.), hairstyle, etc. –environments/objects communicate through: colour, texture, shape, position, opaqueness/light, weight, movement, sound, etc.

modes and media Modes - abstract, non-material resources of meaning-making (e.g. the languages of image, sound, gesture, colour, texture, narrative) Media - specific material forms in which modes are realised (e.g. a photograph, a computer screen, an audio recording, paint, a wall, clothing, a chisel, a poem, a block of wood) Kress, G.and Van Leeuwen, T. (2001). Multi-modal Discourse. London: Arnold.

Communicative potential of modes Colour is also used to convey interpersonal meaning. Just as language allows us to realize speech acts, so colour allows us to realize colour acts. It can be and is used to do things to or for each other, e.g. to impress or intimidate through power dressing, to warn against obstructions and other hazards by painting them orange, or even to subdue people Kress and Leuween (2002) Colour as a semiotic mode: notes for a grammar of colour. Visual Communications. Vol. 1 (3), pp

Many modes through one media: Meaning making in talk In talk, we mobilize language as sounded speech, and we further mean through gestures, posture, facial expression, and other embodied resources such as physical distance, stance, movement or stasis. Iedema, R. (2003)Multimodality, resemiotization: extending the analysis of discourse as multi-semiotic practice. Visual Communications, Vol 2 (1).

recordings as multimodal voice recordings deploy: accentexpression tonepitch languagevolumecadencerhythm still images deploy: colourshape light directionality perspective position framing size

moves from the idea that one medium communicates in a particular way, to the idea that any one medium may employ a number of different modes of meaning Multimodality

from visual methods to multimodal methods common semiotic principles operate in and across different modes modes can be realised in more than one medium Kress, G. and Van Leeuwen, T. (2001) Multimodal discourse. Arnold

Why is multimodality potentially of interest to social researchers?

A quote observing life in educational institutions is my favourite kind of data collection. A pile of documents, a well-written book or a co-operative interviewee may excite others, but I would always prefer to sit and watch something. The data gathered by watching and listening over weeks, even months, are for me the sweetest jams and the most aromatic oils and spices. Sara Delamont (2001: 130)

When we deal with multimodality 1.observations of the field 2.recording of observations 3.analysis of data records 4.representation of research findings N.B. are data recorded, represented, reproduced, reduced, transformed?

Mixing media Mixing media increasingly common in research projects –how do the insights afforded by these different kinds of media differ (in meaningfulness, quality, validity, etc.) –How can the researcher take account of these differences?

Questions When different media conceived in this way are combined together in a single research project, how does the researcher take account of these discrete kinds of communicational value? How to integrate analytic strategies for written records with image-records, for example? Are we talking about the same kind of informational or communicational value? If not, how might these different kinds of value be reconciled?

Hypermedia ethnography project Production and reception of science with/in an interactive science discovery centre How the centre creates environments and spaces for learning How children engage with and receive these environments and spaces The reproduction and performance of science through exhibits and theatre Hypermedia ethnography team: Bruce Mason, Bambo Soyinka, Amanda Coffey, Matthew Williams and Emma Renold

multimedia data-recording Written texts, visual and audio data, documentary data Fieldnotes Interview transcripts Video recordings: montages / raw footage Audio recordings and soundscapes Still photography Documents and graphics

Multimodality in the field todays semiotic landscape increasingly complex designed environments ubiquitous media increasingly mixed

all the modes employed contribute to meaning… science centre exhibits hall

communicative practice involves… Discourse: which discourses are in play in the environment (e.g. childhood as simple & fun; education as entertainment; science as individualised, discrete operations….) Design: what semiotic resources have been selected as most able to articulate the discourses in play (e.g. objects rather than words; interactive not immobile objects.)?

continued… Production : the selection of media, or material resources: material matter adds a level of extra meaning (e.g. photos v. paintings; solid, opaque objects rather than transparent gadgets) Distribution : technological means of communicating e.g. across distances Interpretation : how meaning is re-made by people interacting with environments

framing: a multimodal principle framing works across diverse media: e.g. in interior design; in sound compositions, in paintings. introduces breaks and discontinuities; contrasts and continuities; uses framelines and empty space

framing childrens behaviour

Questions to consider How can multimodal data help us understand the interaction between people and environment in ethnography? How should we capture and represent multimodality in/of the learning environment? Which medium should we use to record our observations? What do different modes of data representations afford? When different media are combined together in a single research project, how does the researcher take account of the discrete kinds of communicational value? How to integrate analytic strategies for written records with image-records, for example? Are we talking about the same kind of informational or communicational value? If not, how might these different kinds of value be reconciled?

Useful references Delamont, S (2001) Fieldwork in Educational Settings. Methods, Pitfalls and Perspectives. London and NY: Routledge. Wolcott H (1981) Confessions of a trained observer, in T S Popkewitz and B R Tabachnick (eds) The Study of Schooling. New York: Praeger pp Kress, Gunter and Theo van Leeuwen (2001) Multimodal Discourse: The Modes and Media of Contemporary Communication. Hodder and Stoughton Publication Kress, G. and Van Leeuwen, T. (eds) (2001) Multimodality. London: Sage. Kress, G. and Ogborn, Jon (1998) Modes of Representation and Local Epistemologies: The Presentation of Science in Education, Subjectivity in the School Curriculum (Working Paper 2, Institute of Education). London: University of London. Iedema, R. (2003)Multimodality, resemiotization: extending the analysis of discourse as multi-semiotic practice. Visual Communications, Vol 2 (1). Dicks, B. Mason, B., Coffey, A. and Atkinson, P. (2006) Qualitative Research and Hypermedia: Ethnography for the Digital Age. London: sage.

The tasks

Task 1: observation in the library Observation: what and how to watch? Observation by broad sweep Observation for nothing in particular Searching for paradoxes Searching for the problem(s) facing the group* Looking at and recording everything is impossible. But: 1)It allows to make reflexive choices on what to focus 2)Tells you what kind of observer you are 3)Data gathered at this stage will inform the written account of the setting for a reader who is unfamiliar with it *Walcott (1981)

Task 2: recording of observations The affordances of media What modes do various media allow us to record, represent, construct? What is the use of combining various media in our research? photographs v. writing (field notes) writing v. video video v. photographs

Presentation 2 Ethics and consent in the field Emma Renold

Ethics and consent in the field Public Space: consent forms –Filming strangers –Visitor interviews School visit and fieldtrip –Ethics and research with children –Guidance and the law –Opting in, Opting out: negotiating consent with young children and adult-gatekeepers

Information poster for staff at research site

Warning poster Used alongside an informational flyer. To let people know that they may be recorded as part of a research project. Used when there is no intent to identify potential subjects caught on camera.

Consent form to be used alongside an informational flyer (for images only)

Detailed Consent form used during interviews

Alderson, P. and Morrow, V. (2004) Ethics, social research and consulting with children and young people. Barnardos. Farrell, A. (2005) (eds) Ethical Research with Children, Open University Press. Ethics and research with children

Informed consent: guidance and the law Legal grey areas (Gillick ruling and competence - prioritizing childrens consent) Professional guidelines: childrens consent must be sought in addition to parental consent (BSA, BERA,ESRC) Informed consent as always negotiated and in a state of renewal (Thorne 1987)

In the case of participatory social sciences research, consent to participate is seen as an ongoing and open- ended process. Consent here is not simply resolved through the formal signing of a consent document at the start of research. Instead it is continually open to revision and questioning. Highly formalised or bureaucratic ways of securing consent should be avoided in favour of fostering relationships in which ongoing ethical regard for participants is to be sustained, even after the study itself has been completed (ESRC, Research Ethics Framework 2006:24, para 3.2.2). ESRC, Research Ethics Framework

Opting in, opting out: negotiating consent with adult gate-keepers Negotiating access with teachers and parents to achieve starting point of seeking provisional (Flewitt 2005) consent with children Why we took an opt-out approach to parental consent Fears and anxieties: separating out participation from dissemination (overly cautious?)

Yes, not sure, no, yes: negotiating consent with young children Temporality of consent –Before: reluctant and keen, impossibility of consent as one-off activity before research begins –During: consent on the move, actively and constantly negotiated (foregrounded with camcorder) –After??: problematics of representation on-going dialogue and debate

Contact details Inna Kotchetkova – Emma Renold – Bambo Soyinka - QUALITI Cardiff University School of Social Sciences