Doing video research on online/social media --- Methodological issues and a case study on Elisabetta Adami Università G. D’Annunzio Chieti-Pescara NCRM MODE// Training Day: Using Video for Digital Multimodal Research 21:08:12 / LKL

and Video-interaction Methodological issues: Research design Data selection Data storage Transcription Analysis Theory Ethical issues Case-Study: Prompt response

and Video-interaction Research design Object of study: video-interaction

and Video-interaction Video responses

and Video-interaction video-interaction: communication threads made of video files Video X Video X.1 Video X.2 Video X.3 Video X.1.2 RE: Video X RE:

and Video-interaction Research design Object of study: video-interaction Research question: how does it work? Data: Videos as texts Video exchanges

and Video-interaction Transcription & Analysis units of transcription units of analysis

Research focus and unit of analysis: The sign-making practices of video-interaction Initial Video RE: Video Response  Minimum syntagm of video-interaction: successful exchange Research focus: relatedness in the exchange

(Kress & van Leeuwen, 2001: 111)Kress & van Leeuwen, 2001: 111 A multimodal theory of communication […] concentrates on two things: 1.The semiotic resources of communication, the modes and the media used, and 2.The communicative practices in which these resources are used. Where Multimodality comes in The semantics and pragmatics of communication through videos

(Kress & van Leeuwen, 2001: 2)Kress & van Leeuwen, 2001: 2 we explore the common principles behind multimodal communication. we move towards a view of multimodality in which common semiotic principles operate in and across different modes […] we want to create a theory of semiotics appropriate to contemporary semiotic practice. (p. 2). Multimodality The semantics and pragmatics of communication through videos

Social Semiotics Hodge, R., & Kress, G. (1988) a theory of all sign systems as socially constituted, and treated as social practices (vii-viii)

Social Semiotics van Leeuwen, T. (2005) in social semiotics the focus changed from the ‘sign’ to the way people use semiotic ‘resources’ both to produce communicative artefacts and events and to interpret them – which is also a form of semiotic production – in the context of specific social situations and practices. (p. xi)

Further Social Semiotics notions: Affordances Interest Prompt – Response … in interaction

and Video-interaction Data selection –Most responded videos of ‘All time’Most responded videos of ‘All time’ (largest threads of video-interaction)

and Video-interaction Data selection Most responded videos of ‘All time’ (largest threads of video-interaction) Bug: e7622d6ab55&hl=en e7622d6ab55&hl=en Online data change! Online platforms may change!

and Video-interaction Data storage YouTube terms of use Prohibit download Sometimes you have to do what you can but cannot declare to

and Video-interaction Data selection Most responded videos of ‘All time’ (largest threads of video-interaction)

21:08:12 / LKL PhD research Request for responses

The corpus: video threads/1 The first Do ∙ 1 initial video └ 792 responses └ 33 sub-responses └ 1 sub-sub response ∙ 1 video summaryvideo summary └ 8 vid.responses └ 1 sub-response Response ex.

WDYT video thread: Multimodal Deployment MODENO. OF VIDEOS Written Hand-writing Type-writing 168 Image Photos Shots 73 Drawings 23 Flags 33 Maps (or satellite shot) 68 Clothing 15 Spoken (43 spoken only) 92 Soundtrack related to the topic 44 Singing 4      Visual  Auditory     

21:08:12 / LKL No (explicit) request

The corpus: video threads/2 Second thread: Best video EVER! –1 initial video └ 613 video responses └ 180 sub-responses

Testing relatedness: The data Best video EVER! –17 March 2008 –Posted by: itschriscrocker –Video Description: ‘1 million views for each blink’ –in 3 weeks: - almost 3 million views video responses Response ex.

The Responses: type of relatedness (1) Correspondence125 Adjacency pair (blink) –Positive (also ironical) –Negative –Evasive

The Responses: type of relatedness (2)  Evaluation on: –Video content ‘bestness’ (…) –CC author/character/history Fame; Sexual orientation (…) Correspondence125 Commentary 97

The Responses: type of relatedness (1+2) Correspondence125 Commentary 97 Mutual understanding ? Coherence  Take up an element of CC’s video as a prompt to which to respond

No reference to video content/author The Responses: type of relatedness (3) St. Patrick’s day related Non-heterosexual prompt –Forum discussion –Meeting place Mutual understanding ??? Coherence  Correspondence125 Commentary 97 Inferentially-related104 Take up a background element as prompt and make it their responded topic

The Responses: type of relatedness (4) Correspondence125 Commentary 97 Inferentially-related104 Remake 49 Original Spoof 34 Initial video Other CC’s video Manipulation, mis-/re-interpretation Recontextualization Implicitness, intertextuality Selection, assemblage Mutual understanding ??? Coherence 

The Responses: type of relatedness (5) Correspondence125 Commentary 97 Inferentially-related104 Remake 49 Original Spoof 34 Unrelated196 Comments: “a lot better than CC” No clues of relatedness Cooperative principles ??? Coherence  Discard them as ‘spam’? (Almost 1 out of 3 videos?) Or rather indicative of some diverse communicative principles?

Relatedness M.U. Coherence 125 Correspondence ?  97 Commentary ?  104 Inferential ???  49 Remake ???  34 Original Spoof ???  196 (apparently) Unrelated ???  The Responses: type of relatedness

Relatedness Any multimodally-constituted element ( prompt ) of the initial video (cor)responded in the video-response Initial Video Video Response Relatedness in video-interaction Unit of analysis within the texts: prompt-response relation

Relatedness M.U. Coherence Correspondence ?  Commentary ?  Inferential ???  Remake ???  Original Spoof ???  Unrelated ???  Prompt-Response  ?  Sign maker’s diversified interests -> chains of semiosis with compatible effects The Responses: type of relatedness

Video-interaction: The interlocutors’ mutual understanding is irrelevant (traditional) incoherent patterns are acceptable Texts = range of prompts Selection, transformation, assemblage, recontextualization of previous signifiers/signified into new signs Cooperation -> individualized participation in chains of semiosis Coherence -> Interest-driven prompt-response relatedness

Contemporary communication: The affordances of contemporary media: –Copy-and-paste techniques (selection and recontextualization) –(hyper)-linking instead of linearity. –Cooperation = Individualized participation in the production of chains of semiosis

Methodological issues: –Research design: your question (from handling your data) –Data selection: qualitatively driven (but quantity does matter) –Data storage: !!! (mind the gap with the copyright issues) –Transcription: searchable; manageable; work in progress –Analysis: starts with transcribing, always testing and ‘shaping’ your theory –Theory: if it’s too short a blanket, it’s already a finding –Ethical issues (covert observer) (You)Tubers as film-makers Portrayed participants as signs/texts and Video-interaction

The big issue: Theory vs. Practice Video-interaction cannot be explained with traditional models of communication − Cooperative principle (Grice 1965, 1975) − Relevance principle (Sperber and Wilson 1986) traditional text analysis notions − Coherence and Cohesion (Halliday & Hasan 1976; van Dijk 1985; Fairclough 1992)

1. Theory Successful communication = understanding each other’s −Intended meaning (inferences) (Grice 1975) −Mutual relevant assumptions (Sperber & Wilson 1986) −‘Point to discourse’ (‘global coherence’, van Dijk 1985: 131)

2. Practice in video-interaction Mutual understanding is irrelevant to successful exchanges

2. Practice in video-interaction Traditional theories useful only to detect how video-interaction does not work More exceptions than regularities

PhD Thesis (2009): Contemporary-changes-in-semiosis-and-communication Elisabetta Adami (2010) Contemporary patterns of communication: The case of YouTube video-interaction. Saarbrucken: LAP Adami, E. (2012) ‘The rhetoric of the implicit and the politics of representation in the age of copy-and-paste’, in Learning, Media and Technology, 37/2, ; Adami, E. (2009) ‘We/YouTube: Exploring sign-making in video-interaction’, Visual Communication, 8/4, NCRM MODE// Training Day: Using Video for Digital Multimodal Research Methodological issues and a case study on Elisabetta Adami 21:08:12 / LKL