1. When a picture and its words don't match: Irony and Integration in Comics Shweta Narayan University of California, Berkeley April 8 th 2006

2. Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

3. Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

4. Irony and Integration in Comics ● Integration? – Mental Spaces ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

5. Mental Spaces ● Theoretical framework – e.g. Fauconnier and Turner, 2002* ● Used to explain a wide variety of linguistic and cognitive phenomena * Fauconnier, Gilles and Mark Turner. 2002. The Way We Think: Conceptual Blending and the Mind's Hidden Complexities. New York: Basic Books.

6. Mental Spaces ● Spaces are cognitive domains – There are mappings between spaces.

7. Mental Spaces ● Spaces are cognitive domains – There are mappings between spaces. ● Let's look at “trashcan basketball” – How can we understand this at all? – What do trashcans have to do with basketball?

8. Mental Spaces ● Trashcan Basketball

9. Mental Spaces ● Trashcan Basketball Domain 1: Trashcans * Trash * Container

10. Mental Spaces ● Trashcan Basketball Domain 1: Trashcans Domain 2: Basketball * Trash * Container * Ball * Hoop * Points

11. Mental Spaces ● Trashcan Basketball * Trash * Container * Ball * Hoop * Points Domain 1: Trashcans Domain 2: Basketball Mappings

12. Integration ● Mental Spaces can be integrated ● Creating new spaces with – inferences mapped from the input spaces – and new inferences that emerge from the process of integration.

13. Integration ● Trashcan Basketball – Inference from Trashcans: You don't retrieve the ball – Inference from Basketball: It's a game – Emergent structure: Slam dunks aren't cool TrashcansBasketball * Trash * Container * Ball * Hoop * Points

14. Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

15. Comics as data

16. Time 1 Static image

17. Comics as data Time 1 Static image Time 2 Static image Panel boundary

18. Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion

19. Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion Time 3 Inferred motion

20. Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion Time 3 Static image Speech bubbles Inferred motion

21. Comics as data ● Linguistic and Pictorial component

22. Comics as data ● Linguistic and Pictorial component ● How do they relate?

23. Comics as data ● Linguistic and Pictorial component ● How do they relate? Type: Entity Property: can-talk

24. Comics as data ● Linguistic and Pictorial component ● How do they relate? Speech Content Type: Entity Property: can-talk

25. Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning Type: Entity Property: is-talking Speech Content

26. Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning ● Complementary information

27. Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning ● Complementary information ● Conflict?

28. Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

29. Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

30. Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

31. Conflict between characters' belief spaces

32. Picture Space Depicted objects: Political signs Speech space Content: implies lack of understanding of ch2's cognitive viewpoint

33. Conflict between characters' belief spaces Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs

34. Conflict between characters' belief spaces Depiction space Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects

35. Conflict between characters' belief spaces * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs Professor (off-screen) Type: Entity Property: has beliefs Property: can see Depiction space

36. Conflict between characters' belief spaces * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs Professor (off-screen) Type: Entity Property: has beliefs Property: can see Content informs beliefs Depiction space

42. Conflict between characters' belief spaces * Ch1 * Ch2 (Agatha) (Adam) * Speech * Depicted bubble object Type: Entity Property: cannot see Property: has beliefs Property: is-talking Speech Content implies beliefs Type: Entity Property: has beliefs Property: can see Content informs beliefs Depiction space

43. Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

45. Concept: “digital age” Information: - content: implies ink and paper are outdated and unnecessary Entity: US Constitution Information: - content: depiction of entity Narrative-level Conflict

46. Concept: “digital age” Information: - content: implies ink and paper are outdated and unnecessary - style: register shift to informal rhetorical question hand-lettering Entity: US Constitution Type: Ink-on-paper Evokes democracy as American cultural ideal Information: - content: depiction of entity - style: evokes action hand-drawing Narrative-level Conflict

47. Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – cues: ● Text space in middle panel – register: factual ● US Constitution – evokes: democracy ● art style – evokes: hand drawing; ink on paper ● Ink and Paper don't matter implies: Voting Machines are a good idea – cues: ● Text space in final panel – register: informal – style: rhetorical question

48. Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – Majority of the cues support this – Elaboration: Democracy ● Ink and Paper don't matter implies: Voting Machines are a good idea – Cues stylistically marked as sarcastic

49. Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – Majority of the cues support this – Elaboration: Democracy ● Ink and Paper don't matter implies: Voting Machines are a good idea – Cues stylistically marked as sarcastic Emergent reasoning: Voting machines are undemocratic

50. Picking a viewpoint ● Integrated spaces have – Multiple characters – Each with a belief space ● But spaces are not all equal – Readers know which viewpoint to side with.

51. Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

54. All the tech belongs to ME! I can make them say whatever I want, and Phil can't stop me. Mwaahahahahaha! http://www.girlgeniusonline.com/cgi-bin/ggmain.cgi?date=20060303

55. * Phil * World * Kaja * Ch1 (Gil) * Drawn Comic * Ch2 (Wooster) * Tech * Ch3 (Agatha) * Muppets * Mad science Domain 1: real lifeDomain 2: Comic Comic Creator1: artist Creator2: tech-person Property: is depiction of narrative world

56. Without Integration ● Kaja is not a mad scientist ● Kaja has a causal relation to the words on the page (she controls the tech) ● Kaja knows about muppets ● Characters do not know about muppets

57. Integration ● The words on the page are what characters say. ● Changing the written words changes the characters' speech. Emergent structure: Kaja can make the characters say things. – Which makes her a mad scientist

58. Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

59. Narrative Simultaneity ● Pictorial and linguistic information are presented “at the same instant” ● Conflicting spaces have to be

60. Characteristics of the Image ● Backgroundedness ● Truth ● Viewpoint

61. The importance of Inference ● The irony is not “in” the comic ● It comes from inferences – Made by the reader – Based on minimal cues – Involving the relations between mental spaces. ● Understanding is an active process.

62. Conclusions ● Multimodal narratives are different from single- modality narratives – Integration of pictorial and linguistic cues – Narrative simultaneity – Backgroundedness – Viewpoint ● Tools from Cognitive Linguistics are useful for analyzing this ● As well as for pure linguistic analysis.

63. Text Space Interaction Entity: voting machine (category)

64. Text Space Picture Space Interaction Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity

65. Text Space Picture Space Interaction Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing

66. Text Space Picture Space Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing

67. Text Space Picture Space Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing Generic space blend?

68. Emergent meaning ● Emergent meaning – Complementary information

69. Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – cues: ● Text space in middle panel – register: factual ● US Constitution – evokes: democracy ● art style – evokes: hand drawing; ink on paper ● Ink and Paper don't matter implies: Voting Machines are a good idea – cues: ● Text space in final panel

70. Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network

71. Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex)

72. Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex) – Conflict

73. Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex) – Conflict – Conflicting information between spaces – All of which does get projected into the blend.

74. Voting Machines: Conflict

75. Base Narrator's Belief Counterfactual Ink and paper matter Voting machines: bad - lead to fraud. Ink and paper don't matter Voting machines: good Narrator Voting machines Fraud

76. Base Narrator's Belief Counterfactual Ink and paper matter - for self- expression (e.g. this comic) Voting machines: bad - lead to fraud. - undemocratic Ink and paper don't matter Voting machines: good Narrator/artist Voting machines Fraud US Constitution Democracy

77. Base Narrator's Belief Counterfactual Ink and paper matter - for self- expression (e.g. this comic) Voting machines: bad - lead to fraud. - undemocratic Ink and paper don't matter Voting machines: good Narrator/artist Comic Voting machines Fraud US Constitution Democracy