1. TAUCHI – Tampere Unit for Computer-Human Interaction Sign & Meaning Introduction to Semiotics Daniel Chandler [3] Tatiana Evreinova Multimodal Interaction Research Group Tampere Unit for Computer-Human Interaction

2. TAUCHI – Tampere Unit for Computer-Human Interaction Contents i ntroduction (3) s ign and typology (10) m odality and visual representation (2) c odes (4) a rticulation (1) i ntertextuality (2) r eferences (1)

3. TAUCHI – Tampere Unit for Computer-Human Interaction What is semiotics about? SEMIOTICS is “concerned with everything that can be taken as a sign” [ Umberto Eco, 1976 ] can take the form of words, images, sounds, gestures or objects studies the capacity of humans to make, disseminate, and understand these signs involves the study not only of what we refer to as 'signs' in everyday speech, but also of anything that 'stands for' something else…

4. TAUCHI – Tampere Unit for Computer-Human Interaction a science which studies the life of signs at the heart of social life [Ferdinand de Saussure, 1971] every thought is a sign [Charles Sanders Peirce, 1857] Whether it’s “linguistic turn” or two views to semiology?

5. TAUCHI – Tampere Unit for Computer-Human Interaction Basic issues general semiotics is considered as the theory of the production and interpretation of meaning meaning is made by the deployment of acts and objects which function as signs in relation to other signs social examine semiotics practices, specific to a culture and community, for the making of various kinds of texts and meanings in contexts of culturally meaningful activity multimedia are based on the principle that all meaning-making necessarily overflows the analytical boundaries between distinct, idealized semiotic resource systems such as language, gesture, depiction or action

6. TAUCHI – Tampere Unit for Computer-Human Interaction Sign / What does it mean? Sign / What does it mean? 1/10 ALL INDIVIDUALS ARE MEANING-MAKERS...because we interpret things as signs unconsciously by relating them to familiar systems of conventions [Peirce, 1931] Why ?

7. TAUCHI – Tampere Unit for Computer-Human Interaction Sign / Dyadic model 2/10 SIGNIFIED - the idea being represented SIGNIFIER - the word doing the representing the sign is the whole that results from the association of the signifier with the signified

8. TAUCHI – Tampere Unit for Computer-Human Interaction SIGNIFIER SIGNIFIED Sign / Signification example Sign / Signification example 3/10

9. TAUCHI – Tampere Unit for Computer-Human Interaction an “ARBITRARY CHOICE” between two things means a choice for no good reason SISTER / English SISKO / Finnish ZUSTER / Dutch SYSTIR / Icelandic SOEUR / French SYSTER / Swedish SESTRA / Slovak SESUO / Lithuanian Why do we use different sound sequences to mean a female sibling? Sign / Arbitrariness Sign / Arbitrariness 4/10

10. TAUCHI – Tampere Unit for Computer-Human Interaction The meaning of word TREE depends on context and its relation to other words Sign / Arbitrariness example Sign / Arbitrariness example 5/10

11. TAUCHI – Tampere Unit for Computer-Human Interaction Language is linear we produce one sound after another and words follow one another SHE + CAN + GO Syntagm is interlinking signs sequentially during constructing sentences Sign / Syntagm Sign / Syntagm 6/10

12. TAUCHI – Tampere Unit for Computer-Human Interaction BUT…

13. TAUCHI – Tampere Unit for Computer-Human Interaction …at the same time as we produce these signs linked to one another, we also choose a sign from a whole range of alternative signs. IRA terrorists overran an army post in Londonderry in Northern Ireland Which sign should journalist choose from a range of possible alternatives? ? Terrorists Freedom fighters Guerillas Active units Paramilitaries IRA By choosing appropriate sign we are defining paradigmatic relationships between signs Sign / Paradigm Sign / Paradigm 7/10

14. TAUCHI – Tampere Unit for Computer-Human Interaction Sign / Denotation, Connotation & Myth Denotative level: a photograph of the movie star Connotative level: associations with glamour, sexuality, beauty or depression or drug-taking and untimely death Myth: the dream factory can produces glamour, but also can crush it

15. TAUCHI – Tampere Unit for Computer-Human Interaction Icon is a sign which is linked to its object by qualitative characteristics Index denotes its object by being physically linked to it, or affected by it Symbols has no qualitative or physical link to its object and defined by social convention most words are symbols Sign / Typology Sign / Typology 9/10

16. TAUCHI – Tampere Unit for Computer-Human Interaction Object denoted – THE QUEEN Object denoted – A TREE Sign / Typology Sign / Typology 10/10

17. TAUCHI – Tampere Unit for Computer-Human Interaction Modality and visual representation Modality and visual representation 1/2 Modality refers to the reality status accorded to or claimed by a sign, text or genre the mental schemata involved in visual recognition may be closer to the stereotypical simplicity of cartoon images than to photographs AMBIGUITY or APPARENT PARADOX?

18. TAUCHI – Tampere Unit for Computer-Human Interaction THE WORD IS NOT THE THING WITHOUT SIGNS NOTHING IS CONCEIVABLE [Sless, 1986] WHETHER ALL SIGNIFIERS ARE SOCIALLY ARBITRARY? Modality and visual representation Modality and visual representation 2/2

19. TAUCHI – Tampere Unit for Computer-Human Interaction Codes Codes (1/4) All perceptual systems are already languages in their own right [Jameson, 1972] Is this a white vase? …Or what? Is this geometric composition? IS ALL THIS AMBIGUOUS?

20. TAUCHI – Tampere Unit for Computer-Human Interaction Codes PROXIMITY SMALLNESS CONTINUITY SURROUNDNESS SIMILARITY SYMMETRY CLOSURE

21. TAUCHI – Tampere Unit for Computer-Human Interaction Codes organize signs into meaningful systems which correlate signifiers and signified BUT While every code is a system,not every system is a code Codes Codes (3/4)

22. TAUCHI – Tampere Unit for Computer-Human Interaction SOCIAL Verbal language Bodily Commodity Behavioral INTERPRETATIVE Perceptual Ideological TEXTUAL Scientific Aesthetic Genre, rhetorical and stylistic Mass media CODES THE WORLD (SOCIAL KNOWLEDGE) THE MEDIUM AND THE GENRE (TEXTUAL KNOWLEDGE) MODALITY JUDGEMENTS Codes Codes (4/4)

23. TAUCHI – Tampere Unit for Computer-Human Interaction Articulation / “Duality of patterning” Articulated object First articulation only Without articulation An object is articulated when having separable sections which are linked together

24. TAUCHI – Tampere Unit for Computer-Human Interaction Intertextuality Intertextuality (1/2) Any text is the absorption and transformation of another [Kristeva, 1980] THE DEATH OF AUTHOR OR THE BIRTH OF VIEWER? connecting the author and reader of a text connecting the text to other texts

25. TAUCHI – Tampere Unit for Computer-Human Interaction Intertextuality / Genette’s typology Intertextuality / Genette’s typology (2/2) META explicit or implicit critical commentary of one text on another HYPO the relation between a text and a preceding 'hypotext' ARCHI designation of a text as part of a genre INTER quotation, plagiarism, allusion PARA the relation between a text and its 'paratext' T EXTUALITY S

26. TAUCHI – Tampere Unit for Computer-Human Interaction References [1] Semiotics Resources (full bibliography) Available at: http://carbon.cudenver.edu/~mryder/itc_data/semiotics.htmlhttp://carbon.cudenver.edu/~mryder/itc_data/semiotics.html [2] Semiotics (by Werner Hammersting) Available at: http://www.olinda.com/edu/CinemaStudies/Semiotics/semiotics.htmhttp://www.olinda.com/edu/CinemaStudies/Semiotics/semiotics.htm [3] Semiotics for beginners (by Daniel Chandler, book) Available at: http://www.aber.ac.uk/media/Documents/S4B/http://www.aber.ac.uk/media/Documents/S4B/ [4] General Semiotics (by Jay Lemke) Available at: http://academic.brooklyn.cuny.edu/education/jlemke/theories.htmhttp://academic.brooklyn.cuny.edu/education/jlemke/theories.htm [5] What is Semiotics? (by Eugene Gorny) Available at: http://www.zhurnal.ru/staff/gorny/english/semiotic.htmhttp://www.zhurnal.ru/staff/gorny/english/semiotic.htm [6] What is Semiotics? (literature review by Scott Simkins) Available at: http://www.chass.utoronto.ca/epc/srb/cyber/sim1.htmlhttp://www.chass.utoronto.ca/epc/srb/cyber/sim1.html [7] Basic terms (by Benjamin Horton) Available at: http://www.geography.dur.ac.uk/teaching/level1/module3/3_7/docs/textanalysis_2.htmlhttp://www.geography.dur.ac.uk/teaching/level1/module3/3_7/docs/textanalysis_2.html [8] Introductory models and basic concepts: semiotics Available at: http://www.cultsock.ndirect.co.uk/MUHome/cshtml/semiomean/http://www.cultsock.ndirect.co.uk/MUHome/cshtml/semiomean/ [9] An introduction to semiotics Available at: http://www.merz-akademie.de/projekte/george.legrady/theory/semiot/anintro.htmhttp://www.merz-akademie.de/projekte/george.legrady/theory/semiot/anintro.htm [10] Signs and language Available at: http://www.rdillman.com/HFCL/TUTOR/Semiotics/http://www.rdillman.com/HFCL/TUTOR/Semiotics/

27. TAUCHI – Tampere Unit for Computer-Human Interaction Thanks for your time & attention Thanks for your time & attention