SLIDE 1IS246 - FALL 2004 Lecture 08: Semiotic Media Theory IS 246 Multimedia Information Prof. Marc Davis UC Berkeley SIMS Monday and Wednesday 3:30 pm – 5:00 pm Fall

SLIDE 2IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 3IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 4IS246 - FALL 2004 Kuleshov Effect –Neutral Face  Soup (“Pensive” Face) –Neutral Face  Dead Woman (“Sad” Face) –Neutral Face  Child playing with toy bear (“Happy” Face) –How do you describe the face? Video has a dual semantics –Sequence-independent stable semantics of shots –Sequence-dependent variable semantics of shots

SLIDE 5IS246 - FALL 2004 Isenhour on Context and Order A1:B; A2:C  A1 NOT EQUAL A2 –Shot context affects shot meaning The shot before affects the shot after B:A1; C:A2  A1 NOT EQUAL A2 –Shot context affects shot meaning The shot after affects the shot before A:B NOT EQUAL B:A –Short order effects shot meaning (A:B):C NOT EQUAL A:(B:C)

SLIDE 6IS246 - FALL 2004 Burch’s Transitions Temporal transitions –Continuous –Discontinuous Temporal ellipsis –Measurable time ellipsis –Indefinite time ellipsis Temporal reversal (flashback, overlapping cut) –Measurable time reversal –Indefinite time reversal

SLIDE 7IS246 - FALL 2004 Burch’s Transitions Spatial transitions –Continuous –Discontinuous Proximal Radically discontinuous

SLIDE 8IS246 - FALL 2004 Barthes’ Action Sequences Consecutive –Temporal succession Consequential –Causal succession Volitive –Action results from an act of will Reactive –Causal succession based on stimulus-response Durative –Indicating the beginning, ending, or duration of an action Equipollent –Necessarily paired actions (e.g., asking a question and answering a question)

SLIDE 9IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 10IS246 - FALL 2004 Semiotic Media Theory Application of semiotic methods of analysis to media, especially cinema Traditionally used for structural and functional analysis In computational media (especially automatic film editing), used also to define the primitives, means of combination, and means of abstraction for cinematic synthesis

SLIDE 11IS246 - FALL 2004 Semiotic Media Theory Key questions for semiotics of cinema –How does cinema mean? –What are the significant structures of cinematic form? –What are the various hierarchical levels of these structures? –How do they compare to linguistic structures? –Does cinema have any unique semiotic properties?

SLIDE 12IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 13IS246 - FALL 2004 The Sign Signified Signifier

SLIDE 14IS246 - FALL 2004 The Linguistic Sign “dog” dog

SLIDE 15IS246 - FALL 2004 The Video Sign “dog”

SLIDE 16IS246 - FALL 2004 Arbitrariness of the Video Sign Theories of video denotation –Iconic (i.e., onomatopoetic) Video is a mechanical replication of what it represents –Arbitrary Video constructs an arbitrary relationship between signifier and signified –Motivated The relationship between the signifier and signified is motivated, but by what? –A “natural” analogy between video and the world? –By the conventions of cinematic language?

SLIDE 17IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 18IS246 - FALL 2004 Articulation –Any form of semiotic organization which produces distinct combinable units Double articulation in natural language –First articulation Morphemes: smallest formal units of significance (e.g., “cow”) Constructed out of phonemes –Second articulation Phonemes: sound units which in and of themselves lack significance (e.g., “c” “ow”)

SLIDE 19IS246 - FALL 2004 Commutation Etymologically “change together” The substitution of one signifier for another produces a change of the signified Example in phonemes to morphemes –Different pronunciations of the “ow” in “cow” will still be understood as “cow” –But we distinguish “cow” “caw” “quay” “coo” “cal”

SLIDE 20IS246 - FALL 2004 Cinematic Articulations Metz –Cinema has no double articulation because its smallest units (“shots”) are significant –Based on Bazinian view of cinema as reproduction of reality Eco –Cinema has three levels of articulation which include sub-shot units –Similar to Eisensteinian view of cinema as construction of representations

SLIDE 21IS246 - FALL 2004 Iconic Figures / Semes / Signs Iconic SignsIconic SemesPhotograms Iconic Figures (deduced from perceptive codes) constituting a paradigm from which units are selected to form…

SLIDE 22IS246 - FALL 2004 Eco’s Photographic Articulations Iconic semes –Example: “a dark-haired man stands here wearing a patterned shirt” Iconic signs –Example: human nose, human eye, shirt, etc. Iconic figures –Example: angles, light contrasts, curves, etc.

SLIDE 23IS246 - FALL 2004 Eco’s Cinematic Articulations Kinesic semes (kinemorphs) –Example: “I’m saying yes to the person on the right” Kinesic signs (kines) –Example: Nod head yes Kinesic figures –Example: move head to right, move head up, move head down

SLIDE 24IS246 - FALL 2004 Kinesic Figures / Kines / Kinemorphs Kinesic Figures Iconic Figures Iconic Signs Iconic Semes Photo- grams Synchrony Diachrony KinesKinemorphs

SLIDE 25IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 26IS246 - FALL 2004 Metz’s Grand Syntagmatique Autonomous Segments Autonomous Shot Syntagmas Achronological Syntagmas Chronological Syntagmas Parallel Syntagma Bracket Syntagma Descriptive Syntagma Narrative Syntagmas Alternate (Narrative) Syntagma Linear (Narrative) Syntagmas Scene Sequences Episodic Sequence Ordinary Sequence

SLIDE 27IS246 - FALL 2004 Metz’s Grand Syntagmatique Autonomous Shot (single shot) –Single-Shot Sequence (complete unto itself) –Inserts (differentiated from shot context) Non-diegetic insert –A single shot which presents objects exterior to the story world Displaced diegetic insert –Diegetic images temporally and/or spatially out of context Subjective insert –Memories, fears, dreams, etc. of character Explanatory insert –Single shots which clarify diegetic events

SLIDE 28IS246 - FALL 2004 Metz’s Grand Syntagmatique Achronological Syntagmas –Parallel Syntagma (alternating) Two alternating motifs without clear spatial or temporal relationship –Bracket Syntagma (non-alternating) Brief scenes without temporal sequence but often organized around a concept Chronological Syntagmas –Descriptive Syntagma (non-narrative) Objects shown to create spatial contiguity to situate action –Narrative Syntagmas (narrative)

SLIDE 29IS246 - FALL 2004 Metz’s Grand Syntagmatique Narrative Syntagmas –Alternate (Narrative) Syntagma Narrative crosscutting showing temporal simultaneity (“parallel action”) –Linear (Narrative) Syntagma Scene (continuous) –Spatial contiguity and temporal continuity across a series of shots Sequences (elliptical) –Episodic Sequence »Symbolic summary of chronological progression usually to compress time (“montage sequence”) –Ordinary Sequence

SLIDE 30IS246 - FALL 2004 Brian O’Connor on Metz

SLIDE 31IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 32IS246 - FALL 2004 Discussion Questions (Barthes) Jaiwant Virk on Barthes –If “The principle of narrative art… it is a matter of producing a discourse which best satisfies the demand for completeness…”, then how does one define the boundaries of the action sequence? How much cultural history does one pack to prevent the “horror of the vacuum”? Or should the sequence be completely abandoned to a subjective analysis of the reader/viewer?

SLIDE 33IS246 - FALL 2004 Discussion Questions (Barthes) Jaiwant Virk on Barthes –What are the possible factors that can be considered to define/label an action sequence for media metadata purpose (motion, etc.)?

SLIDE 34IS246 - FALL 2004 Discussion Questions (Eco) Yongwook Jeong on Eco –According to Eco, because all signs are arbitrary and conventional, we have to learn how to interpret them. This made me have a question about whether it is possible to have more than one languages of cinema. Does this make it possible to distinguish one kind of films from another?

SLIDE 35IS246 - FALL 2004 Discussion Questions (Eco) Yongwook Jeong on Eco –Regarding the cinematic code as the only code carrying a triple articulation, Eco seems to make iconic codes a basis for his cinematic language. Can we apply his arguments to today's films?

SLIDE 36IS246 - FALL 2004 Discussion Questions (Metz) Ryan Shaw on Metz –Metz believes that in cinema, unlike in spoken language, even the smallest identifiable units are complete signs, having a signifier (the image) and a signified (the thing which the image is depicting). Moreover, unlike in linguistic signs, the relationship between the signifier and the signified in these signs is motivated by perceptual similarities ("the image of a dog is like the dog"). –What implications does Metz's view have for the task of programming a computer to understand a film? If the smallest analyzable units of cinema depend on human perception for their meaning, must we solve the problems of computational perception before such a task can be undertaken? Or might the perceptual motivations of cinematic signs be encoded in metadata in such a way that a computer could construct an understanding of a film, even without the ability to perceive the motivations itself?

SLIDE 37IS246 - FALL 2004 Discussion Questions (Metz) Ryan Shaw on Metz –Metz struggles with the application of linguistic models to the question of how films are understood. His comparisons of linguistics and cinema seem to point out more differences than similarities, yet he persists in his effort due to his belief that “the methods of linguistics... provide the semiotics of cinema with a constant and precious aid in establishing units that... are liable over time... to become progressively refined.” –Is this belief justified? Is the semiotics of cinema making progress in refining these units? Or might it be more useful to stop trying to treat cinema as a “language” and investigate other methods (psychological or neurological, perhaps)?

SLIDE 38IS246 - FALL 2004 Today’s Agenda Review of Last Time –Editing II Semiotic Media Theory –The Video Sign –Cinematic Articulations –Syntagmatic Structures Discussion Questions Action Items for Next Time

SLIDE 39IS246 - FALL 2004 Metz on Shot/Word Distinction Shots are infinite in number, contrary to words, but like statements, which can be formulated in verbal language. Shots are the creations of the film-maker, unlike words (which pre-exist in lexicons), but similar to statements (which are in principle the invention of the speaker). The shot presents the receiver with a quantity of undefined information, contrary to the word. From this point of view, the shot is not even equivalent to the sentence. Rather, it is like the complex statement of undefined length (how is one to describe a film shot completely by means of natural language?). The shot is an actualized unit, a unit of discourse, an assertion, unlike the word (which is a purely virtual lexical unit), but like the statement, which always refers to reality or a reality (even when it is interrogative or jussive). The image of a house does not signify “house,” but “Here is a house”; the image contains a sort of index of actualization, by the mere fact that it occurs in a film. Only to a small extent does a shot assume its meaning in paradigmatic contrast to the other shots that might have occurred at the same point along the filmic chain (since the other possible shots are infinite in number), whereas the word is always part of at least one more or less organized semantic field. (Metz 1974: ).

SLIDE 40IS246 - FALL 2004 Administrivia You MUST have a SIMS computer account to do Assignment 1

SLIDE 41IS246 - FALL 2004 Readings for Next Time Wednesday 09/29 –Eisenstein, S.M. Film Form: Essays in Film Theory. Harcourt Brace Jovanovich, Publishers, San Diego, 1949; pp