1. & Introduction to Multimodal Analysis D. Machin (2012)
Reading Images. The grammar of visual design G. Kress and T. Van Leewen (2008)
& Introduction to Multimodal Analysis
D. Machin (2012)

2. The grammar of visual design
Just as grammars of language describe how words combine in clauses, sentences and texts, so visual ‘grammar’ will describe the way in which depicted elements – people, places and things – combine in visual ‘statements’ of greater or lesser complexity and extension.
Grammatical forms are seen as resources for encoding interpretations of experience and forms of social (inter)action.

3. The grammar of visual design
Rhodesia Herald wrote in 1975,
‘A political clash has led to death and injury’,
while the Tanzanian Daily News wrote,
‘Rhodesia’s white suprematist police opened fire and killed thirteen unarmed Africans.’
The same is true for the ‘grammar of visual design’. Think of an example!

4. The grammar of visual design
Boris Nemtsov’s funeral (The Guardian)

5. The grammar of visual design
Boris Nemtsov’s funeral (Il Fatto Quotidiano)

6. The grammar of visual design
The visual, like all semiotic modes, has to serve several representational and communicational requirements.
We have adopted the theoretical notion of ‘metafunction’ from the work of Michael Halliday for this purpose. The three metafunctions which he posits are the ideational, the interpersonal and the textual.
These metafunctions apply to all semiotic modes, and are not specific to speech or writing.

7. Ideational metafunction
Any semiotic mode has to be able to represent aspects of the world as it is experienced by humans.
In other words, it has to be able to represent objects and their relations in the world and the way they are perceived by humans
In visual communication, colour, for example, can represent specific people (colours of uniform), places (flags for nations), things (brand colour). The colour blue might be used on a map to represent the idea of ocean.

8. Ideational metafunction
X-Man movie poster

9. Ideational metafunction
This metafunction can be represented by font, too.
In the X-Man movie poster, the typeface of the ‘X’ is used to illustrate the idea of strength, power and solidness. It is bold, heavy, made of metal.
Metaphorical association: thickness is ‘stronger’

10. Ideational metafunction
Who we are and what we are (what we represent) is communicated visually.
Agency is communicated symbolically through posture, clothing and use of products rather than by what we actually do in the world

11. Interpersonal metafunction
Visual communication also has resources for constituting and maintaining another kind of interaction, the interaction between the producer and the viewer of the image.
Another way of saying this is that images (and other kinds of visual) involve two kinds of participants, represented participants (the people, the places and things depicted in images) and interactive participants (the people who communicate with each other through images, the producers and viewers of images).

12. Interpersonal metafunction
There are three kinds of relations:
(1) relations between represented participants;
(2) relations between interactive and represented participants (the interactive participants’ attitudes towards the represented participants);
(3) relations between interactive participants (the things interactive participants do to or for each other through images).

13. Interpersonal metafunction
Recruitment poster (Alfred Leete, 1914)

14. Interpersonal metafunction
Positioning/aligning the viewer in relation to people inside the image
Gaze: to what extent we are encouraged to engage with the participants
Angle of interaction: this can create power relationships and also involvment
Distance: this is like social distance, suggesting intimacy or remoteness

15. Interpersonal metafunction
Gaze
In pictures, as in real life, the depicted people can look at the viewer so that there is the symbolic ‘contact’, ‘interaction’ (pleading, seductive, arrogant, etc.) between the viewer and the people depicted . This produces an image act, a demand image.
People (or objects) can also be depicted as as not looking at the viewer, there is no interection, and the people (objects) in the picture are looked at, as a kind of exhibit. This produces an offer image.

16. Interpersonal metafunction
Angle of interaction
Our involvement in a scene can also be changed through viewing positions, through the angle of interaction around the horizontal plane (frontal, oblique, side on, back).
There is also the vertical angle. The viewer can either look down on or look up to people to various degrees
Oblique angles are used less frequently in news photography but are often found in cartoons and movies and provide an unsettling effects and suggest tension

17. Interpersonal metafunction
Distance
In pictures, as in real life, distance signifies social relations. This has the associations of physical proximity in real life. In pictures, distance translates as size of frame (close shot, medium shot, long shot).
Close-ups create a sense of intimacy with those represented. Increased distance creates anonimity.

18. Interpersonal metafunction
Clueless movie poster

19. Interpersonal metafunction

20. Interpersonal metafunction

21. Textual metafunction
In Visual Grammar, ideational metafunction looks at the representation of reality, interpersonal metafunction looks at the interaction among participants, textual metafunction looks at composition.
This third element refers to the composition of the whole, the way in which the representational and interactive elements are made to relate to each other, the way they are integrated into a meaningful whole.

22. Textual metafunction
Composition, then, relates the representational and interactive meanings of the image to each other through three interrelated systems:
(1) Information value. The placement of elements (participants and syntagms that relate them to each other and to the viewer) endows them with the specific informational values attached to the various ‘zones’ of the image: left and right, top and bottom, centre and margin.
(2) Salience. The elements (participants as well as objects, facts/situations) are made to attract the viewer’s attention to different degrees, as realized by such factors as placement in the foreground or background, relative size, contrasts in tonal value (or colour), differences in sharpness, etc.
(3) Framing. The presence or absence of framing devices (realized by elements which create dividing lines, or by actual frame lines) disconnects or connects elements of the image, signifying that they belong or do not belong together in some sense

23. Textual metafunction
Information value: the dimension of visual space

24. Textual metafunction Symbolic value/association Left/Right
Left/Given: bad, immorality, sacred, sinful, forbidden, perversion, past, known.
Right/New: good, everyday life, righteousness, present, unknown
Top/Bottom
Top/Ideal: power, upper social status, positive, unreal, abstract, fantasy, generalisations
Bottom/Real: weakness, lower social status, negative, real, material, credibility, details
Centre/Margins
Central elements give meaning and coherence to those in the margin. They convey immediate symbolic meaning.
Elements placed in the margins may not be represented as equal in size, detail and salience. This will give them a rank of importance
Symbolic value/association

25. Textual metafunction Some basic principles of salience: Size Colour
Tone (brightness, highlighting, etc.)
Focus (the background disappears)
Foregrounding
Overlapping

26. Textual metafunction

27. Semiotic resources for creating framing and connectivity
Textual metafunction
Semiotic resources for creating framing and connectivity
Segregation: use of physical frames to create differences
Separation: separation by empty space rather than by frames.
Integration: this is where elements occupy the same space coherently
Overlap: this is where elements are not constrained by frames and spaces. Their meaning can also bleed into other spaces
Repetition: colour, posture, size, etc. can be re-used to create links and relations between elements
Contrast: colour, posture, size etc. can be used to indicate difference