MULTIMODALITY AND GENRE : CLUSTERING WORDS AND IMAGES IN TEXTS Fauziah Taib Faculty of Languages and Linguistics University of Malaya Kuala Lumpur, Malaysia.

Multimodality and Genre : Clustering Words and Images in Texts Five parts of the presentation : Introduction : Multimodality and genre The cluster analysis framework An example of cluster analysis A system network for the genre of business-to-business brochures Summary

Multimodal Texts Texts that use multi-semiotic modes such as language, images, layout, colours, typography and these modes are integrated in the meaning making of the texts. All meaning making practices like early spelling, classroom interaction and scientific papers are multimodal for they deploy more than one mode of communication. (Kress, 1997; Kress and van Leeuwen,2001; Lemke 1998).

Images have meaning potential contact subjectivity social distance

The meaning of composition / layout Margin Ideal Given Margin Ideal New centre Margin Real Given Margin Real New (Kress & van Leeuwen, 1996)

Weight, expansion, slope, curvature, connectivity, orientation etc

Colours have meaning potential Warm colours Cold colours Brightness, saturation, purity, modulation, hue, luminosity

Genre in Systemic Functional Linguistics (SFL) Genre, in SFL, is seen as a staged, goal-oriented social process (Martin, 1992) meaning each stage in the genre contributes with one part of the overall that must be made for the genre to accomplish its aims successfully. Research in this area sought to explicate the distinctive stages, moves of genres together with patterns of lexical, grammatical, and cohesive choices which construct the function of the stages of the genre (Rothery, 1996).

Why the SFL approach to genre theory? Despite focusing on language, the principles of SFL reflect the awareness of the fact that …there are other modes of meaning, in any culture, which are outside the realm of language. (Halliday and Hasan, 1985) It has a systematic model of grammar which makes extensive use of a stratified, metafunctional grammar. (Hyland, 2002)

Example of genre analysis Text : magazine advice column Functional label I lied on my cv Revealing a problem (confession) Should I come clean with my boss Appealing for help (question) Yes Providing a solution (answer) But prepared for the possibility of losing your job if you have a scrupulous Boss Issuing a caveat (warning) The bright side: you will gain her respect if you speak up and accept your mistake Predicting the result (prediction 1) and having got this burden off your chest will help you focus better on your work (prediction 2) Multimodal realisations Language T y p o g r a h Overall communicative purpose : Solution to a problem (van Leeuwen, 2005)

Clusters are functionally related to each other. Cluster Analysis in Multimodal Texts (Baldry and Thibault, 2006) Analyzing clusters which refer to a local grouping of items which may be verbal and/or visual in nature and which are spatially proximate thereby defining a specific region or subregion of the page as a whole. Clusters are functionally related to each other.

Gestalt Psychology The Law of Figure-Ground Discrimination The Law of Closure The Law of Proximity The Law of Similarity & many more… Gestalt Psychology

Business-to-business texts texts produced to advertise products or services to other businesses. texts are all about disseminating information for “content is king for business-to-business marketing” (Murphy, 2007) unlike business-to-consumer brochures which appeal to the emotions of the individual customers (Kotler et al. 2002).

6 - step procedure in analysing large sample of multimodal texts. 1.Clustering 2.Refining the clusters 3.Identifying functions of clusters 4.Labelling clusters 5.Grouping the clusters 6.Identifying major clusters

Step 2 : Refining the title cluster new cluster

Other parameters of clustering ? Local groupings based on spatial proximity. Local groupings based on lexical connectivity. Local groupings based on contents of items.

Step 3 : Labelling cluster No. Clusters Function(s) Step 4 : Identifying function(s) No. Clusters Function(s) Multimodal realisations 1. Detail state date, venue and/or time and/or name of facilitator -Language -typography -layout etc 2. Course highlights state contents of programme 3. Training provider information i. identify the company organizing the programme ii. identify corporate identity 4. Call and contact provide contact details -Typography

Step 5 : Grouping clusters Step 6 : Identifying major cluster Title Detail Methodology resource material Content Focus Target audience training provider information Registration Call and contact testimony Fee Introduction Purpose Objective Benefit Illustration Focuser catch phrase complimentary item certificate of completion discounts and savings claimable fee facilitator’s profile training company’s profile accreditation Groups (9) Major clusters Detail Methodology Resource material Content Focus Target audience Registration Call and contact Testimony Fee Introduction Purpose Objective Benefit Programme information major cluster Response solicitation major cluster Value major cluster Justifier major cluster

Major clusters of business-to-business texts 1. Title 2. Programme information 3. Signature 4. Value 5. Justifier 6. Attention- getter 7. Incentive 8. Credibility 9. Response solicitation

System network for the generic structure of Malaysian business-to-business brochures This would be one of the earliest attempts to establish a system network for a genre. The syntagmatic and the paradigmatic relations of elements can be presented in what is called as the system (Eggins, 2004).

SYSTEM NETWORK FOR THE GENERIC STRUCTURE OF BUSINESS-TO-BUSINESS TEXTS

Detail + Content + Methodology Realisations of each cluster type Syntagmatic relations of major clusters Paradigmatic choices Title One-cluster Title Value One-cluster Testimony Fee Registration Call and contact One-cluster Solicit response Two-cluster Registration + Call and contact Detail + Content Two-cluster Programme information Detail + Content + Target audience Three-cluster Detail + Content + Methodology Four-cluster Detail + Content + Target audience + Course highlight

Summary Cluster analysis + SF Multimodal discourse analysis Additional parameters to be considered in cluster analysis framework A 6 – step procedure in cluster analysis to analyse large corpus of multimodal texts A system network for the generic structure of business advertising texts.

References Baldry, A. and P. Thibault. 2006. Multimodal Transcription and Text Analysis. London/Connecticut: Equinox Publishing Ltd. Eggins, S. 2004. An Introduction to Systemic Functional Linguistics. (2nd Edition). London/New York: Continuum. Halliday, M.A.K. and R. Hasan. 1985. Language, Context and Text : Aspects of Language in a Social-Semiotic Perspective. Geelong, Australia: Deakin University Press. Kress, G. and T. van Leeuwen. 1996. Reading Images : The Grammar of Visual Design. London/New York: Routledge. Van Leeuwen, T. 2005. Introducing Social Semiotics. London/New York: Routledge.

Faculty of Languages and Linguistics THANK YOU Fauziah Taib Faculty of Languages and Linguistics University Malaya 50603 Kuala Lumpur Malaysia. fauziaht@um.edu.my