1 Fauziah Taib Faculty of Languages and Linguistics University of Malaya Kuala Lumpur, Malaysia. MULTIMODALITY AND GENRE : CLUSTERING WORDS AND IMAGES

1

Fauziah Taib

Faculty of Languages and Linguistics

University of Malaya

Kuala Lumpur, Malaysia.

MULTIMODALITY AND GENRE :CLUSTERING WORDS AND IMAGES

IN TEXTS

Multimodality and Genre : Clustering Words and Images in Texts. 12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.

2

Five parts of the presentation :

1. Introduction : Multimodality and genre2. The cluster analysis framework3. An example of cluster analysis4. A system network for the genre of business-to-

business brochures5. Summary

Multimodality and Genre : Clustering Words and Images in Texts


3

Texts that use multi-semiotic modes such as language, images, layout, colours, typography and these modes are

integrated in the meaning making of the texts.

All meaning making practices like early spelling, classroom interaction and scientific papers are multimodal for they

deploy more than one mode of communication. (Kress, 1997; Kress and van Leeuwen,2001; Lemke 1998).

Multimodal Texts


4

Images have meaning potential

contact

social distance

subjectivity


5

The meaning of composition / layout

MarginIdealGiven

MarginIdealNew

MarginRealGiven

MarginRealNew

centre

(Kress & van Leeuwen, 1996)


6

Weight, expansion, slope, curvature, connectivity, orientation etc


7

Colours have meaning potential

Warm colours Cold colours

Brightness, saturation, purity, modulation, hue, luminosity


8

Genre, in SFL, is seen as a staged, goal-oriented social process (Martin, 1992) meaning each stage in the genre contributes with one part of the overall that must bemade for the genre to accomplish its aims successfully.

Research in this area sought to explicate the distinctive stages, moves of genres together with patterns of lexical, grammatical, and cohesive choices which construct the function of the stages of the genre (Rothery, 1996).

Genre in Systemic Functional Linguistics (SFL)


9

Despite focusing on language, the principles of SFL reflect the awareness of the fact that

…there are other modes of meaning, in any culture, which are outside the realm of language.

(Halliday and Hasan, 1985)

It has a systematic model of grammar which makes extensive use of a stratified, metafunctional grammar.

(Hyland, 2002)

Why the SFL approach to genre theory?


10

Overall communicative purpose : Solution to a problem

Text : magazine advice column Functional label

I lied on my cv Revealing a problem (confession)

Should I come clean with my boss Appealing for help (question)

Yes Providing a solution (answer)

But prepared for the possibility of losing your job if you have a scrupulous Boss

Issuing a caveat (warning)

The bright side: you will gain her respect if you speak up and accept your mistake

Predicting the result (prediction 1)

and having got this burden off your chest will help you focus better on your work

(prediction 2)

Example of genre analysis

(van Leeuwen, 2005)

Multimodalrealisations

Language

Typography


11

Analyzing clusters which refer to a local grouping of items which may be verbal and/or visual in nature and which

are spatially proximate thereby defining a specific region or subregion of the page as a whole.

Clusters are functionally related to each other.

Cluster Analysis in Multimodal Texts(Baldry and Thibault, 2006)


12Gest

alt

Psy

cholo

gy

The Law of Figure-Ground DiscriminationThe Law of ClosureThe Law of Proximity The Law of Similarity& many more…


13


14

texts produced to advertise products or services to other businesses.

texts are all about disseminating information for “content is king for business-to-business marketing” (Murphy, 2007) unlike business-to-consumer brochures which appeal to the emotions of the individual customers (Kotler et al. 2002).

Business-to-business texts


15

1.Clustering

2.Refining the clusters

3.Identifying functions of clusters

4.Labelling clusters

5.Grouping the clusters

6.Identifying major clusters

6 - step procedure in analysing large sample

of multimodal texts.


16

Step 2 : Refining the title cluster

new cluster


17

Local groupings based on spatial proximity.

Local groupings based on lexical connectivity.

Local groupings based on contents of items.

Other parameters of clustering ?


18

Step 3 : Labelling cluster

No. Clusters Function(s) Multimodal realisations

1. Detail state date, venue and/or time and/or name of facilitator

-Language

-typography

-layout etc

2. Course highlights

state contents of programme -Language

-typography

-layout etc

3. Training provider information

i. identify the company organizing the programme

ii. identify corporate identity

-Language

-typography

-layout etc

4. Call and contact

provide contact details -Language

-Typography

-layout etc

Step 4 : Identifying function(s)


19

Clusters (26)TitleDetailMethodologyresource materialContentFocusTarget audiencetraining provider informationRegistrationCall and contact testimonyFeeIntroductionPurposeObjectiveBenefitIllustrationFocusercatch phrasecomplimentary item certificate of completiondiscounts and savingsclaimable fee facilitator’s profiletraining company’s profileaccreditation

Step 5 : Grouping clusters

DetailMethodologyResource materialContentFocusTarget audience

RegistrationCall and contact

TestimonyFee

IntroductionPurposeObjectiveBenefit

Groups (9)

Programme informationmajor cluster

Response solicitation major cluster

Value major cluster

Justifier major cluster

Step 6 : Identifying major cluster

Major clusters


20

1. Title

2. Programme information

3. Signature

4. Value

5. Justifier

6. Attention- getter

7. Incentive

8. Credibility

9. Response solicitation

Major clusters of business-to-business texts


21

This would be one of the earliest attempts to

establish a system network for a genre.

System network for the generic structure of Malaysian business-to-business brochures

The syntagmatic and the paradigmatic relations of elements can be presented in what is called as the system (Eggins, 2004).


22

SYSTEM NETWORK FOR THE GENERIC STRUCTURE OF BUSINESS-TO-BUSINESS TEXTS


23

Title One-cluster Title

ValueOne-cluster Testimony

Fee

Solicit responseOne-cluster

Two-cluster

RegistrationCall and contact

Registration + Call and contact

Programme information

Two-cluster Detail + Content

Three-clusterDetail + Content + Target audience

Detail + Content + Methodology

Four-cluster Detail + Content + Target audience+ Course highlight

Syntagmatic relations of major clusters

Paradigmatic choices

Realisations of eachcluster type


24

Summary

1. Cluster analysis + SF Multimodal discourse analysis

2. Additional parameters to be considered in cluster analysis framework

3. A 6 – step procedure in cluster analysis to analyse large corpus of multimodal texts

4. A system network for the generic structure of business advertising texts.


25

References

1. Baldry, A. and P. Thibault. 2006. Multimodal Transcription and Text Analysis. London/Connecticut: Equinox Publishing Ltd.

2. Eggins, S. 2004. An Introduction to Systemic Functional Linguistics. (2nd Edition). London/New York: Continuum.

3. Halliday, M.A.K. and R. Hasan. 1985. Language, Context and Text : Aspects of Language in a Social-Semiotic Perspective. Geelong,

Australia: Deakin University Press.

4. Kress, G. and T. van Leeuwen. 1996. Reading Images : The Grammar of Visual Design. London/New York: Routledge.

5. Van Leeuwen, T. 2005. Introducing Social Semiotics. London/New York: Routledge.


26

THANK YOU

Fauziah TaibFaculty of Languages and Linguistics

University Malaya50603 Kuala Lumpur Malaysia.

[email protected]

Documents

1 Fauziah Taib Faculty of Languages and Linguistics University of Malaya Kuala Lumpur, Malaysia. MULTIMODALITY AND GENRE : CLUSTERING WORDS AND IMAGES