Transcript MULTIMODALITY AND GENRE : CLUSTERING WORDS AND …

MULTIMODALITY AND GENRE :
CLUSTERING WORDS AND IMAGES
IN TEXTS
Fauziah Taib
Faculty of Languages and Linguistics
University of Malaya
Kuala Lumpur, Malaysia.
1
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Multimodality and Genre :
Clustering Words and Images in Texts
Five parts of the presentation :
1.
2.
3.
4.
5.
Introduction : Multimodality and genre
The cluster analysis framework
An example of cluster analysis
A system network for the genre of business-tobusiness brochures
Summary
2
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Multimodal Texts
Texts that use multi-semiotic modes such as language,
images, layout, colours, typography and these modes are
integrated in the meaning making of the texts.


All meaning making practices like early spelling, classroom
interaction and scientific papers are multimodal for they
deploy more than one mode of communication.
(Kress, 1997; Kress and van Leeuwen,2001; Lemke 1998).
3
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Images have meaning potential
contact
subjectivity
social distance
4
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
The meaning of composition / layout
Margin
Ideal
New
Margin
Ideal
Given
centre
Margin
Real
Given
Margin
Real
New
(Kress & van Leeuwen, 1996)
5
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Weight, expansion, slope, curvature, connectivity, orientation etc
6
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Colours have meaning potential
Warm colours
Cold colours
Brightness, saturation, purity, modulation, hue, luminosity
7
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Genre in Systemic Functional Linguistics (SFL)
Genre, in SFL, is seen as a
staged, goal-oriented social process (Martin, 1992) meaning each
stage in the genre contributes with one part of the overall that must be
made for the genre to accomplish its aims successfully.
Research in this area sought to
explicate the distinctive stages, moves of genres together with patterns
of lexical, grammatical, and cohesive choices which construct the
function of the stages of the genre (Rothery, 1996).
8
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Why the SFL approach to genre theory?

Despite focusing on language, the principles of SFL reflect
the awareness of the fact that
…there are other modes of meaning, in any culture, which
are outside the realm of language.
(Halliday and Hasan, 1985)

It has a systematic model of grammar which makes
extensive use of a stratified, metafunctional grammar.
(Hyland, 2002)
9
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Example of genre analysis
Text : magazine advice column
Functional label
I lied on my cv
Revealing a problem (confession)
Should I come clean with my boss
Appealing for help (question)
Yes
Providing a solution (answer)
But prepared for the possibility of losing
your job if you have a scrupulous Boss
Issuing a caveat (warning)
The bright side: you will gain her respect if
you speak up and accept your mistake
Predicting the result (prediction 1)
and having got this burden off your chest
will help you focus better on your work
(prediction 2)
Overall communicative purpose : Solution to a problem
Multimodal
realisations
L
a
n
g
u
a
g
e
T
y
p
o
g
r
a
p
h
y
(van Leeuwen, 2005)
10
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Cluster Analysis in Multimodal Texts
(Baldry and Thibault, 2006)
Analyzing clusters which refer to a local grouping of items
which may be verbal and/or visual in nature and which
are spatially proximate thereby defining a specific region
or subregion of the page as a whole.
Clusters are functionally related to each other.
11
Psychology
Gestalt
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
The Law of Figure-Ground Discrimination
The Law of Closure
The Law of Proximity
The Law of Similarity
& many more…
12
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
13
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Business-to-business texts

texts produced to advertise products or services to other
businesses.

texts are all about disseminating information for “content
is king for business-to-business marketing” (Murphy,
2007) unlike business-to-consumer brochures which
appeal to the emotions of the individual customers
(Kotler et al. 2002).
14
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
6 - step procedure in analysing large sample
of multimodal texts.
1.Clustering
2.Refining the clusters
3.Identifying functions of clusters
4.Labelling clusters
5.Grouping the clusters
6.Identifying major clusters
15
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Step 2 : Refining the title cluster
new cluster
16
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Other parameters of clustering ?
Local groupings based on spatial proximity.
Local groupings based on lexical connectivity.
Local groupings based on contents of items.
17
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Step 3 : Labelling cluster
Step 4 : Identifying function(s)
No. Clusters
Function(s)
Multimodal
realisations
1.
Detail
state date, venue and/or
time and/or name of
facilitator
-Language
-typography
-layout etc
2.
Course
highlights
state contents of programme
-Language
-typography
-layout etc
3.
Training
provider
information
i. identify the company
organizing the programme
ii. identify corporate identity
-Language
-typography
-layout etc
4.
Call and
contact
provide contact details
-Language
-Typography
-layout etc
18
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Step 5 : Grouping clusters
Step 6 : Identifying major cluster
Clusters (26)
Title
Detail
Methodology
resource material
Content
Focus
Target audience
training provider information
Registration
Call and contact
testimony
Fee
Introduction
Purpose
Objective
Benefit
Illustration
Focuser
catch phrase
complimentary item
certificate of completion
discounts and savings
claimable fee
facilitator’s profile
training company’s profile
accreditation
Groups (9)
Major clusters
Detail
Methodology
Resource material
Content
Focus
Target audience
Programme
information
major cluster
Registration
Call and contact
Response
solicitation
major cluster
Testimony
Fee
Value major
cluster
Introduction
Purpose
Objective
Benefit
Justifier
major cluster
19
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Major clusters of business-to-business texts
1.
2.
3.
4.
5.
6.
7.
8.
9.
Title
Programme information
Signature
Value
Justifier
Attention- getter
Incentive
Credibility
Response solicitation
20
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
System network for the generic structure of
Malaysian business-to-business brochures
This would be one of the earliest attempts to
establish a system network for a genre.
The syntagmatic and the paradigmatic relations of
elements can be presented in what is called as the
system (Eggins, 2004).
21
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
SYSTEM NETWORK FOR THE GENERIC STRUCTURE OF
BUSINESS-TO-BUSINESS TEXTS
22
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Syntagmatic relations
of major clusters
Title
Value
Solicit response
Programme
information
Paradigmatic
choices
Realisations of each
cluster type
One-cluster
Title
One-cluster
Testimony
Fee
Two-cluster
Registration
Call and contact
Registration + Call and contact
Two-cluster
Detail + Content
One-cluster
Three-cluster
Four-cluster
Detail + Content + Target audience
Detail + Content + Methodology
Detail + Content + Target audience
+ Course highlight
23
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
Summary
1. Cluster analysis + SF Multimodal discourse analysis
2. Additional parameters to be considered in cluster
analysis framework
3. A 6 – step procedure in cluster analysis to analyse large
corpus of multimodal texts
4. A system network for the generic structure of business
advertising texts.
24
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
References
1. Baldry, A. and P. Thibault. 2006. Multimodal Transcription and
Text Analysis. London/Connecticut: Equinox Publishing Ltd.
2. Eggins, S. 2004. An Introduction to Systemic Functional Linguistics.
(2nd Edition). London/New York: Continuum.
3. Halliday, M.A.K. and R. Hasan. 1985. Language, Context and Text :
Aspects of Language in a Social-Semiotic Perspective. Geelong,
Australia: Deakin University Press.
4. Kress, G. and T. van Leeuwen. 1996. Reading Images : The Grammar of
Visual Design. London/New York: Routledge.
5. Van Leeuwen, T. 2005. Introducing Social Semiotics. London/New York:
Routledge.
25
Multimodality and Genre : Clustering Words and Images in Texts.
12th ESEA Conference. 12th-14th Dec. 2007. KMUTT, Bangkok, Thailand.
THANK YOU
Fauziah Taib
Faculty of Languages and Linguistics
University Malaya
50603 Kuala Lumpur Malaysia.
[email protected]
26