Transcript Assessing L2 writing performance: Reconsidering writing prompts as genre-based tasks Heidi Byrnes German Department Georgetown University [email protected] TBLT 2007 University of Hawai’i at Manoa Sept.

Assessing L2 writing performance:
Reconsidering writing prompts
as genre-based tasks
Heidi Byrnes
German Department
Georgetown University
[email protected]
TBLT 2007
University of Hawai’i at Manoa
Sept. 21, 2007
GUGD curriculum
web page:
http://www3.georgetown.edu/departments/german/programs/curriculum/index.html
Underlying Assumptions
 L2 writing prompts can/should
- be reconsidered as “tasks”, particularly as genre-based tasks
- be linked to quality of performance
- facilitate the assessment of L2 writing performance
 L2 writing in genre-based tasks can
- Provide a favorable environment for integrating “language” and
“content”
 Assessment of L2 writing performance should
- Consider language knowledge and content knowledge in an
integrated fashion
“Tasks” in L2 education (1)
An educational “activity”
 With a particular objective, appropriate content, specified
work procedure, range of outcomes, necessitating the use
of language (Breen 1997, Van den Branden 2006)
 In the target language, focused on meaning rather than
form (Nunan 1989)
 Meaning-oriented language use, resembling language use
in the real world, often occasioned by a problem, involving
a workplan toward achieving a certain outcome (Ellis 2003,
Skehan 1998)
“Tasks” in L2 education (2)
 Users of tasks
Teachers and learners, curriculum developers,
researchers, assessment specialists (Bygate et al. 2001)
 Uses of tasks, as a way of
– Specifying learning goals
– Designing and organizing educational activities in support of
learning
– Ascertaining ability to use language (assessment)
– Exploring implications of findings for curriculum and instruction
“Tasks”:
Relating “language” and “content/meaning”
 Language AND content
 Language knowledge AND content/background
knowledge
 Focus on form AND focus on meaning
 Ability to use/perform AND ability to develop
Additive conceptualization of task
No conceptual reach into content/meaning in
social context
Needed: New conception of “language”
“Our approach to the teaching of any phenomenon
depends critically on our conception of this
phenomenon.
Unless we can base language teaching and learning
on a richly revealing comprehensive account of
what kind of phenomenon language is, we are not
in a position to answer the many questions that
arise in educational contexts.
Similarly, the value and success of any approach
designed to support second or foreign language
teaching … will depend critically on the conception
of language that informs it.” (Matthiessen 2006: 31)
Probing two constructs of “task” (1)
Language
analyzed as
Formal system of rules
Fundamentally functional
social semiotic system
Meaning
making
Expressing things;
Focus on personal
meanings;
“Negotiation of meaning”.
Construing/creating that
which is being
expressed;
Developing a voice
within socially situated,
socially preferred forms
of language use.
Semantics and Inventories
lexicogrammar
Strategic resources
Probing two constructs of “task” (2)
Context and
content
Language
learning as
Given “out there” Socially constructed;
Language as primary semiotic
system;
“Context of culture” + “context
of situation” participating in
the construal of meaning
Gains in
accuracy, fluency,
complexity of
processing in all
modalities
Focus of attention Sentence-level
processing
Learning how to mean;
Building up ‘meaning potential’
in a range of registers and text
types/genres
Oral/written texts as
environment arising from and
construing social structures
and meanings
Probing two constructs of “task” (3)
Processing
orientation
Syntagmatic syntactic
rules orientation
Paradigmatic meaning
and choice orientation
“Knowledge”
imagined and
valued as
Cognition;
“Cognitively
demanding;”
“Higher-order thinking”
Language-based
shaping of world;
Expressed in texts from
2 semiotic perspectives
 congruent semiosis
 non-congruent
(“metaphorical”) semiosis
Integration
through
Psycholinguistic
mechanisms focused on
individual language user
Semiotic processes
focused on construing
and shaping the
physical and social
world = meaning
making
Needed: New understanding and
descriptions of language
 The challenge is to relate sociocultural context and the “content” that is
likely to be enacted to its particular wording in language, that is, we
need rich accounts of semantics and lexicogrammar.
 “On the one hand, we need semantics as an ‘interface’ between
context and lexicogrammar … a fully fledged semantics of text. …
Language learners need to learn semantics as a strategic resource …
a resource for transforming what is not meaning into meaning,
construing their experience of the world as meaning and enacting
social roles and relations as meaning; and this will provide them with
the ‘bridge’ to lexicogrammar.
 On the other hand, both semantics and lexicogrammar need to be
learned as resources rather than as inventories.
Needed …
 ... To meet the challenge of relating language to context,
we also need to have a systematic and comprehensive
account of context … Like language, context is a semiotic
system, but it is a different kind of semiotic system …
language is a denotative semiotic system (that is, a
semiotic system that has its own expression plane),
whereas context is a connotative semiotic system (that is,
a semiotic system that has other semiotic systems as its
expression plane). Context is realized by language, and by
other denotative semiotic systems such as gesture and
facial expressions; and also by non-semiotic, social
systems.” (Matthiessen 2006, 37-38)
SFL approach to relating
meaning/content and language
 The process of meaning making is multifunctional in all
languages
 Three metafunctional modes of meaning
– Construing our experience in the world around us and inside us as
meaning = ideational metafunction (Meaning as “representation”:
participants, processes, circumstances)
– Enacting our social roles and relations as meaning =
interpersonal metafunction (Meaning as “exchange”)
– Presenting the meaning construed and enacted as a flow of
information = textual metafunction (meaning as “message/text”:
“coherence, cohesion”)
SFL approach (ctd.)
 These resonate with three contextual variables
- field: “what” is being talked about (social activity)
(participants, processes, circumstances)
- tenor: “how” the participants relate to each other in
relation to the activity (mood, modality, appraisal)
- mode: “how” the oral/written text is structured
(theme and information structure; logical relationships in
clause complexes; grammatical metaphor)
A trinocular conception of
meaning/context
Language comprises
 Ideational resources naturalize reality
 “Field knowledge” enables participation in domestic, recreational,
academic, public, and professional activities
 Interpersonal resources negotiate social relations
 ”Interpersonal knowledge” enables a way of valuing and assessing
these activities and enacting power and solidarity in relation to shared
values
 Textual resources manage information flow
 “Textual knowledge” enables ways of phasing the ideational and
interpersonal meanings into textures that are responsive/sensitive to
the communicative demands of oral and written discourse.
Metafunction in relation to language,
register, and genre
genre
gen re
field
mode
register
textual
ideational
language
tenor
interpersonal
Register and genre in SFL
 Register – a “constellation”
A constellation of lexicogrammatical features that realizes a particular situational
context, characterized in terms of ranges of field, tenor, and mode values in the
contexts within which they are located
 Genre – a “text type”
 Recurrent configuration of meanings
 Culture as a system of genres
 Staged, goal-oriented social process, realized in text
 Staged: more than one phase of meaning to work through the genre
 Goal-oriented: unfolding phases toward meeting expectations
 Social: genres occur in social settings and are interactively undertaken
 The “teachability” of genres
Big Challenges
 What constitutes good L2 writing?
 How can we link content knowledge and language
knowledge while setting some limits to the assessment of
“content”?
 How does instructed L2 writing develop?
 Does it/should it figure into the assessment of L2 writing
performance?
Writing development in the GUGD:
a “case study”
Writing tasks as genre-based writing tasks

Embedded in a four year integrated content oriented and task-based
curriculum

Learning goal of advanced multiple literacies in several languages

Explicit pedagogy for writing

Writing development integrated into L2 development in all modalities

Assessment of writing development in curricular context
Example: Level III writing task, within 5-level curriculum. Course focus “German
stories, German histories” (Post WWII – present). Unit: fall of the wall.
Foci for genre-based writing tasks
 Task Appropriateness: Understood in terms of
– Generic moves
– Positioning of writer toward an imagined reader
audience
 Content: Understood in terms of
– Information provided within each of the generic moves
 Language Focus: at 3 strata of language system
– Discourse, sentence, lexicogrammatical system
Move to handout for
 End of curricular level expected writing performance profile: Levels I –
IV of the five-level curriculum
 Specification of language foci at 3 strata of language system: Level III
 Genre “(Political) appeal” = Aufruf. Text used in class to exemplify the
genre.
 Analysis of the text’s generic moves
 Lexicogrammatical analysis of the moves “problem statement” and
“appeal for action” in terms of field, tenor, and mode.
 General guidelines for the creation of genre-based tasks
 Aufruf task sheet, in terms of task appropriateness (genre), content,
and language focus
 Feedback sheet for the writing task
 Guidelines for the assessment of writing performances
Linking content/context and language in
genre-based writing tasks: Pedagogical
considerations
 Purposeful texts approximating real-life writing in terms of topics,
possible relationships between writer and audience, purposes, textual
demands in terms of level of L2 ability
 Focus on creating meaning (content/knowledge) in a textual world
through and in and with language “forms” at all strata of the system that
should be chosen for their meaning-making capacity in relation to a
genre.
 Culturally authentic language use within the genre as culturally
validated forms of texts and registers with their preferred bundling of
lexicogrammatical features
 Development of a competent non-native voice within stable, yet flexible
generic structures
Pedagogical considerations (ctd.)
 “Conceptual space” through metalanguage for a focus on content:
manifestations of “field” in the various genre moves
 Motivated development from more narrative genres, which privilege
congruent semiosis, to more abstract, essayistic, public genres, which
privilege noncongruent (metaphorical) semiosis  long-term trajectory
toward the secondary discourses of the academy (see e.g., Byrnes & Sprang,
Christie, Macken-Horarik, Martin, Rothery, Schleppegrell)

“Plausibility”
for preferred lexicogrammatical choices – Pawley and
Syder’s “native-like selection”, particularly at the phrasal and clausal
level, increasingly more important for more advanced learners.
 Link between L1 and L2 literacies; between learning a language and
learning in other disciplinary areas.
Implications for assessment
The previous categories and pedagogical approaches to L2
writing provide ways of considering writing development as
well as writing quality
Assessment can be imagined at the intersection of a number
of the constructs SFL makes available:
 What metafunctions are handled and how well?
 What major subaspects of these metafunctions are
handled and how well?
Implications for assessment (ctd.)
 What language strata are competently used?
from word to phrase, to clause, to clause complex, to text, -- linking into cultural context, and from cultural context
back to language
 What is the breadth and depth of registerial and generic
resources available for use?
 What is the user’s location on the cline between the
resources of the system and instantiation of those
resources in a text?
Assessment (ctd.)
 What kind of knowledge is being created: congruent –
noncongruent forms of semiosis?
 What kinds of logical relationships are established and
how?
In simplex clauses? In complex clauses? What forms of
grammatical are deployed? For what functions?
 How is the phrase level expanded? Expansion of clausal
and phrasal capacity
 How is information structure textured? themes,
hyperthemes; different forms of coherence
Assessment (ctd.)
… and many more!
 In each case, language use is linked to meaning,
content, context.
BUT …
 There are also necessary limits set to what is
being assessed
Benefits of assessment for
teachers and learners
 Predicts systematically the likely linguistic features
that must be in focus
 Spells out the criteria for successful writing
 Helps teachers formalize and be more objective
about their intuitive judgments about writing quality
 Is formative and diagnostic with high potential for
washback into educational processes
Benefits (ctd.)
 Clarifies the demands associated with a particular
writing task
 Therefore has a high likelihood of enhancing L2
writing development
 Provides a language for reflecting on writing
 Links the goals of L2 writing with those in other
subject matter areas
The challenges for this approach
 Requires reconceptualizing language as a social semiotic
and language development in terms of users’ ability to
deploy generically appropriate resources for meaning
making
 Requires much higher levels of awareness (and
responsibility) by practitioners for knowing how
genres/texts work, that is, how meanings are realized in
language
 Requires similar awareness on the part of the testing
community and a willingness to reconsider a number of
constructs that have been central to assessment/testing
How to take up the challenge …
Until we know more about instructed L2 writing development,
perhaps inquiry is best located within an educational
context
- With an explicit curricular trajectory
- That integrates all aspects of educational work: curriculum,
materials development, pedagogy, assessment
- Uses to advantage the specificity that assessment
demands regarding what constitutes competent L2 writing
while also informing assessment on the nature of L2
development in writing,
- And makes appropriate adjustments to improve
educational outcomes.
… and gain large benefits
It constitutes
 Socially responsive work
– Outcomes of our programs: what is the value added on
the basis of our educative work
 Socially responsible work
– Development of sophisticated, multiple literacies in a
multilingual and multicultural world
 Enhances L2 learning
– Links content and language learning, thereby providing
perhaps the most deep-seated form of motivation for
learning languages.