Transcript Manipulating Task Complexity: its impact on L2 production across task types and modes Roger Gilabert Mayya Levkina Universitat de Barcelona TBLT Conference Lancaster September 2009 Thanks to GRAL at.

Manipulating Task Complexity:
its impact on L2 production
across task types and modes
Roger Gilabert
Mayya Levkina
Universitat de Barcelona
TBLT Conference
Lancaster
September 2009
Thanks to GRAL at the University of Barcelona, Ministerio de Ciencia e Innovación –
HUM2007-64302) and Generalitat de Catalunya – 2009SGR137).
Intro: context
Other
INTERACTION
PLANNING TIME
L2
Task –based
research into
performance
and
acquisition
TASK COMPLEXITY
TASK
FAMILIARITY
Brennan,
forthcoming
Intro: context
Other
INTERACTION
PLANNING TIME
L2
Task –based
research into
performance
and
acquisition
TASK
FAMILIARITY
TASK COMPLEXITY
TASK COMPLEXITY
“ task complexity is the result of the attentional, memory,
reasoning, and other information processing demands imposed by
the structure of the task on the language learner.”
(Robinson, 2001:28)
What may be the role of task complexity in a task-based
syllabus?
Pedagogic task
1
Pedagogic task
2
Pedagogic task
3
Pedagogic task
n
Simple
REALWORLD
TASK
Complex
GOAL
Long, 2005
Balanced development of L2
production and acquisition
The predictions of the Cognition Hypothesis
More complex
along
resourcedispersing
variables
(e.g. amount of preparation time,
familiarity, or multi-tasking)
More complex
along
resourcedirecting
variables
Less fluent
Less lexically and
structurally complex
Less accurate
Higher interaction
Less fluent
More lexically and
structurally complex
More accurate
(e.g. the number of elements or the
amount of reasoning)
Higher interaction
Findings of the impact of task complexity on production along
resource directing dimensions
+/- here-and-now
+/- elements
Fluency decreases (Robinson, 1995; Fluency decreases (Michel, Kuiken &
Rahimpour, 1997: Gilabert,
Vedder, 2007; Robinson, 2001)
2005)
Increased lexical complexity (Michel,
Increased lexical complexity
Kuiken & Vedder, 2007; Révész,
(Robinson, 1995; Rahimpour,
forthcoming)
1997; Gilabert, 2005)
No differences in structural
No differences in structural
complexity (Michel, Kuiken &
complexity (Robinson, 1995;
Vedder, 2007; Kuiken & Vedder,
Rahimpour, 1997; Gilabert,
2007; Kuiken, Vedder, & Mos,
2005)
2005) but Révész (forthcoming)
Higher accuracy (Robinson, 1995;
Rahimpour, 1997; Ishikawa,
2001; Iwashita et al.; Gilabert,
2005)
Higher accuracy (Michel, Kuiken &
Vedder, 2007; Kuiken & Vedder,
2007; Kuiken, Vedder, & Moss,
2005) but Révész (forthcoming)
+/- reasoning
demands
Fluency decreases (Niwa, 2001 )
Increased lexical complexity
(Michel, Kuiken & Vedder,
2007; Révész, forthcoming)
No differences in structural
complexity (Robinson,
2007) but Révesz
(forthcoming)
Higher accuracy (Robinson,
2007; Gilabert, 2007) but
Révész (forthcoming)
Goals and Questions
Goal: to explore the impact of task cognitive complexity
accross task types and modes.
Questions:
1) What’s the relationship between general proficiency and
performance?
2) Can differences in task cognitive complexity explain
differences in performance in both monologic and
dialogic tasks?
3) Are the effects of task cognitive complexity the same
across task types?
Experimental design
Design
Participants
Statistical analysis,
transcription, coding
3 task types
42 English-L2
Descriptive statistics
volunteers in monologic Repeated-measures ANOVA
study
RepeatedNon-parametric tests
measures design 50 in dialogic (25 dyads)
Latin square
design
2 institutions (lower/upper- intermediate)
9-point Likert
scale affective
questionnaire
X-Lex / Y-Lex
vocabulary size test
CA mode of CHILDES for
transcription of 252 tasks in
study 1 + and 150 tasks in
study 2 (302 tasks)
Intrarater (97 %//97%)
Interrater (91.5 %/90%)
Task selection: Needs analysis
TV/Cinema
Advertising
Journalism
P.R.
(task-based program at Communication Studies Department at Ramon Llull
University,Barcelona Spain)
Typically they
deal with
storyboards
for campaign
presentations
Journaslists
often have to
find their
ways in
unknown
cities
In crisis
management,
scenario
planning is an
important
aspect
Task 1: narrative
Monologic and dialogic
Operationalization
Begin the story like this: TODAY Mr. and Mrs.
Ropper are in bed. They’re trying to get to sleep
but they can hear music coming from the
apartment above theirs.
Begin the story like this: YESTERDAY Mr. Festenkroud
was shopping at the supermarket. He was checking
his shopping list and looking at prices. An employee
was putting price tags on the products.
SIMPLE here-and-now
COMPLEX there-and-then
Visual presence (here)
No visual presence (there)
Present tense (now)
Past tense (then)
Tasks and operationalization of variables
Tasks and operationalization of variables
Interactive, two-way, closed, convergent, split information
narrative task
SIMPLE
Here-and-now
•Visual presence (here)
•Present tense (now)
COMPLEX
There-and-then
•No visual presence (there)
•Past tense (then)
Results: Affective perception questionnaire
I thought this task
was easy
1
2
3
4
5
6
7
8
9 I thought this task
was difficult
I felt frustrated
doing this task
1
2
3
4
5
6
7
8
9 I felt relaxed
doing this task
I did not do this
task well
1
2
3
4
5
6
7
8
9
This task was
not interesting
1
2
3
4
5
6
7
8
9 This task was
interesting
I don’t want to
do more tasks
like this
1
2
3
4
5
6
7
8
9 I want to do more
tasks like this
Dependent Variable
Narrative
Map
Fire chief
Difficulty
,372
,013*
,006*
Stress
,765
,513
,079
Confidence
,552
,067
,005**
Interest
,371
,262
,912
Motivation
,775
,842
,530
I did this task well
(Based on Robinson, 2001)
Task 2: map task
Monologic and dialogic
Operationalization
Simple
Few landmarks
Clearly distinguishable
landmarks
One axis
(lateral= right, left,
straight)
Complex
Many landmarks
Similar landmarks
More axes
(lateral – right, left,
straight– vertical –
up, down – sagittal
– front, back).
Tasks and operationalization of variables
Interactive, one-way, closed, convergent, split information map task
Simple
Route marked
•Few landmarks
•Clearly distinguishable
landmarks
•One axis
(lateral= right, left, straight)
Route unmarked
Tasks and operationalization of variables
Interactive, oneway, closed,
convergent, split
information map
task
Route marked
Complex
•Many landmarks
•Similar landmarks
•More axes (lateral – right, left,
straight– vertical – up, down –
sagittal – front, back).
Route unmarked
Wayfinding is an important and complex task.
Landmark identification
Path selection
Direction selection
Abstract environmental overviews
Chown, E., Kaplan, S., & Kortenkamp, D. (1995)
Task 3: firechief task
Monologic and dialogic
Operationalization
SIMPLE
COMPLEX
Many resources
Few resources
No particular roles
Particular roles of characters
Few unconnected factors
Intricately connected factors
Interactive, two-way, open, convergent, shared information decision-making task
SIMPLE
•Many resources
•No particular roles
•Few unconnected factors
Interactive, two-way, open, convergent, shared information decision-making task
COMPLEX
•Few resources
•Particular roles of characters
•Intricately connected factors
Complex problem-solving tasks are situations that are:
(1) dynamic, because early actions determine the
environment in which subsequent decisions must
be made
(2) time-dependent, because decisions must be made at
the correct moment in relation to environmental
demands; and
(3) complex, in the sense that most variables are not
related to each other in a one-to-one manner. In
these situations, the problem requires not one
decision, but a long series, in which early decisions
condition later ones.
Quesada et al. (2005)
Experimental design: production
measures
Transcriptions were coded for:
-
Fluency:
Unpruned speech rate A
Pruned speech rate B
-
Structural Complexity:
Lexical Complexity:
Accuracy:
Pauses x minute
Sentence Nodes x AS-Unit.
Guiraud Index of Lexical Density
No. Of errors x 100 words
Repaired to unrepaired errors
Results: Question 1
1) What’s the relationship between general proficiency and
performance?
Proficiency x
Performance in
MONOLOGIC
Simple
Story
Complex
Story
Simple
Map
Complex
Map
Simple
Firechief
Complex
Firechief
Rate A
,792**
,613**
,590**
,613**
,557**
,586**
Rate B
,839**
,758**
,643**
,669**
,667**
,695**
Pauses
,271
,085
,336*
,261
,372*
,197
S-Nodes x
AS Unit
,208
,035
,479*
,336*
,130
,269
Guiraud
Index
,718**
,650**
,618**
,562**
,760**
,716**
Errors x 100
words
-,725**
-,680**
-,816**
-,768**
-,719**
-,738**
Rep to
unrep
,216
,158
,031
-,024
-,104
,119
Moderately strong correlation between PROFICIENCY and PERFORMANCE
Results: Question 1
1) What’s the relationship between general proficiency and
performance?
Proficiency x
Performance in
DIALOGIC
Simple
Story
Complex
Story
Simple
Map
Complex
Map
Simple
Firechief
Complex
Firechief
Rate A
,417*
,329*
-,021
,233
,226
,143
Rate B
,391*
,362*
,375
,335
,334*
,121
Pauses
-,177
-,143
,084
-,093
-,007
-,074
S-Nodes x
AS Unit
,216
,179
-,071
,038
,162
,094
Guiraud
Index
,346*
,478**
,415*
,404
,269
,356*
Errors x 100
words
-,099
-,029
,009
-,231
-,115
-,086
Rep to
unrep
-,023
,228
-,035
,140
-,036
,066
Moderately strong correlation between PROFICIENCY and PERFORMANCE
Results: Question 2
2) Is there an impact of Task Complexity on performance in
both the monologic and dialogic tasks?
Proficiency x
Performance in
MONOLOGIC
Simple Story
Complex Story
Simple Map
Complex Map
Simple Firechief
Complex Firechief
Rate A
,448
,072
,196
Rate B
,069
,227
,404
Pauses
,308
,827
,460
S-Nodes x
AS Unit
,261
,095
,638
Guiraud
Index
,286
,002
,087
Errors x 100
words
,001
,000
,777
Rep to
unrep
,009
,000
,149
Results: Question 2
2) Is there an impact of Task Complexity on performance in
both the monologic and dialogic tasks?
Proficiency x
Performance in
DIALOGIC
Simple Story
Complex Story
Simple Map
Complex Map
Simple Firechief
Complex Firechief
Rate A
,167
,372
,164
Rate B
,256
,309
,229
Pauses
,336
,972
,177
S-Nodes x
AS Unit
,830
,277
,116
Guiraud
Index
,025
,287
,008
Errors x 100
words
,707
,231
,325
Rep to
unrep
,162
,353
,468
Results: tasks compared by dimension
Fluency
140,00
130,13
127,84
Structural Complexity
128,20
120,00
100,00
80,00
60,00
40,00
20,00
0,00
Complex Narrative
Complex Instruction-giving
task
2,00
1,80
1,60
1,40
1,20
1,00
0,80
0,60
0,40
0,20
0,00
Complex Decision-making
task
1,72
5,00
5,00
4,88
Complex Narrative
5,09
3,00
2,00
1,00
0,00
Complex Instruction-giving
task
Complex Instruction-giving
task
Complex Decision-making
task
Accuracy
4,00
Complex Narrative
1,63
1,23
Lexical Complexity
6,00
Sig. difference
Complex Decision-making
task
40,00
35,00
30,00
25,00
20,00
15,00
10,00
5,00
0,00
34,05
Complex Narrative
12,07
13,14
Complex Instruction-giving
task
Complex Decision-making
task
Results: tasks compared by dimension
Fluency
Sig. difference
Struc Cplx
140
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
120
100
80
60
40
20
0
Story Complex
Map Complex
Story Complex
Firechief Complex
Lexical Cplx
Map Complex
Accuracy
6
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
5
4
3
2
1
0
Story Complex
Map Complex
Firechief Complex
Story Complex
Map Complex
Firechief
Complex
Firechief Complex
Discussion: Question 1
As expected, in the MONOLOGIC task, general
proficiency correlated strongly with performance, and
particularly with lexical complexity and accuracy.
The picture is not so clear for the DIALOGIC task,
where interaction seems to mitigate the effects of
proficiency on performance, especially with regard to
accuracy
Discussion: Question 2
In the MONOLOGIC tasks, task complexity shows an
impact on accuracy in the narrative task, while it has an
impact on both lexical complexity and accuracy in the
case of the map task. Higher tasks demands seem to
draw attention to form.
Task complexity has no impact on the decision-making
task. General measures may not be able to capture such
impact.
In the DIALOGIC tasks, task complexity seems to only
affect lexical complexity, and just for the narrative and
the decision-making task.
Discussion: Question 3
In the MONOLOGIC tasks: the map task generated less
structurally and lexically complex speech.
In the DIALOGIC tasks, task complexity seems to only
affect lexical complexity, and just for the narrative and
the decision-making task.
Conclusions
1)As in other task-based research areas (e.g. planning
time studies, task repetition, or interaction), SPECIFIC
PREDICTIONS need to be made for each TASK TYPE.
2)In the same way, predictions need to be adjusted to
EACH MODE, since behavior on monologic and
dialogic tasks differs considerably.
Limitations
1)Small sample sizes
2)Use of general measures only
3)Binary operationalizations of complexity
(simple/complex, not a continuum)
4)Not factoring in individual differences (e.g.
differences in WM capacity)
5)Not using complementary information from native
speaker performance
Ways to go from here
1)Use of more specific measures (task-related,
developmentally sound)
(Pownall, forthcoming) use of “conjoined clauses” as in
Révész (forthcoming) and NPs has found that ‘specific,
task-related measures capture the impact of task
complexity’
and,
because,
so
“The car”
If,
before,
after
“The little boy”
“The funny
little boy”
“The girl that was
reading little boy”
Ways to go from here
1)Use of more specific measures
(Pownall, forthcoming) use of “conjoined clauses” as in
Révész (forthcoming) and NPs has found that ‘specific,
task-related measures capture the impact of task
complexity’
2) Integration of Task Complexity into SEQUENCING
studies.
Thank you
Gràcies
Gracias
Members of the GRAL group: Carme.Muñoz, M. Luz Celaya, Elsa
Tragant, Teresa Navés, Joan Carles Mora, Imma Miralpeix, Raquel
Serrano, Júlia Barón, Natalia Fullana, Laura Sánchez
Interns: Mayya Levkina, Mireia, Anna Marsol
Catherine Daughty
Our students
Roger Gilabert
[email protected]
Universitat de Barcelona
Spain