Transcript TASK-BASED ASSESSMENT IN LANGUAGE LEARNING PROGRAMS: PIECES OF THE PUZZLE Geoff Brindley Macquarie University, Sydney, Australia.

TASK-BASED ASSESSMENT IN
LANGUAGE LEARNING PROGRAMS:
PIECES OF THE PUZZLE
Geoff Brindley
Macquarie University,
Sydney, Australia
Structure of the presentation
1. Introduction
2. Task-based assessment: Definitions and
features
3. Implementing TBA: Differing stakeholder
perspectives
-Theoretical perspectives
-Educational versus managerial perspectives
4. The need for assessment literacy
5. Conclusion
Defining task-based assessment
“… the process of evaluating, in relation
to a set of explicitly stated criteria, the
quality of the communicative
performances elicited from learners as
part of goal-directed, meaning-focused
language use requiring the integration
of skills and knowledge” (Brindley,
1994)
Positive features of TBA


Teachers' and learners' attention is able to be
more focused on language as a tool for real
world communication rather than on
language knowledge as an end in itself, thus
potentially improving student motivation.
Assessment can be integrated into the
learning process through the use of
attainment targets which are directly linked to
course content and objectives.
Positive features of TBA (contd)


Learners can obtain useful diagnostic feedback on
their progress and achievement since explicit
performance criteria are provided against which they
can compare their performances.
Better communication between users of assessment
information and educational institutions can be
established through the use of various forms of
outcome reporting which are couched in performance
terms and are hence intelligible to non-specialists.
The shift to teacher-led assessment
UK
-Teacher assessment given greater importance
following review of National Curriculum Assessment
-Teacher assessment used for reporting from
2007 in Wales
-Government “Excellence and Enjoyment” report (DfES
2003) endorses greater role for teachers in assessment
-Tomlinson Report (2002) endorses use of teacher
assessment for national reporting

The shift to teacher-led assessment
(contd)

USA
-Widespread adoption of “authentic” TBA
in schools & adult education
-Use of authentic teacher-conducted
assessments in some high stakes contexts (eg
Kentucky, Nebraska)
Differing theoretical perspectives on
TBA: What is the construct?

“Weak” versus “strong” view of TBA:
“…in the strong sense, tasks will
represent real-world tasks, and
performance will be judged on realworld criteria, that is, the fulfilment of
the task set” (McNamara 1996)
Differing theoretical approaches to
TBA (contd)

“In second language performance tests
in the weak sense, the focus is on
language performance….The candidate
is required to perform on a task which
may represent tasks he or she may
subsequently face in the real world;
however, the capacity to perform the
task is not the actual focus of the
assessment”
The problem of generalizability

“While the ‘weak’ view is likely to assess
underlying language skills in ways which are
relatively broadly generalizable, the ‘strong’
view is likely to produce judgments which are
more authentic and relevant to the real life
situations towards which candidates may be
moving. These judgments about the quality
of performance may not, however, be
replicable in other contexts” (Wigglesworth
2008).
Some other unresolved theoretical
issues in TBA
The difficulty of “difficulty”
 How can task difficulty be defined &
measured?
 What are the factors affecting task difficulty?
 Can task difficulty be modelled and predicted?
(Bachman, 2002, 2007; Brindley & Slatyer, 2002; Brown
et al, 2002; Elder et al 2002; Ellis, 2008; Robinson,
2001; Skehan, 1998, 2001; Tavakoli, 2009)
Educational versus managerial
perspectives
“ Whereas politicians and government officials
tend to see assessment as a tool for
implementing and managing policy, teachers
and educationists are primarily concerned
with ways in which it can be used for the
improvement of learning “ (Brindley, 2008)
Formative TBA: The educational
policy maker perspective
Effective assessment for all pupils should:
-recognise what pupils can do and reward achievement
-be based on different kinds of evidence
-be a valid reflection of what has been taught or
covered in class
-be reliable in terms of enabling someone else to repeat
the assessment and obtain comparable results
-be manageable, both in terms of the time needed to
complete the task, and in providing results which can
be reported or passed on to other teachers (DfES,
2003:2)

The formative purpose of TBA:
rhetoric or reality?
No reference to:
 monitoring learners’ emerging language
awareness and development: achievement is
highlighted
 assessment as integrated within instructional
discourse. “Taught or covered” suggests oneoff measurement focused assessment
 formative assessment as an on-going process
Leung and Rea-Dickins, 2007)
Popular beliefs about testing &
assessment





There is a test for every population/purpose
Norm-referenced tests are a fair and
objective measure of student ability
Standardized testing raises standards
All tests have pass marks (usually 50%)
Teachers can’t be trusted to do their own
assessment
The media critics



Formative assessment also embraces a
developmental approach to learning, based on the
argument that "students develop and learn at
different rates and in different ways"…
The result? Instead of pass or fail, student progress
or lack of progress is clouded by such politically
correct terms as beginning, established, consolidating
or emerging, solid, comprehensive.
Instead of students facing regular examinations with
consequences for failure, as do those students in
stronger performing education systems overseas,
students are automatically promoted from year to
year, even though many have not mastered the
basics (Donnelly 2005)
The politicians
“The reports I saw allowed for the teacher to assess
students from a range of choices – usually,
consolidating, sometimes, and not yet’. What kind of
nonsense is this?
The educational “experts” with whom I seem to be in
constant battle, give me the constant refrain of
“outcomes assessment”. The ranking of students
against one another is opposed by teacher
advocates. Try telling that to parents. Worse still,
what do they think happens in the real world?
(Nelson 2005)
“ Politicians are simple people; they
like simple people; they like simple
choices and clear guidance.”
(Yes Minister)
The role of assessment literacy

“..training for assessment literacy entails an
appropriate balance of technical know-how,
practical skills, theoretical knowledge, and
understanding of principles but all firmly
contextualized within a sound understanding
of the role and function of assessment within
education and society” (Taylor, 2009)
Assessment literacy for all
“ …an appropriate level of assessment literacy
needs to be nurtured not just among
engineers and technicians who are actively
involved in test development or research
activities, or even among applied linguists
and language teachers…but more broadly in
the public domain if a better understanding of
the function and values of assessment tools
and their outcomes is to be realized
throughout society” (Taylor, 2009)
Assessment and the language teacher:
What skills & knowledge do they need?


Teachers should be skilled in choosing
assessment methods appropriate for
instructional decisions.
Teachers should be skilled in administering,
scoring and interpreting the results of both
externally-produced and teacher-produced
assessment methods
Assessment and the language teacher:
What skills & knowledge do they need?


Teachers should be skilled in developing,
using and evaluating valid student grading
procedures which use student assessments.
Teachers should be skilled in communicating
assessment results to students, educational
decision makers and other concerned
stakeholders.
Assessment and the language teacher:
What skills & knowledge do they need?


Teachers should be skilled in using
assessment results when making decisions
about individual students, planning teaching,
developing curriculum, and institutional
improvement.
Teachers should be skilled in recognizing
unethical, illegal, and otherwise inappropriate
assessment methods and uses of assessment
information
Components of assessment literacy
(Bailey & Brown, Brindley, 2001, InbarLourie, 2008, Taylor, 2009)





The social and political context of assessment
Defining and describing language proficiency
Constructing and evaluating tests and
assessment tasks
Assessment in the curriculum
Putting assessment into practice
Professional development on
assessment: some guiding principles
(Brindley 2001)





Involve the whole system
Capitalize on existing practices
Recognise and address the reality and
constraints affecting assessment
Encourage a research orientation
Plan for change
Teacher involvement in collaborative
test/assessment task development

“…the discussion time and the time to sit
down with people and discuss something has
been very valuable. And then I’m discussing
with other people who are also interested in
these things. And they ask me about tasks
sometimes, they bring me a task and say
‘What do you think about this’?”
Teacher involvement in collaborative
test/assessment task development
“ After doing this project, I realized that setting
a test paper is not an easy task. Instead of
testing what is easy to test, we have to
construct tests that we really discover how
successful the learning experiences had been
for the students rather than to show in what
respects they had been deficient” (Teacher
participant cited in Coniam 2009).
And in conclusion..

“We believe that a new breed of assessment
literate educators with extensive experience
in classrooms will play an important role in
implementing assessment policies that truly
support student learning”.
(Lukin et al 2004)