Transcript SPINE - University of Bristol

DFID/ESRC RES-167-25-0263
Measuring student subject learning: in whose language(s)?
Guoxing Yu, Pauline Rea-Dickins, Zuleikha Khamis*, Oksana Afitska, Haji Mwevura*, Shumbana Said*, Amour Khamis*, Abdulla Mohammed*
University of Bristol, *State University of Zanzibar
Student Performance in National Examinations: the Dynamics of Language (SPINE, 2007-2010)
1. 1 Introduction
3. Further questions
6. Impact on teaching and learning
1. To what extent does the language background of test
takers affect their performance in content-based
assessment delivered in English?
Three aspects of classroom learning and language use:
i) Code switching and mixing and discourse features of
teacher talk, e.g. ‘safe talk’ (Hornberger & Chick (2001)
2. To what extent do test takers’ English language abilities
(esp. reading and writing skills) affect their performance in
such assessments?
accommodating students’ as well as teachers’ low language ability,
translation as coping strategy (Brock-Utne & Holmarsdottir 2004).
ii) Classroom pedagogy: a limited range of pedagogic activities to
3. To what extent is test takers’ performance affected by
language complexity and linguistic demands of test items?
engage learners in subject and language learning, within environments
with very limited resources available.
iii) Learner classroom language: lack of opportunities for
4. How might test accommodations (e.g. modifying linguistic
demands of items) affect performance in content-based
assessments?
participation in class e.g., due to the unfamiliarity with the language of
the instruction.
On a school wall in Kiswahili
1.2 Context
4. Examples of test items
• NCLB Act of 2001 (USA) and Every Child Matters green
paper (2003, UK)
Maths
• World Declaration on Education for All and Framework for
Action (UNESCO, 1990)
1. Given that a2-b2=(a+b)(a-b). Evaluate (204)2-(196)2
• For students in SSA, English is an exoglossic language and
their teachers are learners of English too and are not
necessarily competent users of English.
• … that poor countries often operate expensive and often
complex language policies, whereas rich countries usually
operate simple and relatively cheap language policies .
(Williams 2006:1987)
2. Focus of Study 1
1. What evidence is there that language of examination
constitutes a determining factor in the demonstration of
subject knowledge of students whose first language
differs from the language of examination
2. What assessment accommodations (including linguistic
accommodations) are capable of addressing such
linguistic unfairness in examinations?
3. What are the social and political impact of language of
examination in Sub-Saharan African contexts?
Language of examinations
Impacts on individuals:
performance and progress
Students
Test performance
Impacts on society and
educational policies:
national and international
Teaching and
learning
Interviews with teacher (video) and learners (audio) on the
use of L1 and L2 in science learning
• Calculate the lateral angles in the figure below.
2. In a certain hotel of 30 customers, 17 enjoy Cocacola, 19
enjoy Fanta and 2 don’t enjoy either. Fill in the venn
diagram below and use it to find how many enjoy: i)
Cocacola only, ii) Fanta only and iii) both Cocacola and
Fanta?
4. The combined ages of Juma and Asha are 10 years. The
difference of Asha’s age from twice Juma’s age is 8
years. Find the ages of each one.
Chemistry:
1. Name the process that could be used to obtain the
named compound in the following mixture a) coconut oil
from water, b) iodine and sand, c) salt from salt solution
Biology:
• Give a brief explanation on what will happen on the
following: A locust not suffocating when its head is
immersed in water while the remaining part of body is
outside.
5. Impact on individuals in tests
1. AERA/APA/NCME 1999: 91
For all test takers, any test that employs language is, in
part, a measure of their language skills. This is of
particular concern for test takers whose first language is
not the language of the test.
2. Effects of language background/ability on test
performance
3. Then, can TEST ACCOMMODATIONS (e.g.
modification of linguistic complexity of items,
use of native language, see Butler and Stevens
1997, 2001) provide a quick fix as the most
direct and immediate policy remedy to address
the complex issues surrounding “language in
examinations”?
4. Effects of accommodations on performance:
(a) Modifications of linguistic demands of items
Studies using linguistic modifications for EAL
learners in content-based assessment have
drawn on conceptually from findings of
psychological studies where a changing in the
wording or structure of a test item has been
observed to affect students (mainly L1 learners)
performance, however, in practice, the effects of
linguistic modifications on test performance on
EAL learners appear much more complex,
interacting with students language abilities, type
of items and subject areas, etc.
(b) Using L1, or dual-language or side-by-side
bilingual tests, e.g.
Duncan et al. (2005) and Abedi et al. (2006) both found
dual-language maths tests did not affect
performance, though preferred by test takers (Duncan
et al. 2005).
Eisemon et al. (1993): the language of assessment,
French or Kirundi, profoundly influenced the
measurement of achievement in most subjects tested.
And the most able students were most affected by
being measured through French.
7. Social and political impacts
1. The opportunity to use examinations as a lever for change
(Kellaghan & Greaney 1992, 2004) in monitoring and
improving education quality in SSA are often missed,
misused or even abused, leading to a “serious waste of
scarce educational resources” (Kellaghan & Greaney 2004:
13), and
2. rather seriously, raises issues of social and individual
inequality with discrimination “against minorities, rural
populations, girls, and students whose first language
differs from that of the examination” (ibid. 7)
3. and raises at least three questions in relation to
educational policies:
i) Whether the problems of underachievement are related to
a language of instruction that is different from the
students’ home language as well as their teachers’?
ii) Whose language(s) should be used as a medium of
instruction to meet the targets of quality Education for All?
iii) Which language(s) should be used for formal and high
stakes examinations?
8. Some conclusions on the use of assessment
accommodations
1. There are significant conceptual, methodological and political
challenges of applying accommodations in a reliable, valid and
feasible manner.
2. The inconsistency of research findings may be partly due to
the complex nature and the interactions of accommodations
with numerous other factors such as subject areas, student
characteristics and the ways in which accommodations are
implemented.
3. It is crucial to look beyond accommodations: to analyze the
impact of instruction-embedded factors and the ways in which
learners examination performance may be shaped by the
nature of language use in teaching and learning in order to
gain better understandings of the interrelationships between
the language of assessment and the language of instruction.