Transcript University of Bristol 14th of March, 2008 Presentation
UKFIET International Conference on Education and Development 15-17 September, 2009 Oxford, England
The Politics of Formal Examinations: language policy and progress
Convenors
Pauline Rea-Dickins Guoxing Yu
2
The symposium
• Rationale: all researching in contexts where: – English as an L2/L3 is used in high stakes formal examining contexts: SSA, Malaysia, USA & UK – Politics & policy impacts on issues of fair & ethical assessment – Policy is intended to lead to progress & equitable distribution of resources & opportunities for
ALL
learners – Progress?
The symposium
Dr Ong Saw Lan (Universiti Sains Malaysia
)
Narrowing the science achievement gap with the dual-language test-booklet
Professor Jamal Abedi (University of California, Davis)
Impact of language factors on the assessment outcomes of English language learners
4
UKFIET International Conference on Education and Development 15-17 September, 2009 Oxford, England
The Politics of National Examining Processes: the case of languages in Zanzibar
Abdulla H. Mohamed, Pauline Rea-Dickins, Guoxing Yu
Student Performance in National Examinations: the dynamics of language in school achievement (SPINE)
www.bristol.ac.uk/spine
(ESRC/DfID RES-167-25-0263)
Introduction
• • • Empirical data & documentary evidence reflecting some of the realities of national examining processes: – – – Student performance on exam items Statistical data on student exam performance Implementation of classroom assessment The policy & the politics: historical perspectives: focus on CWA Conclusions : policy & politics & issues of justice in examining – progress?
6
Student performance on exam items
English RC Qn: How whales
resemble
man
• • • • 45 students took this item: 35.6% = no answer 26.7% = wrong answer 28.9% = partially correct answer 8.8% = correct answer
7
Interview: D1 who didn’t answer Q3 explains
D1: Int: D1: Int: D1: “because I did not understand by this this … resemble” (lines 115-117) “If I tell you that resemble means ‘to look like’ … can you do the question now?
“Yes” OK so what’s the answer?
“Man … is warm blooded … and whales also … whales have lungs and man also have lungs …” (122-133)
8
Biology: responses to original item
• • • • No answer Wrong answer = = Partially correct answer = Correct answer = 67.4% 21.7% 6.5% 4.4%
9
Biology: on locusts
10
Biology: on locusts
11
Biology: on locusts
Questions: a) In which picture do you think the locust will/may die?
b) Why do you think it will/may die?
12
Original item modified:
• • • • • • Greater contextualisation Simplification of instruction 2 structured parts: A & B Visual clues to support information retrieval Rephrasing of the item Altering item layout
13
Results
• •
Original item
– – Only 32.6% of students wrote an answer Just under 11% gave a partially correct or correct answer
Modified item
– – – 100% responded to this item 42% gave a partially correct answer to Part A 53% gave a partially correct answer to Part B
14
Changes in student response
OR Sc Written responses on Modified Item MOD Score
H3 0 C2 1 In picture A the locust may/will die I think it will/may dies because the locust get its breathing by using its body I think it will/may die because the boy is dipping the locust in the water to all the bodies with its trachea that used to respiration as a respiratory surface of a locust 2 + 2 2 + 2
Comment (Neil Ingram: biologist)
Understands that locusts breathe using the body Getting very close to a very complex answer “I think a very able pupil indeed”. The right answer but with great difficulty in expressing this in English
15
Means of Form II Exam Results (MoEVT)
2004 2005 2006 2007 2008
KISW 49 Islamic BIO studies 37.1
18.4
46 36 35.5
50.6
44.3
47.9
44.5
47.5
22.6
21.5
21.5
24.5
CHEM MATH PHYS 30.8
34.2
34.3
32.0
34.8
21.1
15.3
15.5
14.3
16.4
25.8
25.4
25.1
23.9
29.1
16
ENG and MATH
Total: 64.717
, school=29.18%, pupil=70.82%
ENGLISH explains (90.4 64.717)/90.4= 28.41% of the maths total variance
Total=90.4
, School=30.75%, Pupil=69.25%
ENG and BIO
Total=65.646
, school=18.17%, pupil=81.83%
ENGLISH alone explains (114.968-65.646)/114.968= 42.90% of the total variance in BIOLOGY
Total=114.968
, school=18.79%, pupil=81.21%
18
English and CHEM
Total=158.281
, school=23.06%, pupil=76.94%
ENGLISH explains (275.993 158.281)/275.993= 42.65% of the total CHEM variance
Total=275.993
, school=22.05%, pupil=77.95%
19
How about KISWAHILI & ARABIC?
• • • Although other two languages (Kiswahili and Arabic) are also significant predictors of the students’ performance in maths, biology and chemistry, it is noted that they are less capable of explaining the variances than ENGLISH. KISWAHILI explains (275.993-88.954)/275.993= 31.54% of CHEM total variance, (114.968-76.585)/ 114.968= 33.39% of BIO total variance, (90.4-75.678)/90.4= 16.29% of MATH total variance.
ARABIC explains (275.993-203.452)/275.993= 26.28% of the CHEM total variance, (114.968-88.569)/114.968= 22.96% of the BIO total variance, (90.4-71.885)/90.4= 20.48% of the maths total variance.
20
Summary of the multilevel models (a)
• • It is very clear that ENGLISH is a significant and substantial predictor of the students’ performance in MATH, BIO & CHEM .
The school-level variances explained in the cons models as well as in the models including ENGLISH as the single explanatory variable demonstrated a
substantial proportion of the variance is attributable to school factors
21 • •
Summary of the multilevel models (b)
Not much improvement in terms of
fitness of the models
(measured by the change of % of school-level variance in the total variance).
Therefore, essential to collect
further school- and pupil-level data
to examine what factors (e.g. English language learning opportunities at home and at school, academic English proficiency) and how much they account for the variances (in the tradition of school effectiveness studies) –
our plan
for nationwide data collection using pupil and headteacher questionnaires and vocabulary knowledge test
22
Coursework assessment as part of national examining: implementation issues
In Form II, Teacher Assessment (TA) was implemented based on the following assessment procedures: • • • • • Class work Oral questions Homework Weekly tests Terminal tests •
Continuous Assessment Format & Guidelines
•
Term: -------- Class: --------- subject---------
Pupils’ name X Month 1 Month Month 2 3 Month 4 Total C/work Exam Grand 60% Total 10% 10% 10% 10% 40% 100% 5 6 3 8 22 30 52
Coursework assessment as part of national examining: implementation issues
• The first three activities make up 40% and the fourth makes 60% which when combined make 100% • The final score (out of 100% ) obtained is sent to the Department of Curriculum and Examination for further processing (CWA is 25% of overall exam score)
Coursework assessment as part of national examining: implementation issues
• • • • However:
significant variability
among teachers: the types of specific activities that constitute teacher assessment the number of activities from which they take marks for each month decisions on how to handle student absenteeism how to get 10% of marks for each month; some picked the one that students did better on, some picked randomly, and some put the average of all marks
Coursework assessment as part of national examining: implementation issues
• • • Understanding of what assessment is to be done Awareness of guidelines Percentage of marks that go to the final examination • Sharing what teachers know and practice about assessment guidelines among teachers within and across schools • Level of monitoring of Teacher Assessment by head teachers.
Continuous Assessment Format & Guidelines
• •
Note:
Coursework assessment will be obtained from class works, home works and weekly tests or as it will be directed otherwise by the Department or the Ministry. All the marks scored by the pupil from the mentioned activities within a particular month, will be combined and transformed into percentage of ten (10%) before filled in the relevant space in this form.
Source:
Ministry of Education Working Guidelines: Guideline No. 9 (1995)
Continuous Assessment format & Guidelines
• •
Additional notes from another version
If the term is less or more than four (4) months, four activities chosen by the teacher him or herself should be picked at equal intervals. The teacher is advised to give as many tests as he can in a term but should choose only four for recording.
In getting pupil’s marks for terminal exam for each subject, the exam itself should contribute 60% and classroom assessments (coursework assessments) should contribute 40%
Coursework assessment as part of national examining: implementation issues
• • • The language to be used is as indicated in the lesson plan.
These notes appear to contradict with the first version which might be leading to confusion. This version requires the teacher to
find average
of marks in a month while the second requires the teacher to
choose one activity
from which the marks can be recorded.
Source:
Ministry of Education Working Guidelines: Guideline No. 9 (1995)
The Policy & the Politics: historical perpsectives
• The need to make Assessment procedures more friendly, less competitive and practically focused • The prescription that student assessment must: – – – Include works that can enable students to function effectively in their environments Cover both student performance in theoretical works and in practical productive works Be done throughout the year
The Policy & the Politics: historical perpsectives
• The National Examination Council of Tanzania (NECTA), formalised the new examination system by including projects, character assessment, exercises and tests in the assessment system • Teachers were then given responsibility to: measure students’ ability to use knowledge and skills acquired, in their local environments and, • keep ongoing records of their progress
The Policy & the Politics: historical perpsectives
•
Tanzania and Zanzibar Education Policies
Both The Tanzania Education and Training Policy (1990:79) & The Zanzibar Education Policy (2006:28) emphasised that continuous assessment combined with the final exam paper would be the basis for awarding of certificates at Secondary Education levels • For selection purposes and quality control, consistency between teacher assessment and that of the National examinations was emphasized in
The Zanzibar Education Master Plan
of 1996-2006
Language Policy and Assessment
• • Students assessment is affected by Language policy because students have to demonstrate knowledge and ability through a language accepted by the system. The Zanzibar Education Policy of 1995, Kiswahili version Section 5.72 (
28),
The Zanzibar Education Master Plan of 1996 (43), the Education Policy of 2006, all emphasized the use of English Language as a language of instruction for all subjects in secondary education except for Kiswahili and Islamic studies.
Language Policy and Assessment
So, English, as a language of instruction in teaching science, maths and so forth at secondary level must be a language of assessment Does this have to be the case?
35
Policy & Politics: progress for learners & learning?
Given that English is politically& policy wise the language of the examinations: • • • “No test taker shall be harmed by the test (Hamp Lyons 1989:13) No learner should be disadvantaged, in the interests of justice for
ALL
learners Ethical issues re.
fairness
: need to give individual a fair chance
36
Policy & Politics: What progress towards social justice?
Impact/Potential Disadvantage (examples) Consequences/Injustice: (examples)
• Learners do not engage or respond poorly in examinations • Subject area (e.g. Biology, maths) construct can only be assessed where a linguistic construct has been successfully negotiated • Loss of self-esteem & motivation for learning • Learners fail to reach their potential (glass ceiling effect) or fail altogether • Leave school as unsuccessful (e.g. at end of Basic Education) • Unequal access to available resources, educational experiences & work opportunities • Unskilled & unable to join the workforce in turn contributing to social & economic deprivation
37
Policy & Politics: What progress towards social justice?
Impact/Potential Disadvantage Consequences/Injustice
• Teachers do not use full range of LTA procedures & processes • Use of inappropriately constructed assessment frameworks • Inaccurate CWA of learners • CWA implemented as a series of tests • Learners not fully supported in their language & content knowledge development & fail to reach potential • Test performance valued over learning • Leave school with poor educational outcomes • Inadequate d-base for decision making about student & learning progression