Transcript Slide 1
English as a second language and assessments to assist pupil learning Katharine Bailey CEM assessments • Approach depends on intended use of data • Accommodations permitted • Developed ability measures Scores Charts IPRs (Individual Pupil Record Sheets) Look for sections that are inconsistent Also available based on MidYIS, Alis, SOSCA & INSIGHT scores Achievement gap • Studies from the US and the UK have shown English Language Learners (ELLs) to have lower scores than their native English speaking peers • It is not a white vs black issue • It is not entirely a socio-economic issue • Most studies based on high-stakes assessment Research • Coleman report 1966 – Verbal and non-verbal reasoning, reading and maths • Swann report 1985 – West Indian children underachieving, IQ not significant factor • Harvard meta-analysis – 11 studies with 23,000 participants – ELLs have lower scores in maths and science Research • Department for Education, 2010 – On average, children of any black background achieved below the national level in reading, writing, mathematics and science at KS1 and KS2 • TIMSS 2008 – Children whose parents are both born in the UK were likely to score higher in maths – Children who always or almost always spoke language of the test generally more able Effect size • Means and standard deviations calculated for the two groups • Effect size reports the magnitude of the difference in achievement between the two groups International schools • Lack of large scale research into ELL issues in international schools • Why? – Perception of the problem – Strategic perspective/accountability • Difficulties of classification CEM research • Assessments developed in the UK • Stage by stage approach to looking at how the assessments work for international students • First evidence comes from – Vocabulary assessment – Reading assessment – For 5-11 year olds Participants • UK-based sample • Sample from group of international schools in East-Asia Analysis • Compared the responses of each sample to each question in the test • Differential item functioning analysis used to see whether items were easier for one group or another Vocabulary Drenched Daffodil Saxophone Lantern Luggage Aquarium Transport Reading – Word recognition Reading – Word decoding -5 -10 Fog Still Cliff Class Seat Water Snail Brain Wash Book Jar Curtain Squeak Found Chair Happy Carry Speech Thread Ear Adventure Dream Looking Curve Hospital Do Go Candle Force Interesting Knife Operate Cubicle Probably Raising Curious Wound Heavier Machine Pressure Poisonous Fright Creature Create Cycle Science Although Disqualify Devious Obey Avoid Special Infection Operation Taught Necessary Photograph Receive Believe Unofficial Illegal Vehicle Accommodation Carriage Neighbour Enough Descend Design Receipt Ferocious Fascination Cynical Phenomenon Rhythm Chasm Reading – Spelling Spelling difficulties 15 10 5 0 What does this tell us? • The East Asia sample find some questions harder to answer than others • Roughly the same number of questions are easier for the East Asia sample • The sections of CEM assessments analysed do not appear to advantage non-ELL pupils What can the US studies tell us? • Various accommodations have been suggested to level the playing field • Making specific changes to the test format or test conditions • Varying degrees of success • Findings must be: – Effective – Valid What worked? • • • • • • • 0.15 – a small effect Simplified English English dictionaries or glossaries Bilingual dictionaries or glossaries Tests written in native language Dual language test booklets Dual language questions for English passages Extra time Kieffer, Lesaux, Rivera and Frances, 2009 Findings • Only English language dictionaries or glossaries was found to have a positive and significant effect • Practical impact of use of English dictionaries or glossaries might be reduction of achievement gap by between 10% and 25% • The accommodation was found to be valid as it did not improve the non-ELL scores • This was true for a homogenous group of students • Not controlled for other variables e.g. non-verbal ability Other recommendations • Reduce language load – helps all students • Include graphical or visual support • Include local and situated perspectives in test development • Provide alternative norms • Test preparation support – Not violating ethical norms – Should not increase scores without corresponding increase in mastery of the curriculum Fairbairn and Fox, 2009 Way forward • Further analysis of item level data in different subjects • Gather more detail through questionnaire or interview – Details of language use at home, with family and with friends – Literacy experiences – Level of parental education – History of education experiences – Age of arrival • Re-contextualisation of assessment information More information • Fairbairn, 2009. Inclusive achievement testing for linguistically and culturally diverse test takers • Kieffer, Lesaux, Rivera and Frances, 2009. Accommodations for English Language Learners Taking Large-Scale Assessments: A Meta-Analysis on Effectiveness and Validity. • 2005 National Assessment of Educational Progress, National Center for Education Statistics • Jamal Abedi, Professor of Education, Vanderbilt University, Tennessee • AERA (American Educational Research Association) • BERA (British Educational Research Association)