Transcript www.cem.org
CEM Primary Overview
Formative role
Means of providing feedback to teachers about on going progress in learning. It has a direct influence on the quality of pupils’ learning experiences and thus on the level of attainment which can be achieved.
Summative role
It is the means for communicating the nature and level of pupils’ achievements at various points in their schooling and when they leave.
Certification role
It is used as a means of summarising, for the purposes of selection and qualification, what has been achieved.
Evaluative or quality control role
It provides part of the information used in judging the effectiveness of educational institutions and of the system as a whole.
(Harlen, Gipps et al, 1992)
Pupils are exposed to assessment on a daily basis Observations of play Listening to reading Oral questioning Spelling tests Teacher assessment Standardised assessment Informal Low stakes The role of assessment can be unclear National testing e.g. SATS, GCSE Formal High stakes
The social consequences of using a particular test for a particular purpose Standardised assessment National testing e.g. SATS, GCSE • • The higher the stakes, the more likely decisions will be made about pupils that may affect their educational future The higher the stakes, the more important it becomes that the assessment is good i.e. it measures fairly and measures well
Developed from items that are good indicators of later progress Reliable •Test/retest (around 0.98) •Internal (0.8 – 0.98) •Concurrent (between 0.71 and 0.86 with KS2 SATS) Manageable Appropriate for the child’s ability Equitable
Special educational needs Physical or emotional disabilities Diverse language and cultural backgrounds
Studies from the US and the UK have shown English Language Learners (ELLs) to have lower scores than their native English speaking peers It is not a white vs black issue It is not entirely a socio-economic issue Most studies based on high-stakes assessment
Coleman report 1966 ◦ Verbal and non-verbal reasoning, reading and maths Swann report 1985 ◦ West Indian children underachieving, IQ not significant factor Harvard meta-analysis ◦ 11 studies with 23,000 participants ◦ ELLs have lower scores in maths and science
Department for Education, 2010 ◦ On average, children of any black background achieved below the national level in reading, writing, mathematics and science at KS1 and KS2 TIMSS 2008 ◦ Children whose parents are both born in the UK were likely to score higher in maths ◦ Children who always or almost always spoke language of the test generally more able
To what extent is the test itself responsible for some of the gap?
Assessments developed in the UK Stage by stage approach to looking at how the assessments work for international students First evidence comes from ◦ Vocabulary assessment ◦ Reading assessment ◦ For 5-11 year olds
UK-based sample Sample from group of international schools in East Asia
Compared the responses of each sample to each question in the test Differential item functioning analysis used to see whether items were easier for one group or another
Reading – Word recognition
Reading – Word decoding
15 10 5 0 -5 -10 Do Go HK08 NI07
Vocabulary
Daffodil Drenched Saxophone Lantern Luggage Aquarium Transport
The international pupils find some questions harder to answer than others Roughly the same number of questions are easier for the international pupils The sections of CEM assessments analysed do not appear to advantage non-ELL pupils
Various accommodations have been suggested to level the playing field Making specific changes to the test format or test conditions Varying degrees of success Findings must be: ◦ Effective ◦ Valid
Effect size reports the magnitude of the difference in achievement between the two groups 0.2 small effect 0.8 large effect
0.15
– a
Simplified English English dictionaries or glossaries Bilingual dictionaries or glossaries Tests written in native language Dual language test booklets Dual language questions for English passages Extra time What worked?
small effect
Kieffer, Lesaux, Rivera and Frances, 2009
Only English language dictionaries or glossaries was found to have a positive and significant effect Practical impact of use of English dictionaries or glossaries might be reduction of achievement gap by between 10% and 25% The accommodation was found to be valid as it did not improve the non-ELL scores This was true for a homogenous group of students Not controlled for other variables e.g. non-verbal ability
Reduce language load – helps all students Include graphical or visual support Include local and situated perspectives in test development Provide alternative norms Test preparation support ◦ Not violating ethical norms ◦ Should not increase scores without corresponding increase in mastery of the curriculum
Despite our best efforts, no test is likely to measure all students fairly – so how can standardised tests, which impact upon real lives, be made equitable?
Further examination of achievement gaps Appropriate interpretation of standardised scores for certain groups Development of interpretative tool
Fairbairn, 2009. Inclusive achievement testing for linguistically and culturally diverse test takers Kieffer, Lesaux, Rivera and Frances, 2009. Accommodations for English Language Learners Taking Large-Scale Assessments: A Meta-Analysis on Effectiveness and Validity.
2005 National Assessment of Educational Progress, National Center for Education Statistics Jamal Abedi, Professor of Education, Vanderbilt University, Tennessee AERA (American Educational Research Association) BERA (British Educational Research Association)
If you are interested in being involved in research on equitable assessment, please contact me.