Learning Assessments

Regional and Global Initiatives Dr John Cresswell SEA-PLM Penang, October, 2014 Australian Council for Educational Research

Overview

• • •

Global Assessments

– PIRLS, TIMSS, PISA, ICCS, EGRA

Regional Assessments

– SACMEQ, PASEC, LLECE – ASER/Uwezo

Learning Metrics Taskforce

Change happens - Korea

• • In Korea, in the 1950s, the literacy rate was 22% and less than 20% of children attended secondary school. Education reforms undertaken by Korea led to them having one of the most advanced education systems in the world.

20 10 0 50 40 30 100 90 80 70 60 13 1

Change happens - Korea

Approximate percentage of adults with high school

1990s

qualifications

1980s 1970s 1960s

1 27

Change happens - Finland

• • Until the late 1960s, in Finland most children left public school after six years. Some children went to private schools, academic grammar schools or folk schools. At that time only the privileged and fortunate were exposed to quality education.

A series of reforms was undertaken over many years and in 2000 Finland achieved at the highest level in PISA.

Education reform cycle

• • Assessment is just one part of the process of educational improvement.

Simply doing more assessment will not, by itself, improve educational outcomes of students.

Reform cycle

Education reform is not a straight line activity Reform discussion Student assessment and reporting Policy dialogue and decisions Policy implementation Teaching and learning

Regional and International Assessments

• • • In 1964, the International Association for the Evaluation of Educational Achievement (IEA) conducted the first internationally comparative study in mathematics; 12 countries participated.

By 2009, about 70 per cent of the countries in the world participated in some form of regional or international assessment program. Since the 1980s, the number of national assessment programs has also increased.

PIRLS (Progress in Reading Literacy

Study) • International Association for the Evaluation of Educational Achievement (IEA) • Sample – grade 4.

• Subject – reading.

• Focus – curriculum.

• Other – student, parent, teacher and school questionnaires.

PIRLS 2011

300 350 400 Hong Kong SAR 450 500 550 600 Singapore Chinese Taipei New Zealand Australia Iran, Islamic Rep. of Indonesia

The boundaries indicate the 95% confidence limits The centre line is the estimated mean for the country 500 is the mean international score

TIMSS (Trends in Mathematics and

Science Study) • International Association for the Evaluation of Educational Achievement (IEA) • Sample – grade 4 and grade 8.

• Subjects – mathematics and science.

• Focus – curriculum.

• Other – student, teacher and school questionnaires.

ICCS – the International Civics

and Citizenship Study • • • Preparing young people to undertake their roles as citizens Two dimensions of civics & citizenship – Student knowledge and understanding – Attitudes, perceptions and activities Focus of ICCS reporting – International comparisons – Variations within countries – Factors explaining variation

PISA – Programme for International

Student Assessment • • • • • • • • Organisation for Economic Cooperation and Development (OECD) Reading, mathematics, science Also problem solving, financial literacy Student and school questionnaires Sample is 15-year-olds 72 systems in PISA 2015 Focus in preparedness for the future – ie not curriculum based.

The OECD also conducts TALIS and PIAAC.

EGRA - Early Grade Reading

Assessment • • • US AID/RTI Also Early Grade Mathematics Assessment (EGMA) Implemented in over 70 countries in 100 languages.

SACMEQ - Southern and Eastern

African Consortium for Monitoring Educational Quality • 15 countries; since 1995.

• Compares student performance, and the factors associated with that performance.

• Capacity building has been a major focus of SACMEQ

LLECE Latin American Laboratory

for the Evaluation of Quality in Education • 15 countries; since 1997. • second assessment (SERCE) study in 2007. • TERCE followed.

PASEC -

Programme for the Analysis of Educational Systems of CONFEMEN • • The Conference of Education Ministers of Countries which include French as a Language of Communication (CONFEMEN) First assessment 1991; 24 participating countries in a series of assessments.

• The first phase of PASEC had tests at the beginning and end of grades 2 and 5, but is now moving into a new phase with just one test in each of grades 2 and 6.

ASER and Uwezo

• • • Household based assessments.

The Annual Status of Education Report (ASER): implemented annually in India for the last ten years, 600,000 out-of-school children.

Uwezo: an ASER based assessment in Africa. Kenya, Tanzania, Uganda and in the future Senegal and Mali.

Regional participation

PIRLS PISA TIMSS ICCS EGRA PASEC Country Brunei Cambodia Indonesia Lao PDR Malaysia Myanmar Philippines Singapore Timor Leste Thailand Vietnam 2001 2006 2011 2000 2003 2006 2009 2012 1995 1999 2003 2007 2011 2009 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • * • • • 2012 • • •

What do countries learn by participating?

• Gauge of educational health – Calls to action when results surprise – PISA: eg Germany, Japan – Setting of national expectations that are consistent with performance throughout the world

What do countries learn by participating?

• Influence on curriculum orientation and content – TIMSS (the role of statistics, probability and data in the mathematics curriculum) – PISA and its literacy orientation – Non-continuous text as an element of reading • Capacity development – Assessment task development – Research design, sampling, scaling, analysis…

What do countries learn by participating?

• • Trend information – Is our performance increasing or decreasing?

– Have gender differences narrowed?

– Has participation of girls increased?

Policy development – Inform options for policy – Identify correlates of outcomes (eg teacher qualifications, resource access)

LMTF - Learning Metrics Taskforce

Post-2015 Development Goals • • The overarching goal of the Learning Metrics Task Force is to catalyse a shift in the global conversation on education from a focus on simply access to a focus on access plus learning.

Assessment is a necessary part of the process to find out if students are learning.

Area of Measurement Learning for All:

Age and Education

Matter for Learning: Reading: Description of Indicator Combine measures of completion and learning (reading proficiency at the end of primary school) into one indicator. Measure timely entry, progression and completion of schooling, and population-based indicators to capture those who do not enter or those who leave school early. Measure foundational skills by Grade 3 and proficiency by the end of primary school. Numeracy: Ready to Learn: Citizen of the World:

Breadth of Learning

Opportunities: Measure basic skills by end of primary and proficiency by lower secondary school. Measure acceptable levels of early learning and development across a subset of domains by the time a child enters primary school. Measure among youth the demonstration of values and skills necessary for success in their communities, countries and the world. Track exposure to learning opportunities across all seven domains of learning.

LMTF 2.0

Learning Metrics Partnership • • • Strategic partnership between LMTF, UIS and ACER to develop common global metrics for reading and numeracy Opportunities for alignment between SEA-PLM and the global metric ACER is a member of the LMTF and is providing all technical support for the development of the common metrics