Transcript No Slide Title

Lessons from international comparisons
Barry McGaw
Director for Education
Organisation for Economic Co-operation and Development
Forum to Celebrate First Anniversary of
Education for a New Era
Educational Reform in Qatar
1
Doha, Qatar
15 March 2005
Messages from OECD’s
Programme for International Student
Assessment (PISA)
2
Key features of PISA 2003 assessment

Information collected

Subject matter coverage
– Mathematics, Science, Reading, Problem solving

Volume of questions
– 3½ hours of mathematics assessment
– 1 hour for each of reading, science and problem solving

From each student
– 2 hours on paper-and-pencil tasks (subset of all questions)
– ½ hour for questionnaire on background, learning habits, learning
environment, engagement and motivation

From school principals
– questionnaire (school demography, learning environment quality)

Geographic coverage


3
275,000 15-year-old students randomly sampled
43 countries in 2000/2000+, 41 in 2003, 59 in 2006
Overall performance of education systems
What PISA shows students
can do in mathematics.
4
PISA Proficiency Levels in Mathematics
OECD
Level 6
Level 5
10
%
Level 4
18
%
Level 3
22
%
Level 2
21
%
Level 1
15
%
Below
Level 1
5
4%
11
%
Conceptualise, generalise and use
information based on investigations and
modelling of complex problems. Link and
move between different information
sources and representations.
Extract and use relevant information
from single source. Employ basic
algorithms, formulae, procedures.
Answer questions in familiar contexts
where all relevant information present.
Carry out routine procedures.
Unable to use mathematical skills in
ways required by easiest PISA tasks.
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 2.5a, p.354.
0%
20%
6
Mexico
Turkey
Greece
Italy
Portugal
USA
Spain
Hungary
Poland
Luxembourg
Norway
Slovak Rep
Ireland
Germany
Austria
Level 5
40%
80%
Sweden
Level 6
Level 4
60%
France
Denmark
Iceland
Czech Rep
New Zealand
Australia
Switzerland
Belgium
Canada
Japan
Netherlands
Korea
Finland
% at each mathematics proficiency level (OECD)
Finland:
7% at Level 6
7% below Level 2
1.5% below Level 1
USA:
2% at Level 6
26% below Level 2
10% below Level 1
Level 3
Level 2
Level 1
100%
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 2.5a, p.354.
Below Level 1
0%
60%
80%
7
100%
Hong Kong-China
Finland
Korea
Netherlands
Liechtenstein
Japan
Canada
Belgium
Macao-China
Switzerland
Australia
New Zealand
Czech Republic
Iceland
Denmark
France
Sweden
Austria
Germany
Ireland
Slovak Republic
Norway
Luxembourg
Poland
Hungary
Spain
Latvia
United States
Russian Federation
Portugal
Italy
Greece
Serbia
Turkey
Uruguay
Thailand
Mexico
Indonesia
Tunisia
Brazil
% at each mathematics proficiency level (All)
Level 6
Level 5
20%
Level 4
40%
Level 3
Level 2
Level 1
Below Level 1
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 2.5a, p.354.
Overall performance of education systems
Countries vary markedly in terms of:
distributions of performances
mean performances
and at sub-national levels - geographic, linguistic.
8
Equity of learning opportunities
Measured by relationship between students’ and
schools’ socio-economic status and performance.
9
High
Mathematics performance
Hong Kong-China
High performance
Finland
Korea
Low social equity
Netherlands
Japan
Canada
Belgium
Switzerland
Australia
New Zealand
Liechtenstein
Macao-China
Strong relationship
between social status
and performance
Ireland
Poland
Latvia
Russian Federation
Portugal
10
Czech Republic
Iceland
Denmark
France
Sweden
Austria
Germany
Slovak Republic
Norway
Luxembourg
Hungary
High performance
High social equity
540
520
Moderate relationship
between social status
and performance
Spain
United States
500
480
Italy
Low performance
Low performance
Low social equity
High social equity
Low
Greece
Performance
460
High
Mathematics performance
Hong Kong-China
High performance
Netherlands
Low social equityLiechtenstein
Belgium
Strong relationship
between social status
and performance
Hungary
Germany
Czech Republic
Denmark
France
Sweden
Austria
Ireland
Slovak Republic
Poland
Portugal
High performance
High
Japan
Canada
Switzerland
Australia
New Zealand
Norway
Luxembourg
United States
11
Finland
Korea
social equity
540
Macao-China
520
Iceland
Moderate relationship
between social status
and performance
Spain
Latvia
500
480
Russian Federation
Italy
Low performance
Low performance
Low social equity
High social equity
Low
Greece
Performance
460
Equity of learning opportunities
Countries vary markedly in terms of:
relationship between students’ social background and
their educational achievement
12
Consistency of performance across schools
Between and within-school variation
in performance.
13
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
Iceland
Finland
Norway
Sweden
Poland
Denmark
Ireland
Canada
Spain
New Zealand
Australia
United States
Mexico
Portugal
Luxembourg
Switzerland
Greece
Slovak Republic
Korea
120
Netherlands
Austria
Germany
Italy
Belgium
Japan
140
Czech Republic
14
Hungary
Turkey
Variation in mathematics performance varies
Variance for OECD
as a whole = 100
100
80
60
40
20
0
Variation in mathematics performance
100
80
Variation of performance
within schools
60
40
In some countries, schools are
similar and quality is high.
20
0
In some most of the variation
lies between schools.
-20
Variation of performance
between schools
-40
-60
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
Iceland
Finland
Norway
Sweden
Poland
Denmark
Ireland
Canada
Spain
New Zealand
Australia
United States
Mexico
Portugal
Luxembourg
Switzerland
Greece
Slovak Republic
Korea
Netherlands
Austria
Germany
Italy
Belgium
Japan
Czech Republic
15
Hungary
Turkey
-80
Variation in mathematics performance
100
80
Variation of performance
within schools
60
40
In countries with highly stratified schools, much of the
intended variation between schools can be explained by
socio-economic inequalities in learning opportunities.
20
0
-20
Variation of performance
between schools
-40
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
Iceland
Finland
Norway
Sweden
Poland
Denmark
Ireland
Canada
Spain
New Zealand
Australia
United States
Mexico
Portugal
Luxembourg
Switzerland
Greece
Korea
Brazil
Uruguay
Netherlands
Austria
Germany
Italy
Belgium
Japan
Slovak Republic
16
Hungary
Turkey
-80
Czech Republic
Variation explained by socio-economic
level of students and schools.
-60
Comparing different regression slopes
within and between schools.
17
Performance vs socio-economic background
Germany
High
800
Student performance
and SES - overall
Regression based
on all students in
country
Student performance
School performance
and SES
Low
Student performance and
SES - within schools
School: point
proportional
to size
500
Regression
based on
school means
200
-3
Disadvantage
18
Regression based
on students
within schools
-2
-1
0
1
2
PISA Index of social background
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Figure 4.13, pp.199-203.
3
Advantage
High
Performance vs socio-economic background
Japan
800
Student performance
and SES - overall
Student performance
School performance
and SES
Low
Student performance and
SES - within schools
School: point
proportional
to size
500
200
-3
Disadvantage
19
-2
-1
0
1
2
PISA Index of social background
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Figure 4.13, pp.199-203.
3
Advantage
Performance vs socio-economic background
Canada
High
800
Student performance
and SES - overall
Student performance
School performance
and SES
Low
Student performance and
SES - within schools
School: point
proportional
to size
500
200
-3
Disadvantage
20
-2
-1
0
1
2
PISA Index of social background
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Figure 4.13, pp.199-203.
3
Advantage
What might make a difference to national
quality and equity?
21
Will increased expenditure do it?
575
Countries above line do better
550 than could be expected
Korea
given expenditure
525 levels.
Czech Republic
500
Slovak Republic
Hungary
Poland
475
Finland
Netherlands
Japan Canada
Belgium
Switzerland
Australia
Iceland Denmark
France
Austria
Ireland Germany Sweden
Norway
Spain
United States
Portugal
450
Greece
425
Countries
below line do less well
than could be expected
given expenditure levels.
400
Mexico
375
0
22
Italy
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
Cumulative expenditure per student from Grade 1 until age 15 PPP US$
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 2.6, p.358.
Will reducing class size do it?
No evidence to support this idea from international
comparisons because the high-performing
East Asian countries have relatively large classes.
23
Will devolving budget responsibility do it?
Hong Kong-China
Liechtenstein
Netherlands
Performance
Switzerland
Australia
New Zealand
Czech Republic
Denmark
France
Sweden
Austria
Ireland
Germany
Slovak Republic
Norway
Luxembourg
Poland
Not
devolved
United States
Portugal
Devolved
540
Japan
Canada
Belgium
Hungary
Finland
Korea
Spain
Latvia
Macao-China
520
Iceland
500
480
Russian Federation
Italy
460
24
Equity
Will private schooling do it?
Govt
Govt dependent
Private
0
20
40
60
80
Observed
Adj student SES
Adj student & school SES
100
-100
-50
0
50
100
Luxembourg
Japan
Italy
Switzerland
Finland
Denmark
Czech Republic
Sweden
Hungary
Austria
Portugal
United States
Netherlands
Slovak Republic
Korea
Ireland
Spain
Canada
Mexico
New Zealand
25
Germany
OECD
United Kingdom
Private
schools
perform
better
Public
schools
perform
better
Will instructional organisation do it?

In many of the best performing countries

Schools and teachers have explicit strategies for
teaching heterogeneous groups of learners
– high degree of individualised learning processes
– disparities related to SES and migration accepted as challenges

Schools offer
– differentiated support structures (e.g. school psychologists)
– variety of extra-curricular activities

26
Institutional differentiation is introduced, if at all, at
later stages than age 15
and so on…
27
Changes from PISA 2000 to PISA 2003

Poland




Germany


28
only country to improve on all scales
improved average by improving poorer performers
major reform to make schools heterogeneous and to
remove selective entry
improved on some scales
improved average by improving higher performers
Further information

Websites


OECD education work: www.oecd.org/edu
OECD/PISA: www.pisa.oecd.org
– All national and international publications
– Complete micro-level database

Staff contacts at OECD



29
[email protected]
[email protected]
[email protected]