Transcript No Slide Title

1
OECD Programme for International Student Assessment (PISA)
Improving both quality and equity
Insights from PISA
Hong Kong, 21 November 2003
Andreas Schleicher
Head, Indicators and Analysis Division
OECD
2
Improving both quality and equity
1. Why we need to worry

The significance of educational
outcomes for individuals and societies
2. Where we are today

What PISA shows about student performance
and the social distribution of learning
opportunities in OECD countries
3. Where we can be

What the best performing countries show can
be achieved
4. How we can get there

Levers for policy that emerge from
international comparisons
3
The significance of educational
outcomes.
Changing contexts for education systems.
4
Changing contexts for education
 Knowledge workers
the only fast growing share of the workforce

By 2002 - about the time when school
reforms put in place today will begin to
show effects in labour markets...
...Manufacturing output in OECD area is likely to double…
…Manufacturing employment is likely to shrink
to 10% of the workforce

Knowledge as the key
economic and social resource
– and the only scarce one
Traditional jobs are changing too…
5
Literacy skills in the manufacturing sector
Pages
16000
13866
14000
Total number of pages of
repair manuals for Opel cars
12000
10000
from 1933 to 1998
8000
8012
6000
4526
1933
1951
1966
1983
Omega B
Omega A
Record E
1727
Record D
Record C
Record B
Record A
Record P2
Record P1
Olympia 53
Olympia 50
Olympia 47
Olympia 1.5
Year
Olympia 1.3
0
990 1175 1253 1213
710
663
202 130 170 210 255 470
1.3l
2000
1.2l
Source : Georg Spöttl, 2002
4000
1998
Unique opportunities
6

Unlimited upward mobility…


Knowledge changes rapidly
Everybody starts from ignorance
– Knowledge differs from traditional means of
production in that it cannot be inherited or
bequeathed

Knowledge is “public”
– Knowledge has to be put in a form in which it can be
taught and is therefore universally accessible

Every impediment to mobility…
…is perceived as a form of discrimination
… if our education systems deliver on their
promises
Rise in baseline qualifications over one generation
7
Proportion of the population with completed upper secondary education
by age group (37 countries)
100
90
55-64
9
45-54
4
35-44
25-34
1
10
80
70
60
50
31
1
40
11
30
14
24
20
10
37
26
34
Data for Hong Kong
unavailable
Thailand
Indonesia
Mexico
Brazil
Spain
Italy
Peru
Chile
Argentina
Korea
France
OECD average
Finland
United Kingdom
Japan
Canada
Germany
United States
0
8
Where we are today.
What PISA shows about student
performance and the social distribution of
learning opportunities in OECD countries.
PISA - The OECD Programme for
International Student Assessment
9

A regular assessment of the yield of
education (2000, 2003, 2006, 2009,…)


Comparable skill measures


that can guide policy decisions
Insights into the mix of factors which
contribute to the development of knowledge
and skills


including and beyond the curriculum
and how these factors operate similarly or
differently across countries
A strong substantive and cross-cultural
core for defining performance targets
PISA - The OECD Programme for
International Student Assessment
10

The most comprehensive international
assessment to date

Geographic and economic coverage
– 340,000 students randomly sampled
– All 30 OECD countries plus a growing number of nonOECD countries

Subject matter coverage
– Reading, Mathematics, Science
– Cross-curricular competencies


Variety of task formats
Depths
– A total of 7 hours of assessment material
11
Three broad categories of
key competencies
Using “tools”
interactively to
engage with the world
To analyse, compare, contrast, ande.g.
evaluate
Using language, symbols and texts
Toinformation
think imaginatively
Interacting with
Capitalising on the potential
PISA 2000: A new
concept
of literacy
of technologies
Acting
Interacting
in
Accessing,
managing,
integrating
autonomously
diverse groups
and evaluating
written information
e.g.
e.g.
in order
to develop
andwithin
potential,
Acting
the bigger picture
Relating
wellones
to knowledge
others
and to participate in, and contribute to, society
Co-operating, working in Forming
teams and conducting life plans
Taking responsibility and
Managing and
resolving situations
conflicts
To apply knowledge
in real-life
understanding rights and limits
To communicate thoughts and ideas effectively
12
Using “tools”
interactively to
engage with the world
To analyse, compare, contrast, ande.g.
evaluate
Using language, symbols and texts
Toinformation
think imaginatively
Interacting with
Capitalising on the potential
Reading
literacy
of technologies
Acting
Interacting in
Using, diverse
interpreting
autonomously
groups and reflecting
e.g.
on written
material
e.g.
Acting within the bigger picture
Relating well to others
Co-operating, working in Forming
teams and conducting life plans
Taking responsibility and
Managing and
resolving situations
conflicts
To apply knowledge
in real-life
understanding rights and limits
To communicate thoughts and ideas effectively
13
Using “tools”
interactively to
engage with the world
To analyse, compare, contrast, ande.g.
evaluate
Using language, symbols and texts
Toinformation
think imaginatively
Interacting with
Capitalising on the potential
Mathematical
literacy
of technologies
Acting
Interacting
in
Emphasis is on
mathematical
knowledge put into
autonomously
diverse groups
functional use in a multitude
of different e.g.
situations
e.g.
Acting within the
bigger picture
well to
others
in varied,Relating
reflective
and
insight-based
ways
Co-operating, working in Forming
teams and conducting life plans
Taking responsibility and
Managing and
resolving situations
conflicts
To apply knowledge
in real-life
understanding rights and limits
To communicate thoughts and ideas effectively
14
Using “tools”
interactively to
engage with the world
To analyse, compare, contrast, ande.g.
evaluate
Using language, symbols and texts
Toinformation
think imaginatively
Interacting with
Capitalising on the potential
Scientific
literacy
of technologies
Acting
Interacting in
Using scientific knowledge, identifying scientific
autonomously
diverse groups
questions, and drawinge.g.evidence-based conclusions
to
e.g.
Acting the
within
the bigger
picture
understand and
make well
decisions
about
natural
world
Relating
to others
Co-operating, working in Forming
teams and conducting life plans
Taking responsibility and
Managing and
resolving situations
conflicts
To apply knowledge
in real-life
understanding rights and limits
To communicate thoughts and ideas effectively
15
Using “tools”
interactively to
engage with the world
Under development:
Interacting in
PISA assessment
of
diverse groups
– Problem-solving skills
PISA self-reports on:
– Dispositions to learning
– Learning strategies
– Engagement with school
Acting
autonomously
16
Using “tools”
interactively to
engage with the world
Interacting in
diverse groups
Acting
autonomously
Not yet developed
Levels of
reading literacy
– Retrieving information
Reading Literacy Level 5:
17
Hong Kong: 10%
– Locate and sequence/combine multiple pieces
Germany: 9% of deeply embedded information, some of
USA: 12%
Finland: 19% which may be outside text
Level 5
10%
Level 4
22%
Level 3
29%
Reading Literacy Level 1:
Infer which
information on text is relevant
– –Retrieving
information
22%
Level 2
12%
Level 1
OECD
Average
Below Level 1
6
%
to the task
– Locate one or more independent pieces of
– Deal
with highly
plausible
competing
explicitly
stated
information,
information
– typically meeting a single criterion
– Interpreting
texts
– With little
or no competing information in th
– Construe
text the meaning of nuanced language
 Overall,
Hong
Kongunderstanding
has few
Demonstrate
full and
detailed
– –Interpreting
texts
of a text
low-performers
– Recognise
the main theme or author’s purpos
Hong Kong: 7%
– Reflection
andabout
evaluation
Germany:
13% in atext
a familiar
But
the
risk
for topic
boys in Hong
USA: 12%
Critically
evaluate
or hypothesise,
drawing
– –Reflection
andto
evaluation
Kong
perform
poorly
is 3 on
specialised knowledge
– Maketimes
a simpleasconnection
high as between
for girls !
– Deal
with
concepts
that
are
contrary
to
Hong Kong: 3%information in the text and common
– Percentages at Levels 1
expectations
France: 4%
knowledge
and below of long and
Germany:
– 10%
Draw
on1:deep understanding
Below
Level
USA: 6%
complex texts – Boys: 12%
– Many
of these students have technically
learned to read… – Girls: 4%
…but they can not use reading for learning
20%
30%
Level 4
31
32
14
24
27
24
United_States
France
United_Kingdom
16
12
60%
Level 3 70%
80%
Level 2 90%
Level 1100%
Below Level 1
1
6
33
5-7
29
14
5
2
5
30
3-9
17
18
7
8
3
3
28
8
7-9
20
31
27
11-16
10-20
15
22
9
11
4
4
5
9
1
6
19
19
27
30
22
34
4
21
20
33
31
22
40%
50%
Mexico
19
Germany
10%
9
Italy
Level 5
10
Spain
0%
Ireland
Finland
18
Hong Kong
Percentage of students at each of the
proficiency levels in reading literacy
16
21
12
6
19
17-21
26
12
4
21
19-24
26
14
5
21-25
22
28
13
10
16
22
-30
37
16
28
8
35
4
32
7
2
1
10
United States
Russian Federation
United Kingdom
Indonesia
Australia
Thailand
Korea
Canada
Finland
Sweden
9
Brazil
6
Japan
21
Hong Kong-China
Italy
-10
Germany
19
Variation
in reading literacy performance
170
150
130
110
90
70
50
30
10
-100
22
9
37
16
28
8
Brazil
United States
Russian Federation
United Kingdom
4
2
1
Finland
Sweden
7
Canada
32
Korea
Thailand
35
Australia
Indonesia
6
Japan
21
Hong Kong-China
100
Italy
-80
Germany
20
Variation
in reading literacy performance
80
60
40
20
0
-20
-40
-60
10
Variation
in reading literacy performance
21
100
Variation of
performance within
schools
80
60
40
20
0
-20
-40
United States
Russian Federation
United Kingdom
35
4
32
7
2
1
10
Sweden
8
Finland
28
Canada
16
Thailand
37
Australia
9
Indonesia
6
Brazil
21
Japan
22
Italy
-100
Germany
-80
Hong Kong-China
-60
Korea
Variation of
performance between
schools
Social Background and Student Performance
22 High
Student performance in PISA
performance
Low
Social background is a powerful factor
influencing student performance
(Parental occupation, wealth, cultural resources, parental
education, family structure, immigrant status)
But poor performance
does not automatically follow
PISA Index of social background
Social
Advantage
Social Background and Student Performance
23 High
performance
Finland
Japan
Hong Kong
UK
France
US
Italy
Germany
Student performance in PISA
600
550
500
450
400
350
-2.5
Low
-2
-1.5
-1
-0.5
0
0.5
PISA Index of social background
1
1.5
2
Social
Advantage
24
Where we can be.
What the best performing countries show
can be achieved.
High
Performance
550
Finland
25
High performance
Low social
Low
Social equity
High performance
Canada
Australia New Zealand
Ireland
Korea
525
Hong Kong-China
/
equity
Japan United
Kingdom High social equity
Sweden
Austria Belgium/ Iceland
Norway
France
High
United States
500
Denmark
Switzerland
Social equity
Czech Republik Spain
Italy
Germany
Liechtenstein
Hungary
Poland
Greece
475
Portugal
Russian Fed.
Latvia
Israel
450
Luxembourg
Thailand
Bulgaria
425
Mexico
400
Low performance
Low social equity
-25
-20
-15
-10
Argentina
Chile
-5
Brazil
Low performance
375
FYR Macedonia
Indonesia
350
Albania
325
Peru
0
Low
Performance
5
High social equity
10
15
20
25
High
Performance
26
550
High performance
New can
Zealand
 Quality
and
equity
United Kingdom
Australia
525
Low social equity
achieved together
be
Finland
Canada
Ireland
High performance
Hong Kong-China
High social equity
Sweden
Austria
Norway
United States
Low
500
Denmark
Switzerland
Social equity
Spain
Czech Republik
Germany
Italy
Liechtenstein
Poland
475
Greece
Hungary
Portugal
Russian Fed.
Belgium
France
• ‘dumbing down’ is not an inevitable
consequence of the pursuit of
equity
Korea
Japan
Iceland
High
Social equity
Latvia
Israel
450
Luxembourg
• ‘levelling up’ is achievable
(e.g. Finland, Korea, Canada)
Bulgaria
425
Argentina
Chile
400
Low performance
Low social equity
Brazil
Thailand
Mexico
Low performance
FYR High
Macedonia
social
375
equity
Indonesia
350
Peru
-25
-20
-15
-10
Albania
325
-5
0
Low
Performance
5
10
15
20
25
27
Policy levers.
Overall findings
Policy Levers
28

Performance in reading

Students from advantaged backgrounds…
… have a greater chance of coming to school more
engaged in reading and entering into a virtuous circle
of increasing reading interest and improved reading
performance
… but not all engaged students come from
privileged homes…
… and those from more modest backgrounds who read
regularly and feel positive about it are better
readers than students with home advantages and
weaker reading engagement

Schools can make a significant difference to
bring students into the virtuous circle
–
Seeking mutual reinforcement of cognitive skills and
motivation, particularly for boys
Policy Levers
29

Student approaches to learning

The ability to manage one’s learning is both an important
outcome of education and a contributor to student
literacy skills at school
–

Different aspects of students’ learning approaches are
closely related
–

Learning strategies, motivation, self-related beliefs,
preferred learning styles
Well-motivated and self-confident students tend to invest
in effective learning strategies and this contributes to
their literacy skills
Immigrant students tend to be weaker performers
… but they do not have weaker characteristics as learners

Boys and girls each have distinctive strengths and
weaknesses as learners
–
–
Girls stronger in relation to motivation and self-confidence
in reading
Boys believing more than girls in their own efficacy as
learners and in their mathematical abilities
Policy Levers
30

Student engagement at school

An important outcome in itself
–
–

The prevalence of disaffected students varies
significantly across schools in each country
–
–

Only weak link to student’s social background – there is
thus scope for school policy/practice to engage students
But strong link to school’s social background
Students in schools with strong average engagement
tend to perform well
–
–

Disaffection at age 15 can potentially be a precursor to the
onset of more serious problems among vulnerable young
people
Engagement at age 15 is likely to influence students’
choices and educational pathways
Engagement and performance seem to work complementary
The school climate seems to make more of a difference
than resources
For individual students, strong performance does not
necessarily ensure strong engagement at school
–
Relationship complex
31
Policy levers.
Some characteristics shared by
some strongly performing countries
Combining the empirical results obtained through
PISA with qualitative information on the sociocultural conditions and education policy strategies.
32
Sympathy doesn’t raise standards –
aspiration does

In the countries studied


National research teams report a strong
“culture of performance”
– Which drives students, parents, teachers and
the educational administration to high
performance standards
PISA shows that students perceived a high
degree of teacher support
– Which should not be simply equated with
“achievement press”
Governance of the school system
33

In the countries studied…


Decentralised decision-making is combined with
devices to ensure a fair distribution of
substantive educational opportunities
The provision of standards and curricula at
national/subnational levels is combined with
advanced evaluation systems
– That are implemented by professional agencies

Process-oriented assessments and/or
centralised final examinations are complimented
with individual reports and feed-back
mechanisms on student learning progress
High
Performance
550
34
Finland
New Zealand
United Kingdom
Australia
525
Canada
Ireland
Hong Kong-China
Korea
Japan
Sweden
Austria
Norway
United States
Low
500
Denmark
Switzerland
Spain
Social equity
Czech
Republik
Germany
Italy
Liechtenstein
Poland
475
Greece
Hungary
Portugal
Russian Fed.
Belgium
France
Iceland
High
Social equity
Latvia
Israel
450
Luxembourg
Bulgaria
E.g. Learning environment
and course offering
High degree of autonomy
Low degree of autonomy
-15
Mexico
Brazil
FYR Macedonia
375
Indonesia
350
Peru
-20
Argentina
Chile
400
% Variance between schools
-25
Thailand
425
-10
Albania
325
-5
0
Low
Performance
5
10
15
20
25
Organisation of instruction
35

In the countries studied…

Schools and teachers have explicit strategies
and approaches for teaching heterogeneous
groups of learners
– A high degree of individualised learning processes
– Disparities related to socio-economic factors and
migration are recognised as major challenges


Students are offered a variety of extracurricular activities
Schools offer differentiated support structures
for students
– E.g. school psychologists or career counsellors

Institutional differentiation is introduced, if at
all, at later stages
– Integrated approaches also contributed to reducing
the impact of students socio-economic background on
outcomes
High
Performance
Early selection and institutional
stratification
Low degree of stratification
36
550
Finland
New Zealand
United Kingdom
Australia
525
High degree of stratification
Canada
Ireland
Hong Kong-China
Korea
Japan
Sweden
Austria
Norway
United States
Low
500
Denmark
Switzerland
Social equity
Spain
Czech Republik
Germany
Italy
Liechtenstein
Poland
475
Greece
Hungary
Portugal
Russian Fed.
Belgium
France
Iceland
High
Social equity
Latvia
Israel
450
Luxembourg
Bulgaria
Thailand
425
Argentina
Chile
400
Mexico
Brazil
FYR Macedonia
375
Indonesia
350
Peru
-25
-20
-15
-10
Albania
325
-5
0
Low
Performance
5
10
15
20
25
Support systems and professional
teacher development
37

In the countries studied…





Effective support systems are located at
individual school level or in specialised support
institutions
Teacher training schemes are selective
The training of pre-school personnel is closely
integrated with the professional development of
teachers
Continuing professional development is a
constitutive part of the system
Special attention is paid to the professional
development of school management personnel
38
Summary of common characteristics
Uniformity
Diversity
“hit and miss”
Universal high standards
“Inputs”
Outcomes
Bureaucratic
Devolved responsibility
Look up
Look outwards
Received wisdom
Data and best practice
Evaluation to control
Prescription
Motivating feedback and
incentivising success and
innovation
Informed profession
39
One challenge – different approaches
The future of education
systems needs to be
“knowledge rich”
Informed professional
judgement, the teacher as
a “knowledge worker”
Informed
prescription
National
prescription
Professional
judgement
Uninformed
prescription, teachers
implement curricula
Uninformed professional
judgement
The tradition of
education systems has
been “knowledge poor”
Further information
40

www.oecd.org
www.pisa.oecd.org
email: [email protected]

[email protected]


… and remember:
Without data, you are just another
person with an opinion