Transcript PISA2009_KGZ_Results_EnglVersion

PISA 2009.International survey of reading literacy for
15 years old
Several slides were used from PPT
presentation “What Students know
and can do” by Andreas Schleicher.
7 December 2010.
www.oecd.org
In Kyrgyzstan PISA 2009 survey was
undertaken by the CEATM under the
order by Ministry of Education and
Science of the Kyrgyz Republic within
the WB-financed Rural Education
Project
Programme for International Student Assessment
What PISA is?
PISA is an international comparative survey of
functional literacy for 15 years old.
Which is executed by:
• Organization for Economic Co-operation and
Development (OECD)
• Consortium comprising the world’s leading
international testing organizations jointly with the
national testing centers
2
PISA 2009
Partner countries
OECD countries
Austria, Australia, Belgium,
Great Britain, Hungary,
Germany, Greece,
Denmark, Israel, Ireland,
Spain, Island, Italy,
Canada, Luxemburg,
Mexico, Netherlands, New
Zeeland, Norway, Poland,
Portuguese, Slovakia,
Slovenia, USA, Turkey,
Finland, France, Czech
Republic, Chili, Switzerland,
Sweden, Estonia, South
Korea, Japan.
Azerbaijan, Albania,
Argentina, Bulgaria, Brazil,
Shanghai (China), Hong
Kong, Dubai (UAE),
Indonesia, Jordan,
Kazakhstan, Qatar,
Columbia, Kyrgyzstan,
Latvia, Lithuania,
Lichtenstein, Macao
(China), Panama, Peru,
Russian Federation,
Rumania, Serbia,
Singapore, Thailand,
Chinese Taipei, Trinidad
and Tobago, Uruguay,
Croatia, Tunis, Montenegro.
PISA countries
World Economy
1998
2000
2009
2001
2006
2003
85%
87%
86%
83%
81%
77%
4
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World
Background: http://www.oecd.org/dataoecd/60/1/39727764.ppt
What PISA is?
• The assessment takes place in three-yearly
cycles. First survey was undertaken in 1998.
•
-
Study areas:
reading literacy (2000),
mathematics literacy (2003),
science literacy (2006),
reading literacy (2009).
5
PISA goal is…
…to assist various economies in…
…teaching their students to
overcome the rapidly changing
world…
…getting them ready to professions
required by the modern society…
…training them to use modern
technologies…
…giving them problem-solving
skills…
6
PISA goal is…
• To show the countries possible educational
achievements
• To support various governments in drafting
modern education policy goals based
achieved
• To compare the world education progress
and country’s achievements
• To share their practical experience in
education policy area
7
How the demand for skills has changed
Mean task input as percentiles of the 1960 task distribution
Economy-wide measures of routine and non-routine task input (US)
65
Routine manual
60
Nonroutine manual
55
Routine cognitive
50
Nonroutine analytic
45
Nonroutine interactive
40
1960
Andreas Schleicher
What Students know and
can do. 7 December 2010.
www.oecd.org
1970
of 2002
schools:
1980 The dilemma
1990
The skills that are easiest to teach and test
are also the ones that are easiest to digitise,
(Levy and Murnane)
automate and outsource
Countries that joined PISA in 2009:
•
•
•
•
•
•
•
•
Albania
Shanghai (China)
Kazakhstan
Panama
Peru
Trinidad and Tobago
Dubai (UAE)
Singapore
9
Why the survey results are reliable?
Everyone using
survey results
should be
confident that
the data reflect
the actual
situation in
education sector
On this reason
Consortium
controls the
quality of
survey
administration
following the
developed
quality
standards
10
Why the survey results are reliable?
• Sampling (65 countries, about 450 thousands students)
• Keeping cultural balance in test items
• Identical testing tools
• Thorough control over translation
• Administering Field Trial
• Involving 4 independent experts to analyze open questions and
etc.
• Measuring student’s efforts spent to make test
11
What PISA assesses
• PISA assesses functional literacy acquired by
boys and girls and their readiness for
independent life in society.
12
Functional literacy
• Is the scope of knowledge and skills that enable a
human being to act successfully and adopt to
modern conditions, and his capability to a lifelong
learning
• Modern definition of functional literacy is as follows
- knowledge and skills necessary for a human
being to become self realized in the modern
society.
13
What’s new about PISA 2009?
• Electronic Reading Assessment of e-texts
reading literacy (20 countries only)
• More detailed assessment of reading
literacy for students with different readiness
level (7 levels instead of 5)
• Special focus to education progress
(comparing 2000 and 2009 results)
• New personnel student information (attitude
towards reading, usage of library resources
and notions about learning strategies)
14
PISA 2009
• PISA 2009 paid special attention to reading
literacy
• More than two thirds of the testing time was
dedicated to it
15
PISA 2009 in Kyrgyzstan
Карта RDesign©.
http://www.GoKyrgyz
stan.net
16
PISA 2009 in Kyrgyzstan
• 4986 15-years old from 173 secondary education
institutes:
• 116 rural education institutes;
• 30 educational institutes from oblast centers and small
cities;
• 28 education institutes from Bishkek city.
• All country’s regions were included into the sampling
• Testing languages: Kyrgyz, Russian, Uzbek
17
PISA 2009 in Kyrgyzstan
• The 2006 was the first time when the Kyrgyz
Republic took part in the survey
• The Kyrgyz Republic was one of the first Central
Asian countries to take part in the survey
18
PISA 2009 in Kyrgyzstan
• In Kyrgyzstan the survey was conducted by the
Center for Education Assessment and Teaching
Methods (CEATM)
• The work was undertaken under the order by Ministry
of Education and Science of the Kyrgyz Republic
within the WB-financed Rural Education Project
19
What Kyrgyzstan needs this for?
• The goal is to assess literacy level of
Kyrgyzstan students and, therefore,
effectiveness of the education system of
Kyrgyzstan with those of other countries’
education systems in both highly
developed country and lowly
progressing ones
20
What Kyrgyzstan needs this for?
• Search for the most successful education
approaches used in the world practice
• Track changes in literacy level of the Kyrgyzstan
students
• Assess factors effecting learning outcomes and
effectiveness
• Elaborate recommendations and strategies to reform
21
Kyrgyzstan’s education system
PISA assessment tools
Testing
• Multiple-choice questions
• Open-constructed response
• Short response
• Closed constructed response
• The text lengthened 2 hours
22
PISA assessment tools
Questioning
• Questions to student
about learning process,
family, personnel interests
and etc.
• Questions to school
administrations where
PISA survey was
administered
23
Reporting results from PISA 2009
24
«Мыслитель» (фр. Le Penseur) — одна из самых известных скульптур Огюста Родена, созданная в период от 1880 до 1882.
Image from: http://www.museum.ru/alb/image.asp?26801
Kyrgyzstan in PISA 2009
Just like in 2006, the average score
demonstrated by Kyrgyzstan is the lowest
among all survey targeted countries in all
three subject areas:
• reading literacy;
• mathematic literacy;
• scientific literacy.
25
PISA standards
• The minimum international standard is
considered to be level 2 out of 6 levels available
in science literacy and mathematics literacy
scale, while reading literacy scale anticipates 7
maximum levels of performance.
• It considers that students achieving level 2 starts
to successfully demonstrate skills which let them
effectively act in any vitally important situations.
26
In Kyrgyzstan the following percentage of
students DID NOT achieve the minimum
international standard:
• 83,2% of students in reading literacy;
• 86,6% of students in mathematic literacy;
• 82% of students in scientific literacy.
27
What the difference in student scores
means?
• 73 points correspond to 1 literacy level according to the
PISA literacy scale (except for levels 5 and 6)
• Based on PISA’s definition, a 39 point disparity means 1
academic year in the PISA literacy scale.
• Average scores of 90% of countries varies between
levels 2 and 3 .
28
How valuable is the difference is
students’ scoring
• For example, difference between Kyrgyzstan and the
highest ranked participant (Shanghai-China) in
reading literacy area makes 242 points
• In Kyrgyzstan the scores vary from 155 (the weakest
performers’ group) and the strongest performers
proficient at 483 which means that the difference
reports 328 points
29
A Profile of Student Performance
in Reading
30
PISA 2009 defines reading literacy as
" understanding, using, reflecting on and
engaging with written texts, in order to
achieve one's goals, to develop one's
knowledge and potential, and to participate
in society."
31
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in reading literacy.
Percent of students per each scale level of Reading Literacy.
32
Continue on the next slide
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in reading literacy.
Percent of students per each scale level of Reading Literacy.
33
The PISA reading literacy assessment is built
on three major characteristics :
• Texts
• Aspects
• Situations
These three characteristics were used to build
reading scale and subscale
34
Basic characteristics of text structure for the reading
literacy
Texts
TEXTS What kind of
text must students
read?
In what form does
the text appear?
Electronic
Paper
Text format
Text type
Continuous texts
Non-continuous texts
Mixed texts
Multiple texts
Descriptive
Narration
Exposition
Argumentation
35
Basic characteristics of text structure for the reading
literacy
ASPECTS
Aspects
What is the reader’s purpose
and approach to the text?
Access and
retrieve information
Integrate and
interpret
Reflect and
evaluate
36
Basic characteristics of text structure for the reading
literacy
SITUATION
SITUATION:
What is the intended
use of the text,
from the author’s
point of view?
Personal
Public
Occupational
Educational
37
38
How proficient are students in reading?
Shanghai-China
Korea
Finland
Hong Kong-China
Canada
Singapore
Estonia
Japan
Australia
Netherlands
New Zealand
Macao-China
Norway
Poland
Denmark
Chinese Taipei
Liechtenstein
Switzerland
Iceland
Ireland
Sweden
Hungary
Latvia
United States
Portugal
Belgium
United Kingdom
Germany
Spain
France
Italy
Slovenia
Greece
Slovak Republic
Croatia
Czech Republic
Lithuania
Turkey
Luxembourg
Israel
Russian Federation
Austria
Chile
Dubai (UAE)
Serbia
Mexico
Romania
Bulgaria
Uruguay
Thailand
Trinidad and Tobago
Colombia
Jordan
Montenegro
Brazil
Tunisia
Argentina
Indonesia
Albania
Kazakhstan
Qatar
Peru
Panama
Azerbaijan
Kyrgyzstan
Students at Level 5 can handle
texts
are3 unfamiliar
in either
or and in some cases
Tasks
atthat
Level
require students
to form
locate,
%
content. They can find information
in such
demonstrate
detailed
recognise
the texts,
relationship
between,
several pieces of information that
understanding, and infer which
information
is relevant
to theInterpretative
task. They are
must
meet multiple
conditions.
tasks at this level require 100
also able to critically evaluatethe
such
textstoand
build hypotheses
about
reader
integrate
several parts
of a them,
text in order to identify a main
drawing on specialised knowledge
and accommodating
concepts
that may
idea, understand
a relationship
or construe
the meaning of a word or 80
be contrary to expectations. phrase. Reflective tasks at this level may require connections,
comparisons, and explanations, or they may require the reader to
60
evaluate a feature of the text.
Tasks at Level 6 require students to make multiple inferences,
40
comparisons and contrasts that are both detailed and precise. They
Tasksrequire
at Level
4 that involveofretrieving
require the reader
demonstration
a full andinformation
detailed understanding
of one or more
20
to locate
and
organise
several
pieces
of
embedded
information.
Some
texts and may involve integrating information from more than
one text.
tasksTasks
at thismay
levelrequire
require
interpreting
the meaning
of nuances
the
reader to deal
with unfamiliar
ideas,ofin the presence
0
language
in a section
of text by
taking intoand
account
the textabstract
as a whole.
of prominent
competing
information,
to generate
categories
Some taskstasks
at Level
2 require
students to
locate
one or more pieces of
Otherfor
interpretative
require
understanding
and
applying
interpretations.
Reflect
and
evaluate tasks
may
require the reader to
information,
which context.
may need
to be inferred
and
may need to meet
20
categories
in an unfamiliar
Reflective
at this
hypothesise
about or critically
evaluate
a tasks
complex
textlevel
on an unfamiliar
severalto
conditions.
require
recognising
the main idea in a text,
require
readers
useaccount
formalOthers
or
public
knowledge
to hypothesise
topic,
taking into
multiple
criteria
or perspectives,
and applying
understanding
relationships,
or
construing
meaning
within
a limited part
Tasks
atorLevel
1aTasks
require
students
to
locate
one
or
aboutsophisticated
critically
evaluate
a
text.
Readers
must
demonstrate
an
40
at Level 1b require
students
locate a
understandings
from beyond
theto
text.
of
the
text
when
the
information
is
not
prominent
and
the
reader
must
more
independent
pieces
of long
explicitly
stated
accurate
understanding
of
or complex
textsinformation
whose content
single
piece
of explicitly
stated
in a or
make
low
level
inferences.
information;
to unfamiliar.
recognise
theposition
main theme
or syntactically simple
form may be
60
prominent
in a short,
author’s purpose text
in a with
text about
a
familiar
topic,
or
a familiar context and text type, such as a
to make a simple narrative
connection
information
or between
a simple list.
There is minimal
80
in the text and common,
everyday
knowledge
competing information.
100
Andreas Schleicher
What Students know and
can do. 7 December 2010.
www.oecd.org
Kyrgyzstan. Results in reading
literacy
• 16,8% of Kyrgyzstan students
aged 15 can perform reading
literacy tasks at the second level
of complexity and above.
• Only few students can
demonstrate high performance
results (0,1%) (levels 5-6)
• The Kyrgyzstan’s mean score in
reading literacy is 314 points or
level 1b
40
Kyrgyzstan:
Percent of students who answered right: 40.0
Percent of students who MISSED the item: 18.9
Kyrgyzstan:
Percent of students who answered right: 22.1
Percent of students who MISSED the item: 35.0
Levels of Reading Literacy
OECD Kyrgyzstan
Level 6
Level 5
Level 4
Level 3
Level 2
Level 1а
Level 1b
Below level
1b
0.8%
6.8%
20.7%
Russia Azerbaijan Kazakhstan
0 %
0.3%
0 %
0 %
0.1 %
2.8 %
0 %
0.4 %
1.0 %
28.9%
4.2 %
24.0%
11.5%
11.1 %
26.8 %
31.6 %
0.5 %
5.3 %
21.5%
3.7 %
13.1 %
24.1 %
Минимальный стандарт
13,1%
4.6%
1.1%
23.8 %
29.7%
29.8%
19 %
6.8 %
1.6 %
36.9%
26.1 %
9.7 %
30.7 %
20.4 %
7.5 %
43
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1a
Kyrgyz student performance in various reading
areas
Continuous texts
-21
Non-continuous texts
+5
Reflect and evaluate
-14
Access and retrieve information
-15
Integration and interpretation
+13
-25
-20
-15
-10
-5
0
5
10
15
Kyrgyzstan’s mean score in reading literacy scale: 314
44
Kyrgyz student performance in various
reading areas
• It is much harder for Kyrgyzstan students to
execute the tasks related to info access and
retrieving as well as thinking and evaluation.
• Tasks on Integration and interpretation were
executed a bit better
• But nevertheless Kyrgyzstan held the very last
position in the ranking list amongst 65
participants with regard to no single reading
aspect
45
Gender gap between boys and girls in
reading literacy
• Girls demonstrate much higher reading literacy results then
boys do in any countries.
• But the gap in some countries is much stronger then in others
• In Finland the greatest difference between scores by boys
and girls makes 55 points while in Columbia it equates to only
9 points.
In Kyrgyzstan:
• Score gap between boys and girls makes 53 points
• 19,1% of girls and 41,1% of boys demonstrate low
performance results below level 1b
• 21,8% of girls and 11,5% of boys are scored at level 2 and
above.
46
Kyrgyzstan. % of boys and girls per each reading literacy
level
45
41.1
40
35
29.2
30
30.1
29
25
20
19.1
boys
girls
18.2
14.7
15
8.2
10
5
5.5
2.7
0.61.4
0 0.2
0 0
0
below
Level 1
Level
1b
Level
1а
Level
2
Level
3
Level
4
Level
5
Level
6
47
Science literacy
48
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in science literacy.
Percent of students per each scale level of Science Literacy.
49
Continue on the next slide
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in science literacy.
Percent of students per each scale level of Science Literacy.
50
Science literacy
• About 18% of Kyrgyzstan students aged
15 can perform the science literacy
tasks (level 2 and above)
• About 82% of students did not achieve
the minimum international standard
•
Only a few can demonstrate high
performance results (level 5,6)
• Kyrgyzstan’s mean score in science
literacy is 330 points (below level 1)
51
52
Percent of Kyrgyzstan students that above the difficulty of item : 12.9
OECD and CIS countries students who achieved
minimum standard and above in science literacy
90.0
82.1
82.0
78.0
80.0
70.0
70.0
55.4
60.0
50.0
44.6
40.0
30.0
30.0
20.0
22.0
18.0
17.9
10.0
0.0
OECD
Russia
Below minimum standard
Kazakhstan
Azerbaijan
Kyrgyzstan
Minimum standard and above
53
Gender gap between boys and girls in science
literacy
• On the whole, gender gap between boys and girls is
insufficient in science literacy.
• In Jordan girls are greater than boys for 35 points
• In Columbia boys are greater than girls for 21 points
•
• In Kyrgyzstan the mean score of girls are greater than boys for
22 points
54
Kyrgyzstan. % of boys and girls per each science
literacy level
70.0
60.0
50.0
%
юноши
boys
57.9
48.2
girls
%
девушки
40.0
31.6
30.0
26.4
20.0
15.2
11.2
10.0
3.8 4.2
0.6 0.8
0.0 0.0
0.0 0.0
уровень 4
уровень 5
уровень 6
0.0
ниже уровня 1
уровень 1а
уровень 2
уровень 3
55
Mathematics literacy
56
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in mathematics literacy.
Percent of students per each scale level of Mathematics Literacy.
57
Continue on the next slide
PISA. International comparative survey of reading literacy for 15 years old.
Main Survey. May-April 2009.
Results by participant countries in mathematics literacy.
Percent of students per each scale level of Mathematics Literacy.
58
Mathematic literacy
• About 13,4% of Kyrgyzstan
students aged 15 can perform the
mathematic literacy tasks (level 2
and above)
• About 86,6% of students did not
achieve the minimum international
standard
•
Only a few can demonstrate high
performance results (level 5,6)
• Kyrgyzstan’s mean score in
mathematics literacy is 331 points
(below level 1)
59
60
Percent of Kyrgyzstan students that above the difficulty of item : 1.6
OECD and CIS countries students who achieved
minimum standard and above in mathematics
literacy
100.0
86.6
90.0
78.2
80.0
71.4
70.0
59.1
54.7
60.0
45.3
50.0
40.9
40.0
28.6
30.0
21.8
20.0
13.4
10.0
0.0
OECD
Russia
Below minimum standard
Azerbaijan
Kazakhstan
Kyrgyzstan
Minimum standard and above
61
Gender gap between boys and girls in
mathematics literacy
• On the whole, boys in any OECD country demonstrate
higher math scores then girls. The gap makes 12 points.
•
In Columbia and Lichtenstein gender gap between boys
and girls reported 32 and 24 points accordingly.
• In Qatar, Lithuania Albania, Trinidad and Tobago girls
are greater than boys in math literacy from 5 to 11
points.
• In Kyrgyzstan girls also are greater than boys for 6
points
62
Kyrgyzstan. % of boys and girls per each
mathematics literacy level
70.0
65.7
64.1
60.0
%boys
юноши
50.0
40.0
%girls
девушки
30.0
20.7
22.7
20.0
9.3 9.4
10.0
3.4 3.1
0.8 0.7
0.1 0.0
0.0 0.0
уровень 4
уровень 5
уровень 6
0.0
ниже уровня 1
уровень 1а
уровень 2
уровень 3
63
Factors effecting student
performance
What explains gaps and dispersions
in student performance?
64
You results could have been a bit higher in case you…
(Some factors that effect student performance)
• were a girl,
• had better socio-economic background,
• visited private school and/or
• visited the school with students with more
encouraging socio-economic background,
• were enrolled into a 1-year pre-school
preparation program,
• were possessed of stronger and effective
learning strategy,
• had positive attitude to reading,
• willingly read various materials for at least
half an hour a day,
• visited a school with Russian language of
instruction in Bishkek
Total: 484
+53 scores
+40 scores
+103 scores
+62 scores
+47 scores
+87 scores
+24 scores
+15 scores
+53 scores
… Your literacy score could be higher then the mean
score in top performing countries (approximately
484 scores) and higher then the scores of:
• boys,
• students with bad socio-economic background
• students with no prior experience with pre-school
preparation class
• students from schools with no effective learning strategy
• those who have no positive attitude to reading
• students who visit a far distance rural state-run school.
66
Factors explaining low performance of Kyrgyz
students
• Low GDP level
• Poor costs allocated per one student
• Relationship between school performance and schools’
socio-economic background
• Lack of teaching staff
• Lack of teaching and learning resources
• Few students with pre-primary preparation
• Students are not aware of effective learning strategies
• Access to education materials at home
• and etc.
67
GDP Level
• Students representing countries with higher
GDP are used to having better literacy results
• GDP level explains a 6% performance gap
between students from different countries
• Kyrgyzstan has the lowest real gross domestic
product among all PISA countries – USD 1994
(as per 2007).
68
GDP level
• But it does not mean that is impossible to reach high
performance with a low GDP
• For example, in Shanghai-China GDP makes only USD
5430, which is much lower then relevant results in most
PISA 2009 countries
• Nevertheless, Shanghai-China is ranked first country in
all three surveyed subject areas, this is, reading,
mathematics and science literacy
69
Per capita education costs for 6-15 years
old
• There is a positive correlation between a per capita
expenses and demonstrated literacy results
• As a rule, the greater are per capita student
expenses, the bigger are the average literacy
results
• Per capita student expenses explains a 9%
performance gap between students from different
countries
70
Per capita education costs for 6-15 years
old
• In Kyrgyzstan capitation expenses per one
students aged 6-15 make USD 3000
• This is the least amount among all surveyed
countries that provided the data (41 countries)
71
Per capita education costs for 6-15 years
old
• In other countries similar capitation expenses vary
from USD 106 000 in USA to USD 12 700 in Turkey.
• In Russian Federation capitation expenses make
USD 17 500. No other CIS country provided required
data.
72
Per capita education costs for 6-15 years
old
• But the country should not necessarily have the
lowest literacy results if its expenses are not the
greatest ones
• For example, Estonia and Poland where capitation
expenses reach USD 40 thousands demonstrated
great literacy results almost similar to those by
USA and Norway who spent USD 100 thousands
per each student
73
Socio, economic and cultural student
indices
• Social, economic and cultural aspects are the strongest that
effect student performance results
Components of social, economic and cultural indices:
• Occupational status of parents, educational level of parents
with regard to schooling period;
• Home educational resources relevant for culture
• Items measuring the existence of educational resources at
home including a desk and a quiet place to study
• Computer that students can use for schoolwork, educational
software, books to help with students’ school work, technical
reference books and a dictionary
• Dishwasher, tape-recorder, DVD player, cell phone (s), TV
74
set (s), computer(s), car (s), etc.
Socio, economic and cultural
student indices
• In all countries, children from families with better socio and
economic background usually receive high PISA scores
• In Kyrgyzstan, socio-economic gap explains almost a 14,5%
differences in student scores
•
In case socio-economic status of students increases for 1
index, one can forecast a 40 point increase in student
performance
75
Socio, economic factors at
country level
• Countries, where students have higher socio-economic
status, usually demonstrate higher reading literacy
results
• But in Kyrgyzstan and Shanghai with almost similar
socio-economic status of students, Shanghai showed the
highest scores, while Kyrgyzstan was the last one
76
Socio, economic factors at
country level
77
Family structure
• In the absolute majority of families students from twoparent families usually demonstrate higher literacy
results then the ones from incomplete families. In
Estonia, Kazakhstan, Peru, Croatia children from
incomplete families show higher literacy results
• In Kyrgyzstan the mean literacy score by children from
incomplete families is 9 points higher then the one by
students from two-parent families
78
79
Impact of school intake index over literacy
results
• Direct relationship between socio-economic status of
most students and literacy level
• In case socio-economic status of school increases for
1 index, one can forecast a 62 average point increase
in student performance. This means that schools
where enrolled students have better socio-economic
background show better higher performance results
• As majority of Kyrgyz students have poor socioeconomic status, enrollment of selected welfare
students increases and strengthens education inequity
and worsens the overall country’s literacy level
80
Pre-primary school preparation
• Students who have a prior experience of
visiting a pre-school preparation class for
more then one year usually show higher
performance results at the age 15 then those
with no prior experience
81
Pre-primary school preparation
• In OECD countries, the average difference between
students who have a prior experience of visiting a preschool preparation class for more then one year usually
show higher performance results (54 points) then those
with no prior experience
• In Kyrgyzstan the difference between these two groups
makes 72 points
82
Pre-primary school preparation
67%
80.0%
72%
70.0%
60.0%
50.0%
40.0%
30.0%
19.5%
19.5%
17.8%
11.0%
20.0%
10.0%
0.0%
More
then
one year
более
года
Кыргызстан
Kyrgyzstan
A год
yearили
and меньше
least
Россия
Russia
ОЭСР
OECD
83
Attitude to reading
• All countries where students noted their personnel desire in reading
showed much higher literacy results then in those countries where
students do not enjoy reading
• IN OECD countries possible changes in scores that might have occurred
due to “enjoying reading” index gives 39 additional points or 1 year of
studies. Moreover the gap between students who enjoy reading and
those who do not reaches almost 1,5 of studies on reading literacy scale.
• In Kyrgyzstan and Azerbaijan possible changes in scores that might
have occurred due to “enjoying reading” index gives 24-23 additional
points accordingly. In Russian this index reaches 49 points, while in
Kazakhstan it has no statistical significance.
• The total score difference between students with personnel desire in
reading and those who do not enjoy reading have the following values:
88 points in Russia; 43 points in Kyrgyzstan; 29 points in Azerbaijan and
10 points in Kazakhstan which the smallest difference
84
Attitude to reading
• On the average, much more students in Kyrgyzstan,
Kazakhstan and Azerbaijan noted that they read for
pleasure then those in OECD countries
• For example, in Russia and OECD countries 21,4% and
37,4% accordingly noted that they never read for pleasure,
while only 11,7% of students in Kyrgyzstan and 7,2% in
Kazakhstan gave similar answers
• Also only 4,5% of students in OECD countries and 6,9% of
students in Russia answered that they read for more then 2
hours a day, while in Kyrgyzstan 14,5% of students noted
that they for more then 2 hours a day and this is the highest
value among all PISA countries except for Azerbaijan where
almost 15,2% answered in this way
85
Attitude to reading
• In 36 countries, students who enjoy reading for more then 30
minutes a day received 30 additional points more then students who
do not enjoy reading at all
• Nevertheless in Kyrgyzstan the gap between students who enjoy
reading for more then 30 minutes and those who do not enjoy
reading at all makes only 15 points
• Thus, in Kyrgyzstan dependence between the time that students
spent for reading for more then 30 minutes a day and those who
never read for pleasure, is much poor as compared with other PISA
countries
86
Attitude to reading
• As “reading for pleasure” index is tightly connected with
literacy scores, students’ interest to reading in the school
and beyond should be stimulated
• It is important to encourage a child to reading every day
for pleasure
• Everyday reading during 30 minutes is enough to give a
significant impact over literacy scores
87
Attitude to reading
• Students who read a scope of
various materials also have higher
reading literacy results
• Reading of books, novels,
magazines and journals influences
the score
• In Kyrgyzstan students who read
magazines and journals regularly
received 26 additional points in
reading literacy scale
88
Knowledge of learning strategies
• Students that apply strategies to understand
and remember the information, as well as
summarise information strategies received
mush higher scores then those that know no
strategies and do not apply them in practice
89
Strategies to understand and
remember
Most effective:
• After I read the text I discuss it with other people
• I underline most important text parts
• I generalize the text in my own words
Less effective:
• I focus on text parts that are easier to understand
• I briefly look through the text two times
• I read the text aloud to some other person
90
Strategies to summarise information
Most effective
• I thoroughly check if the key text elements are
represented in the text summary
• I briefly look through the text underlining key elements
and rewrite them with my own world as a summary
Less effective:
• I write a summary of each paragraph and check is all
paragraph is included into the summary as it is critical to
insert content of each paragraph
• Prior to drafting a summary I reread the text as many
times as possible
Least effective:
• I carefully rewrite as many sentences as possible
91
Knowledge of learning strategies
• OECD students that apply most effective strategies
for remembering and understanding the facts
received 70 additional points in literacy scale as
compared with students who do not apply this
strategies in practice. This score difference
correspond to 1 level in reading literacy scale
• In Kyrgyzstan and the Russian Federation this gap
makes 87 points and 91 points in Kazakhstan
92
Knowledge of learning strategies
• Kyrgyz students are less aware of learning strategies then
the others
• In Kyrgyzstan and Azerbaijan index for remembering and
understanding is the least among all PISA countries and
makes -0,69. In the Russian Federation and Kazakhstan
reported -0,11 and -0,33 accordingly
• In Shanghai which showed the highest reading literacy
results, remembering and understanding index reached
0,14
93
Knowledge of learning strategies
% students who use effective learning strategies
Kazakhstan
Казахстан
46
Azerbaijan
Азербайджан
54
42
Russia
Россия
58
57
Kyrgyzstan
Кыргызстан
37
OECD
ОЭСР
20
Superficial
поверхностные
63
73
0
Thoughtful
вдумчивые
43
40
27
60
80
100
Thoughtful students are those who use effective learning strategies
Superficial students are those who don’t know what learning strategies are most effective
94
Kyrgyzstan. Impact of student questioners’ indices over reading literacy
expressed in terms of regression numeric parameter
95
School administration
characteristics
96
Selection and breakdown of
students
• Relatively high rate of grade repetition
• Distribution of students by different education programmes up to age 15
• Transferring of students to another schools based on low performance
results, behavior problems and to meet their specific education needs
• Practice of students’ distribution by their abilities by all subjects…
…. not only effect negatively on the average level of
literacy but strengthens the impact of the family’s social
and economic factor on literacy performance.
97
School autonomy in the area of
curriculum development and
assessment
• Higher literacy level is commonly observed in those
education systems where schools are granted a
significant autonomy in drafting curriculums and
executing assessment. Impact of social and economic
factors in these systems is eased
• It becomes true just in case national standards are
available
98
School autonomy in the area of
curriculum development and
assessment
• All CIS countries that took part in PISA assessment
demonstrated weaker level of autonomy in learning process
planning then the others OECD countries.
• Most dependent in the area of school learning planning is
Kazakhstan with the index -0,98 and the most independent
is Kyrgyzstan with the index -0,25
• Index volume in Russia and Azerbaijan makes -0,36 and
-0,64 accordingly
99
Assessment
• Even if take into account the per capita income, those
countries where national examinations are as a rule
usually have higher literacy results
• Students received 19 additional points to their literacy
scores in schools using mandatory national exams then
those from schools that do not apply national exam
system
• In 18 participant countries where school administration
post student performance results in public access
usually have higher literacy scores that school with
restricted access to performance-related information.
• After we take into account all socio-economic factors, we
will have only 5 countries left. But Kyrgyzstan is still
among them
100
Teaching staff
• Teaching salaries directly effect the overall literacy
level in the country
• In case countries have equal teachers’ remunerations,
below average results will be achieved by the school
that invests into the teachers’ salaries, not fullness of
classes
101
Teaching staff
• When comparing answer sheets by school
administrators from CIS countries it is obvious that in
Kyrgyzstan teachers’ shortage is the most critical
among all and reports 0,90
• In Russia corresponding figure makes 0,13, in
Kazakhstan – 0,47. In Azerbaijan shortage of teaching
staff is close to an average among OECD countries
and makes -0,02
102
Material resources
• No correlation is observed between availability of extra
educational resources and literacy scores
• But it does not mean that these resources have no impact on
the literacy level
• It might be caused by the reason that there is no great
variety in equipping all OECD schools
•
We are confident that schools lacking adequate teaching
staff, shortage of textbooks and other infrastructure problems
are more likely to get the lowest literacy results
103
Material resources
• As of 2006, material resources shortage index was the
highest among all participating countries and reported -1,72
• In other CIS countries is also a bit material resources
shortage index then the OECD countries’ average
• In Kazakhstan index valued -0,76, in Russia and Azerbaijan
it made -0,63 and -0,58 accordingly
• In Kyrgyzstan the material resources shortage index is
relatively high throughout the country. CIS countries have
single schools equipped much better then those from OECD
countries (material resources index is almost one level
higher then in standard OECD country )
104
Material resources
• It is meant that school performance does not depend on
material resources provision, while in Kyrgyzstan impact of
material resources is much higher then the one by school
administration characteristics
• Before we take into account socio-economic factors, literacy
score will mostly depend on educational resources, school
administration, assessment system and student enrollment
and distribution system last of all.
105
Values of factors effecting Kyrgyz
performance
Educational
resources
(teaching
staff,и
образовательные
ресурсы
(педагогические
кадры
материальные
ресурсы вresources
школе)
school material
etc.)
7.1
Assessment
system
(national exams
and
система
оценивания
(национальные
экзамены
и
1.2
опубликование
показателей
успеваемости)
publication
of performance
results)
18.4
4.3
Administration (autonomy in drafting
школьное управление (автономия в области учебного
curriculum) 1.2
плана)
Enrollment
and distribution
students
(grade
набор
и распределение
учащихся of
(перевод
из школы
в
школу,
оставление
"на второй
год" и др.)
repetition,
transferring
to another
school)
1
0
8.7
4.1
5
10
15
20
25
30
regardless of
factor
независимо
отsocio-economic
социально-экономических
факторов
with regard
to socio-economic factor факторами
вместе
с социально-экономическими
% deviation percentage that could be explained due to this factors
106
Kyrgyzstan. Comparing PISA 2006
and 2009 results
• An absolutely similar assessment approach in PISA
is used once in 9 years only
• That is why the only true comparison of PISA results
could be made in the years 2015, when PISA comes
back to assess science literacy
107
Kyrgyzstan. Comparing PISA 2006
and 2009 results
• Regardless to the fact that just like in 2006
Kyrgyzstan took the last place among all ranked
counties, one could observe slight positive changes
as compared with 2006
• Both mean values and the number of students
achieving the minimum literacy standard has
increased
108
Kyrgyzstan. Comparing PISA 2006
and 2009 mean scores
340
330
330
330
322
320
313
311
310
300
290
285
280
270
260
Reading literacy
грамотность
чтения
2006
Mathematicsграмотность
literacy
математическая
Science literacy
естественнонаучная
грамотность
2009
109
Kyrgyzstan. Comparing PISA 2006
and 2009
Percentage
ofучащихся,
students которые
who achieved
theминимального
minimum literacy
standard
in reading
Процент
достигают
стандарта
в чтении
18,0%
16,0%
14,0%
12,0%
10,0%
16,6%
8,0%
6,0%
11,8%
4,0%
2,0%
0,0%
2006
2009
110
Kyrgyzstan. Comparing PISA 2006
and 2009
Percentage
of учащихся,
students who
achieved
theминимального
minimum literacy
standard
in mathematics
Процент
которые
достигают
стандарта
в математике
14,0%
12,0%
10,0%
8,0%
13,1%
6,0%
10,6%
4,0%
2,0%
0,0%
2006
2009
111
Kyrgyzstan. Comparing PISA 2006
and 2009
Процент
учащихся,
которыеwho
достигают
минимального
стандарта
в естественных
Percentage
of students
achieved
the minimum
literacy
standard inнауках
science
20,0%
18,0%
16,0%
14,0%
12,0%
10,0%
8,0%
6,0%
18,3%
13,6%
4,0%
2,0%
0,0%
2006
2009
112
Thank you for your attention!
Sources:
1. OECD (2010), PISA 2009 Results: What Students Know and Can Do – Student Performance in Reading,
Mathematics and Science (Volume I). http://dx.doi.org/10.1787/9789264091450-en
2. OECD (2010), PISA 2009 Results: Overcoming Social Background – Equity in Learning Opportunities and
Outcomes (Volume II). http://dx.doi.org/10.1787/9789264091504-en
3. OECD (2010), PISA 2009 Results: Learning to Learn – Student Engagement, Strategies and Practices
(Volume III. )http://dx.doi.org/10.1787/9789264083943-en
4. OECD (2010), PISA 2009 Results: What Makes a School Successful? – Resources, Policies and Practices
(Volume IV). http://dx.doi.org/10.1787/9789264091559-en
5. PISA Data Analysis Manual: SPSS® SECOND EDITION. www.sourceoecd.org/education/9789264056268
6. OECD Database. http://pisa2009.acer.edu.au/
7. PISA 2009. Evaluating systems to improve education. Presentation. Andreas Schleicher. Special advisor to
the Secretary-General on Education Policy. Head of the Indicators and Analysis Division, EDU
Some photos used in presentation were extracted from Microsoft Office Clipart online and should not be associated with PISA survey
113
www.testing.kg