Transcript PISA for Development Technical Issues draft for ToR to present

PISA for Development
Technical issues to be addressed (for ToRs)
Contextual Questionnaires and data collection instruments
A.
Review of PISA constructs and indices (and variables) to identify those
that are likely to be operable and not operable in developing country
contexts, beginning with participating country contexts
B.
Scale expansion and equating with current PISA indices identified to be
operable and relevant in these contexts
C.
Review of relevant evidence and examples (including input from
participating countries) to identify relevant constructs to be considered
for inclusion , including but not limited to the following:
•
Educational history of student (cognitive, non-cognitive)
•
School resources (including possible scales) that have effects on
learning outcomes
•
Health and well-being of student
•
Socio-economic status
PISA for Development
Technical issues to be addressed (for ToRs)
Contextual Questionnaires and data collection instruments
D.
Review of contextual data collection used in PASEC, SACMEQ, STEP,
LAMP, and participating countries… to identify relevant and viable
constructs (indices and variables) to be considered and possible paths for
incorporation into the field work of PISA for Development.
E.
Review of evidence, methods and examples of collecting contextual
information from out-of-school children (and students) that may not be
able to engage with and respond the paper-and-pencil questionnaires.
F.
Review of availability and quality of data on education systems and data
collection systems in participating countries
PISA for Development
Technical issues to be addressed (for ToRs)
Test-targeting, descriptive power and comparability of cognitive
items
A1. Scale expansion and equating with current PISA scales and proficiency
levels at the lower ends of performance (i.e. 1a, 1b and below in reading)
for all three domains (including data available from PISA countries with
majority of student performance at lower levels)
A2. Review of scaling models used in PISA and how these could be
operational with new participants (e.g. specific variables used for
conditioning) starting with challenges found and the data sets available
from PISA countries
B.
Review and discussion of the PISA frameworks and constructs being
assessed in reading, mathematics and science in light of component skills
included in frameworks of PASEC, SACMEQ, Pre-PIRLS and others
C.
Review of available item pool of PISA to classify those that may be used
for the revised instruments based on item difficulty, framework coverage
and bias/non bias for developing country contexts (item parameters).
PISA for Development
Technical issues to be addressed (for ToRs)
Methods and approaches for including out-of-school children in
data collection and assessment
A.
Review of specific challenges and technical issues to be addressed in the
data collection methods with OOSCI including, but not limited to,
unavailability of sampling frame of population, identification and
engagement with children, and instruments for contextual information.
This should identify specific field examples to be explored.
B.
Review of methods for implementing manually adaptive testing
methods using gateways/channels so that an initial basic instrument
identifies the appropriate streams of questions/data collection. This
should include other methods such as verbal surveys.
C.
Review of evidence and approaches to the assessment of knowledge, skills
and competencies in reading and numeracy/mathematics (and science)
with out-of-school children. This should include frameworks, constructs
and variables used in the assessment, scaling and reporting.
PISA for Development
Technical issues to be addressed (for ToRs)
Methods and approaches for including out-of-school children in
data collection and assessment
D.
Review of evidence and relevant studies to establish an appropriate
definition of the target population - ‘out of school’ children with regards
to PISA data collection - starting with the target population of 15-yearolds. As much as possible, this should then be discussed in relation with
the population of participating countries (preliminary).
E.
Technical review of possible application of the 5DE framework of the
UNICEF/UIS study in PISA for Development (for students and OOSC),
relating to issues such as disparities between age/grades (e.g. many 15years olds enrolled in primary school). As much as possible, this should
refer to information and contexts in participating countries (including
sub-groups of particular interest to countries, e.g. SEN, ethnic
minorities).