Transcript The PISA framework for assessment of ICT literacy

The PISA framework for
assessment of ICT literacy
Wouter van Joolingen
University of Amsterdam
ICT Expert Panel
Panel Members
Project Staff and Counsel
Irwin Kirsch (Chair), United
States
Robert Kozma, United States
Chul-Hwan Lee, Korea
Jörgen Nissen, Sweden
Catherine Régnier, France
Jean-Paul Reeff, Luxembourg
Arnold Spee, The Netherlands
Wouter van Joolingen, The
Netherlands
Tracy Gray, United States
Marylou Lennon, United States
Jay Moskowitz, United States
Maria Stephens, United States
Claudia Tamassia, France
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Why measure ICT
literacy?

One goal of PISA is to measure how well young
–


adults are prepared to meet the challenges of the future.
For many, if not most, that future will include ICT
Therefore, PISA should include a measure of ICT



skills as one of its cross-disciplinary
competencies.
ICT Panel members felt that the rationale for such

an assessment is grounded in a number of issues of concern to
policy makers and others that include:
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Why measure ICT
literacy?

ICT is changing the very nature and value of
knowledge and information
 ICT literacy in its highest form has the
potential to change the way we live, learn and
work
 ICT literacy cannot be defined primarily as the
mastery of technical skills
 There is a lack of information about the
current levels of ICT literacy both within and
among countries
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Defining ICT Literacy
ICT literacy is the interest, attitude and ability of
individuals to appropriately use digital
technology and communication tools to access,
manage, integrate and evaluate information ,
construct new knowledge, and communicate
with others in order to participate effectively in
society.
Universiteit van Amsterdam
Graduate school of Teaching and Learning
ICT processes
Access
Knowing about and knowing how to collect and/or
retrieve information.
Manage
Organising information into existing classification
schemes.
Integrate
Interpreting, summarising, comparing and contrasting
information using similar or different forms of
representation.
Evaluate
Reflecting to make judgments about the quality,
relevance, usefulness, or efficiency of information.
Construct
Generating new information and knowledge by
adapting, applying, designing, inventing, representing
or authoring information.
Communicate
Conveying information and knowledge to various
individuals and/or groups.
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Test setup for PISA

Computer-based testing
– Students will work on a notebook computer

Closed web environment with test
scenario’s
– Simulated “real ICT-world”
 Small web
 Simulated responses from the system and the
external world
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Building ICT Tasks
SCENARIOS
Technology Environments
Web
Desktop
E-learning simulations
Contexts
Personal
Educational
TASKS
Processes
Access
Integrate
Construct
Manage
Evaluate
Communicate
Public
Occupational
Functionality
Word Processing
E-mail
Presentation
Database
E-commerce
Spreadsheet
Concept Mapping
Browser (includes search engines)
Representations
Text
Numbers
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Images
Graphics
A possible scenario

Your friend, Jim, has a new hobby,
collecting stamps. He just started it, and
is still a beginner on this topic. His
birthday is approaching and you decide
you want to give Jim a book on stamp
collecting, that matches his level of
expertise. You don’t know anything about
stamps, so you start searching for a
suitable book.
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Some considerations
To find good books, learn something on
the topic yourself
 Use smart search strategies to identify
central topics
 Evaluate reliability of reviews and
different bookshops (including 2nd hand)
and compare deals.
 Need to make more explicit than in
“natural” settings.

Universiteit van Amsterdam
Graduate school of Teaching and Learning
Scoring
Access
Search strategies – key words used
Recycling key words, sites used.
Go halfway in the buying process
Manage
Organization of reviews
Integrate
Linking reviews, books and other knowledge
Evaluate
Assess trust in reviews and shops – sources used
in task products
Construct
Build ordered lists of findings
Communicate
Make deal with on-line shop
Present findings to others
Universiteit van Amsterdam
Graduate school of Teaching and Learning
The framework

Allows for the construction of an
assessment instrument that
– Does justice to many aspects of ICT literacy
– Allows for testing in realistic situations
– Goes beyond self assessment
– Provides an objective account of an
individual’s ICT skills and knowledge.
Universiteit van Amsterdam
Graduate school of Teaching and Learning
Plans

Feasibility study
– Try-out with small number of test scenarios
– USA, Australia, Japan
– Data collected – analysis in progress
Test development, field trial (2004-2005)
 Inclusion in PISA 2006

– At least 15 OECD countries
Universiteit van Amsterdam
Graduate school of Teaching and Learning