Transcript Slide 1

eAssessment for 21st Century
Learning and Skills
Anusca Ferrari
Christine Redecker & Yves Punie
JRC-IPTS
IPTS- Who we are
IPTS: Part of Joint Research Centre
of the EC: 7 Research Institutes
across Europe
Mission: “to provide customerdriven support to the EU policymaking process by developing
science-based responses to policy
challenges that have both a socioeconomic as well as a
scientific/technological dimension”
IPTS Studies on ICT for Learning
2005
2006
2007
2008
2009
2010
2011
2012
Future Learning Spaces in 2020
eLearning in the EU10 MS
Review ICT
Impact
- Digital Competence
- Active Ageing & Learning
Learning 2.0 in formal E&T
Learning 2.0 in informal settings
Creativity & Innovation in
schools in EU
The Future of Learning
Tell-Net Teacher Networking
Digital Competence
ICT for the Assessment of
Key Competences
Where we stand
1990
Personalised learning
1995
2000
2005
Generation 1
Generation 2
Automated administration
and/or scoring of
conventional tests
Adaptive Testing
2010
2015
Generation 3
2020
Generation 4
ContinuousData Mining & Analysis
Personalised
integrated
feedback and
Behavioural tracking
assessment
tutoring
Learning Analytics
Multiple Choice
Short Answer
Automated Feedback
Replication of complex
real life situations within
a test
Intelligent Tutors
Generation Re-invention
Transformative strategy: use technologyGames
Virtual Worlds
to change test formats
Online collaboration
Virtual laboratories
Peer assessment
Simulations
Learning 2.0
Collaborative multimedia learning
environments
Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L.
Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan
Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.),
International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier.
Efficient testing
Where we stand
1990
Personalised learning
1995
2000
2005
Generation 1
Generation 2
Automated administration
and/or scoring of
conventional tests
Adaptive Testing
2010
2015
2020
Generation 3
Data Mining & Analysis
Behavioural tracking
Learning Analytics
Multiple Choice
Short Answer
Automated Feedback
Intelligent Tutors
Replication of complex
real life situations within
a test
Generation Re-invention
Transformative strategy: use technology
to change test formats
Computer-Based Assessment (CBA)
Games
Virtual laboratories
Simulations
Continuous
integrated
assessment
Generation 4
Personalised
feedback and
tutoring
Virtual Worlds
Online collaboration
Peer assessment
Learning 2.0
Collaborative multimedia learning
environments
Technology-enhanced learning
Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L.
Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan
Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.),
International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier.
Efficient testing
Examples
1st Generation CBA:
Problem-Solving
http://www.worldclassarena.org/v5/flash/0
9_year_old/backtofront.swf
World Class Arena Sample Test: Problem
Solving (9 years)
Examples
Generation Re-Invention:
Problem-Solving
http://www.worldclassarena.org/v5/flash/
13_year_old/pyramids.swf
Problem Solving 13 years
1st Generation CBA:
Examples Situational Judgement
EPSO sample situational judgement test questions; http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm
EPSO sample assessment centre e-tray exercise
http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm
Generation Re-Invention:
e-tray exercise
Some Research
findings on CBA
•
•
•
•
•
Advantages of CBA:
Standardization of test administration;
Machine-scorable responses;
Opportunities for more interactive question types
Automatic scoring enables teachers to focus on analysis and interpretation
Automatically generated reports supporting formative assessment
• Evidence:
• Computer-based tests have (arguably) a positive effect on students' motivation,
concentration and performance;
• Evidence on the comparability of computer- and paper-based testing
inconclusive;
• Reliability and validity are continuously being improved;
• CBA also useful for formative assessment.
• Automated Scoring
• For short-answer free text responses automatic scoring is at least as good as
human scoring;
• For essays, automatic and human scoring are highly correlated; discrepancies
arise at the top end of scores;
• Automatic scoring of highly predictable speech (e.g. one sentence answers)
matches with human scoring.
Future eAssessment
Strategies
1990
Personalised learning
1995
2000
Generation 1
Automated administration
and/or scoring of
conventional tests
2005
2010
Generation 2
2015
2020
Data Mining & Analysis
Adaptive Testing
Behavioural tracking
Learning Analytics
Multiple Choice
Automated Feedback
Intelligent Tutors
Generation Re-invention
Transformative strategy: use technology
to change test formats
Computer-Based Assessment (CBA)
Continuous
integrated
assessment
Generation 4
Short Answer
Replication of complex
real life situations within
a test
Generation 3
Games
Virtual laboratories
Simulations
Personalised
feedback and
tutoring
Virtual Worlds
Online collaboration
Peer assessment
Learning 2.0
Collaborative multimedia learning
environments
Technology-enhanced learning
Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L.
Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan
Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.),
International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier.
Efficient testing
Examples
Towards Generation 3:
Datamining
Signals
http://www.itap.purdue.edu/tlt/
signals/signals_final/index.htm
Similar: GPS at Northern
Arizona University
http://resourceconnect.nau.edu/
grade-performance-status.asp
Examples
Towards Generation 3: Behaviour Tracking
Example: SNAPP
SNAPP is a software tool
that allows users to
visualize the network of
interactions resulting from
discussion forum posts and
replies. Teachers can
rapidly identify patterns of
user behaviour. SNAPP
can extract all user
interactions from various
commercial and open
source learning
management systems, such
as BlackBoard and
Moodle.
Examples
Towards Gen 4:
Intelligent Tutoring Systems
AutoTutor
https://www.youtube.com/watch?v=k6HDEEnYuIA
&feature=player_embedded
GnuTutor (http://gnututor.com/)
OpenSource version of AutoTutor
https://www.youtube.com/watch?v=k6HDEEnYuIA
&feature=player_embedded
Examples
Learning 2.0:
Virtual Laboratories
Example from Molecular Workbench
http://workbench.concord.org/database/activities/19.html
http://workbench.concord.org/database/
Virtual Frog Dissection Kit
http://froggy.lbl.gov/cgi-bin/dissect
Examples
Virtual
Laboratories
BioLogica
http://biologica.concord.org/
Ingredients:
• enjoyable and engaging
learning activities
• monitoring progress
• electronic tutoring
• portfolio building
Examples
Immersive Games
Quest Alantis
Quest Atlantis (QA) is an
international learning and
teaching project that uses a
3D multi-user environment
to immerse children, ages 916, in educational tasks. QA
combines strategies used in
the commercial gaming
environment with lessons
from educational research on
learning and motivation.
Examples
Immersive Games
Global Conflicts
http://www.globalconflicts.eu/
Educational game,
designed to help teach
concepts in citizenship,
geography, and media.
Developed by Serious
Games International, it
has detailed lesson plans
and assignments for
students.
Example: Child labour in Bangladesh:
http://www.youtube.com/watch?v=mf9Ju86IVYw
Conclusions
1990
1995
Personalised learning
2000
2005
2010
Generation 1
Generation 2
Generation Re-invention
- mainstream use for high stakes testing
- further development of innovative test formats
- increased ease of use at school level
- trend towards automated scoring
→ cultural change:
increased acceptance of CBA
We are in a transitional period: Moving from the “Testing
Paradigm” to the (future) “Tutoring Paradigm” in which
explicit testing becomes obsolete
Computer-Based Assessment (CBA)
2015
2020
Generation 3
Generation 4
- Multimedia collaborative
learning environments support
21st century skills
-Data mining and analysis will
make integrated assessment and
tutoring possible in the future.
Learning 2.0
Technology-enhanced learning
Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L.
Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan
Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.),
International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier.
Efficient testing
The Paradigm Shift
Era of Computer-Based Assessment (CBA)
Era of Technology-enhanced learning
Focus on: Efficient testing
1990
1995
Focus on: Personalised learning
2000
2005
2010
2015
2020
Technology
CBA
(Gen.1)
Adaptive
Tests
(Gen.2)
Automatic
Scoring
Gen. ReInvention
Learning
2.0
Immersive
Learning
Learning
Analytics
Intelligent
Tutors
Pedagogy
Efficient
Testing
Validity of
Scores
Quick
Authentic
Feedback Contexts
?
?
?
?
→ Primacy of Pedagogy:
Pedagogical guidelines for intergrated assessment
Transparency & teacher mediation
• Thank you very much for your attention!
• Anusca Ferrari
• Christine Redecker
• Yves Punie
• [email protected]
• [email protected]
• [email protected]
• http://is.jrc.ec.europa.eu/pages/EAP/eLearning.html
further information ……
http://is.jrc.ec.europa.eu
Newsletters:
http://is.jrc.es/pages/newsletter.html
Publications:
http://is.jrc.ec.europa.eu/pages/Publications.html
Contact:
[email protected]