Transcript Computer-based Testing

2004 International Conference of NTNTC GIMS and RDC;
Technology-Based Testing and Standard-Based Assessment
Introduction to Education in Japan:
Web-based Learning Environment
Ｗｅｂを基盤とした学習環境
(e-Learning)
&
Instructional Design
Eiji Muraki
Professor, Department of Educational
Informatics, Tohoku University
1
IT Revolution
時代状況としてのＩＴ革命






IT＝Information Technology
Developed at CERN, Internet started as a tool
for communication among scientists
IT is our current condition surrounded us
IT requires us to communicate in new ways
IT is the technological innovation, but we
need to develop some new ideological
foundations to live in this IT world
How do we live happily in this IT age?
2
Emergence of e-Learning



Since 1990’s
Virtual University、 Online Education
Kenneth C. Green

American Higher Educational Institutions

e-Learning course





1999
2000
2001
2002
46.5%
55.5%
59.0%
62.5％
Outbreaking period 1995－1997
3
The definition of “e-Learning”


e-Learning is learning in which students do
and learn by trial and error through practice
on the virtual environment created by
multimedia and computer networks, aided by
rich informational resources, knowledge
support, and feedback（山崎将志、 eラーニン
グ：実践的スキルの習得技法）。
Closed virtual learning enviroment
4
Background of e-Learning(USA)
吉田文（アメリカの高等教育におけるeラーニング：日本への教訓）

Advancement of IT technology and Spread of Internet









Mutual communication
Development of multimedia function
Basic communication is inexpensive
Increased adult population for higher education: Adult working
population
Economic merits by academic degrees：Technical occupations,
Occupational education
Emergence of commercialized universities and
Commercialization of traditional universities
Improved Course Management System：WebCT, BlackBoard,
TopClass, LearningSpace, etc．
Emergence of media specialists：Instructional Designer with
other specialists (Teaching Faculty, Graphic Designer, Media
Editor, Computer Specialist, Content Specialist, etc).
Internationalized curriculum
5
Current e-Learning Situation



Leading market principle and Lack of research
on e-Learning educational process
Poor evaluation process
Difficulties of face-to-face communication,
Socializational factor of educational process



Business education, workshop：Web-based
Training (WBT)、Corporate University, and
Technical knowledge
Suitable for education of working adults, but is it
suitable for academic graduate education
Is it impossible to construct individualized eLearning?
6
Transition of Views on Learners





Physiological Psychology
Behaviorism
Neo-Behaviorism
Cognitive Psychology
Constructionism
7
Instructional Design
The systematic and reflective
process of translating principles of
learning and instruction into plans for
instructional materials, activities,
information resources, and
evaluation.
8
Teaching vs. Instruction

Teaching


Learning experiences that are facilitated by
a human being – not a videotape, textbook,
or computer program, but a live teacher.
Instruction


All learning experiences in which the
instructional support is conveyed by
teaching and other forms of mediation.
A live teacher is not essential to all
instruction.
9
Instructional Design Process
1.
2.
3.
Perform an instructional analysis to
determine “where we’re going.”
Develop an instructional strategy to
determine “how we’ll get there.”
How will we know when we have
arrived? (What should our tests look
like? How will we evaluate and revise
the instructional materials?)
10
Instructional Design Process

Analysis 分析



Learning contexts, Learners, Learning task
Write test items
Strategy 方策

Determine



Organizational strategies, Delivery strategies, Management
strategies
Write and produce instruction
Evaluation 形成的評価


Conduct formative evaluation
Revise instruction
11
Teacher-centeredから
Learner–centered
Evaluation&Feedback
 Diagnostic test
 Computer-adaptive test
 Cognitive Science
 Constructivism

12
Formative Evaluation
形成的評価




Norm-referenced test vs. Criterionreferenced test
Formative test vs. Summative test
学力assessment vs. 学力diagnostic test
B. S. BloomのMastery Learning Theory
完全習得学習理論
13
Classification of Computer-Based
Testing
by Bunderson et al. (1989)
1.
2.
3.
4.
Computerized Testing CT
Computerized Adaptive Testing
CAT
Continuous Measurement CM
Intelligent Measurement IM
14
コンピュータ版テスト
Computer-based Testing
＆
e-Testing
形成的評価
Formative Evaluation
15
Merits of CBT




マルチメディア機能を用いた新しいタイプのテスト項
目：Multimedia capability - New Item Types
新しい心理構成概念：New Psychological
Constructs
ゲームのような新しいタイプのテスト項目：New
types of questions: simulation of real life or
game-like testing
認知的な診断：Cognitive diagnosticーa profile of
the strengths and weaknesses
16
Current CBT



Most of test items are Multiple Choice or TrueFalse：二値反応項目(dichotomous response item)
試験の機密管理（Test Security）に多大な費用(very
expensive)
Technical difficulties for individual identification of
test takers 被験者の個人認定に技術的な問
題:particularly on the Internet


But no problem for formative evaluation 形成的評
価に用いることには問題はない
It is very suitable for e-Testing for e-Learning.
17
New CBT item types






作業式課題：Performance-oriented
相互作用タイプ：Interactive responses
シミュレーションタイプ：Real situations or
simulated reality
高度な認知機能：Higher cognitive
function
New constructs
多段階評価：Graded scoring
18
世界的標準化
（International Standardization）
e-Learning とe-Testing





Hardware
Software
Environment
Identification
Test security
19
Automatic Essay Scoring
for CM & IM
自動作文採点





Standardization of Scoring Process 採点プロセス
の標準化
Human ratersの採点行動Scoring practiceはおおく
の余分な要因prone to make errorsを持っている
説明できる採点過程 Scoring process is
explainable.
採点者（あるいは採点プロセス）の信頼度 rater
reliability
作文採点 essay scoring: e-rater, PEGなど
20