Transcript Document

MURI: Training Knowledge and
Skills for the Networked Battlefield
ARO Award No. W9112NF-05-1-0153
Alice Healy and Lyle Bourne,
Principal Investigators
Benjamin Clegg, Bengt Fornberg,
Cleotilde Gonzalez, Eric Heggestad,
Ronald Laughery, Robert Proctor,
Co-Investigators
Project Mission
(As Defined by the BAA)
Objectives
“Develop and evaluate models that predict performance
improvement or decrement for a range of militarily
significant individual and collective tasks that can be
linked to various types and amounts of training while
considering the effects of aptitude and experience.”
Proposed Project
(As Defined in Executive Summary)
Goals
• Construct a theoretical & empirical framework for
training
• Predict the outcomes of different training methods on
particular tasks
• Point to ways to optimize training
Statement of Work
The work to be performed falls into 3 interrelated categories:
(1) Experiments
(a) Development & testing of training principles
(b) Acquisition & retention of basic skill components
(c) Levels of automation, individual differences, & team
performance
(2) Taxonomic analysis
(a) Training methods
(b) Task types
(c) Performance measures
(d) Training principles
(3) Predictive computational models
(a) Formulated from experimental data
(b) Applied to military tasks
Parts of Project
(1) Experiments
(a) Development & Testing of Training Principles
(b) Acquisition & Retention of Basic Components of Skill
(c) Levels of Automation, Individual Differences, & Team
Performance
(2) Taxonomy
(3) Models
(a) ACT-R
(b) IMPRINT
(c) Model Assessment
Three Major Parts of Present
Meeting
(I) Introduction
(II) Plans For Project and Progress So Far
(A) Experiments
(B) Taxonomy
(C) Models
(III) Summary and Reactions
Introduction to MURI Personnel
(1) University of Colorado (CU)
Alice Healy, Principal Investigator
Lyle Bourne, Co-Principal Investigator
Bengt Fornberg, Co-Investigator
Ron Laughery, Co-Investigator
Bill Raymond, Research Associate
(2) Carnegie Mellon University (CMU)
Cleotilde Gonzalez, Co-Investigator
(3) Colorado State University (CSU)
Ben Clegg, Co-Investigator
Eric Heggestad, Co-Investigator
(4) Purdue University (Purdue)
Robert Proctor, Co-Investigator
Roles in Project
(1) Overview and Coordinate
CU, Healy & Bourne
(2) Experiments
(a) Development & Testing of Training Principles
CU, Healy & Bourne
(b) Acquisition & Retention of Basic Components of Skill
Purdue, Proctor
(c) Levels of Automation, Individual Differences, & Team
Performance, CSU, Clegg & Heggestad
(3) Taxonomy
CU, Raymond
(4) Models
(a) ACT-R
CMU, Gonzalez
(b) IMPRINT
CU, Laughery
(c) Model Assessment
CU, Fornberg
Key Comments from Review
(1) Tighter integration of the modeling effort with the learning research and
experimentation is needed and should take place during the first few months of
the project.
(2) Data-tractability (how much data on the training and the subjects are needed to
make reasonable evaluations) and computational tractability need to be addressed in
greater depth.
(3) Training in a complex networked environment could be addressed at greater depth,
but, since even training for more elementary tasks is not yet understood, the proposed
work is reasonable.
(4) More emphasis on software and less emphasis on papers published in professional
journals and books is needed in the deliverables.
(5) There is a question about how the obligations of one senior MURI team member to
a company and to the Advanced Decision Architectures Collaborative Technology
Alliance will be coordinated with that member’s obligation to the MURI.
Outline of Plans for Project and Progress So Far
(I) Preliminary Investigator’s Meeting, Boulder, May 25, 2005
(II) Preparation of Investigators’ WIKI and Public MURI website
(III) Experiments
(A) Development & Testing of Training Principles
Healy & Bourne
(B) Acquisition & Retention of Basic Components of Skill
Proctor
(C) Levels of Automation, Individual Differences, & Team Performance
Clegg & Heggestad
(IV) Taxonomy
Raymond
(V) Models
(A) ACT-R
Gonzalez
(B) IMPRINT
Laughery
(C) Model Assessment
Fornberg
Development and Testing of Training
Principles
• Summary of 30 Training Principles: Prepared for NASA
cooperative agreement
• Two Examples of Training Principles
Strategic-Use-of-Knowledge Principle
When a large amount of new factual information must be
learned and retained, that information should be related
to the learner’s existing knowledge in any way possible.
Principle of Contextual Interference
Introduce sources of interference into training material.
Interference may weaken performance during training
but should strengthen retention and transfer.
Development and Testing of
Training Principles: Proposed and
In-Progress Experiments
(1) Tests of the generality across tasks of individual
principles -- 1 in-progress on strategic use of
knowledge
(2) Tests of multiple principles in a single task -- 1 inprogress on serial position, list length, and
chunking effects
(3) Tests of principles in complex, dynamic
environments -- 1 in-progress on contextual
interference
CU Experiments: Communication with
Modelers to Date
(1) Data Entry: Fatigue Effects; Speed-Accuracy Tradeoffs
Sent to Gonzalez & Laughery data from 2 previously published
experiments
(2) Hand-Eye Coordination: Specificity of Training; Retention and
Transfer Effects
Sent to Gonzalez & Laughery data from 1 unpublished experiment
(3) Further Work on Data Entry: Multiple Principles in a Single Task
Sent to Gonzalez & Laughery data from 8 previously published
experiments examining (a) specificity of training, (b) procedural
reinstatement, (c) depth of processing, (d) phonological coding
Sent to Gonzalez & Laughery data from 2 newly completed
experiments examining (a) cognitive and motoric fatigue, (b) feedback
and cognitive load
Data Entry Experiments
Task: Subjects see a 4-digit number, and they type it on a
computer keypad
Design: In each session half, subjects see and type 5 blocks of
64 numbers
Measures: Both typing accuracy (proportion correct) and
typing speed (total response time) are measured
Healy, Kole, Buck-Gengler, & Bourne (2004)
Experiment 1
0.91
2.70
2.68
0.90
Proportion Correct
Total Response Time
2.66
0.89
2.64
0.88
2.62
0.87
0.86
2.60
1
2
3
Block
4
5
2.58
Total Response Time (in s)
Proportion Correct
Kole, Healy, and Bourne (2005)
Experiment 1
0.92
0.90
Proportion Correct
Suppression
Silent
Suppression
0.88
Silent
0.86
0.84
0.82
1
2
3
Block
4
5
No Weight
Weight
Kole, Healy, and Bourne (2005)
Experimernt 2
0.92
0.90
Proportion Correct
Feedback
No Feedback
Feedback
0.88
No Feedback
0.86
0.84
0.82
1
2
3
Block
4
5
Data Entry
Multiplication
CU Experiments: Communication with Modelers
Planned for Next Year
(1) Data Entry: Mental Rehearsal
2 experiments on repetition priming and motor imagery
(2) Hand-Eye Coordination: Further Work on Specificity of Training
1 experiment assessing relative merits of specificity and variability of training
1 experiment on strategy instructions and gender effects
1 experiment on immediate testing and transfer
(3) Duration Estimation: Functional Task Principle
1 experiment varying presence of secondary task
2 experiments varying features of secondary and primary tasks
2 experiments varying difficulty and modality of secondary task
CU Experiments: Expanded Work on Complex Tasks
(1) RADAR Task from CMU
Test of Training Difficulty Principle
(2) First Responder Navigation Task with Emergencies from NSF
SGER Grant
Test of Memory Constriction Hypothesis
Test of Look-Up Speed in Emergency Check-List
Summary of Response to Comments from
Review
(1) Tighter integration of modeling effort and experimentation
Experimenter-taxonomist-modeler interactions are on-going, facilitated by
meetings and WIKI
(2) Data and computational tractability
Assessments of data-computation compatibility are on-going, facilitated by
meetings between Fornberg and research personnel
(3) Training in a complex networked environment
Experiments underway using complex and more naturalistic tasks, such as
RADAR tracking, emergency response teams, flight simulation
(4) More emphasis on software in the deliverables
ACT-R and IMPRINT software products will be available at various points in
the future
(5) The multiple obligations of one senior MURI team member
Periodic meetings between senior member and research associate, enabling use of
IMPRINT by other team members to model existing data
Present and Future Activities
(1) Activities
(a) Experiments
(b) Taxonomy
(c) Modeling
(2) How do we propose to get from the current state of knowledge
to the final goal of predicting performance as a function of
training