MURI: Training Knowledge and Skills for the Networked BattlefieldARO Award No. W9112NF-05-1-0153

Alice Healy and Lyle Bourne, Principal Investigators

Benjamin Clegg, Bengt Fornberg, Cleotilde Gonzalez, Eric Heggestad, Ronald Laughery, Robert Proctor,

Co-Investigators

Project Mission(As Defined by the BAA)

Objectives

“Develop and evaluate models that predict performance

improvement or decrement for a range of militarily

significant individual and collective tasks that can be

linked to various types and amounts of training while

considering the effects of aptitude and experience.”

Proposed Project(As Defined in Executive Summary)

Goals

• Construct a theoretical & empirical framework for training

• Predict the outcomes of different training methods on particular tasks

• Point to ways to optimize training

Statement of Work

The work to be performed falls into 3 interrelated categories:

(1) Experiments(a) Development & testing of training principles(b) Acquisition & retention of basic skill components(c) Levels of automation, individual differences, &

team performance

(2) Taxonomic analysis (a) Training methods(b) Task types (c) Performance measures(d) Training principles

(3) Predictive computational models(a) Formulated from experimental data (b) Applied to military tasks

Parts of Project(1) Experiments

(a) Development & Testing of Training Principles

(b) Acquisition & Retention of Basic Components of Skill

(c) Levels of Automation, Individual Differences, & Team Performance

(2) Taxonomy

(3) Models

(a) ACT-R

(b) IMPRINT

(c) Model Assessment

Three Major Parts of Present Meeting

(I) Introduction

(II) Plans For Project and Progress So Far

(A) Experiments

(B) Taxonomy

(C) Models

(III) Summary and Reactions

Introduction to MURI Personnel

(1) University of Colorado (CU)Alice Healy, Principal InvestigatorLyle Bourne, Co-Principal InvestigatorBengt Fornberg, Co-InvestigatorRon Laughery, Co-InvestigatorBill Raymond, Research Associate

(2) Carnegie Mellon University (CMU)Cleotilde Gonzalez, Co-Investigator

(3) Colorado State University (CSU)Ben Clegg, Co-InvestigatorEric Heggestad, Co-Investigator

(4) Purdue University (Purdue)Robert Proctor, Co-Investigator

Roles in Project

(1) Overview and CoordinateCU, Healy & Bourne

(2) Experiments(a) Development & Testing of Training Principles

CU, Healy & Bourne(b) Acquisition & Retention of Basic Components of Skill

Purdue, Proctor(c) Levels of Automation, Individual Differences, & Team

Performance, CSU, Clegg & Heggestad (3) Taxonomy

CU, Raymond(4) Models

(a) ACT-RCMU, Gonzalez

(b) IMPRINTCU, Laughery

(c) Model AssessmentCU, Fornberg

Key Comments from Review(1) Tighter integration of the modeling effort with the learning research and experimentation is needed and should take place during the first few months of the project.

(2) Data-tractability (how much data on the training and the subjects are needed to make reasonable evaluations) and computational tractability need to be addressed in greater depth.

(3) Training in a complex networked environment could be addressed at greater depth, but, since even training for more elementary tasks is not yet understood, the proposed work is reasonable.

(4) More emphasis on software and less emphasis on papers published in professional journals and books is needed in the deliverables.

(5) There is a question about how the obligations of one senior MURI team member to a company and to the Advanced Decision Architectures Collaborative Technology Alliance will be coordinated with that member’s obligation to the MURI.

Outline of Plans for Project and Progress So Far

(I) Preliminary Investigator’s Meeting, Boulder, May 25, 2005(II) Preparation of Investigators’ WIKI and Public MURI website(III) Experiments

(A) Development & Testing of Training PrinciplesHealy & Bourne

(B) Acquisition & Retention of Basic Components of SkillProctor

(C) Levels of Automation, Individual Differences, & Team Performance Clegg & Heggestad

(IV) TaxonomyRaymond

(V) Models(A) ACT-R

Gonzalez(B) IMPRINT

Laughery(C) Model Assessment

Fornberg

Development and Testing of Training Principles

• Summary of 30 Training Principles: Prepared for NASA cooperative agreement

• Two Examples of Training Principles

Strategic-Use-of-Knowledge Principle When a large amount of new factual information must be learned and retained, that information should be related to the learner’s existing knowledge in any way possible.

Principle of Contextual InterferenceIntroduce sources of interference into training material. Interference may weaken performance during training but should strengthen retention and transfer.

Development and Testing of Training Principles: Proposed and

In-Progress Experiments

(1) Tests of the generality across tasks of individual principles -- 1 in-progress on strategic use of knowledge

(2) Tests of multiple principles in a single task -- 1 in-progress on serial position, list length, and chunking effects

(3) Tests of principles in complex, dynamic environments -- 1 in-progress on contextual interference

CU Experiments: Communication with Modelers to Date

(1) Data Entry: Fatigue Effects; Speed-Accuracy Tradeoffs

Sent to Gonzalez & Laughery data from 2 previously published experiments

(2) Hand-Eye Coordination: Specificity of Training; Retention and Transfer Effects

Sent to Gonzalez & Laughery data from 1 unpublished experiment

(3) Further Work on Data Entry: Multiple Principles in a Single Task

Sent to Gonzalez & Laughery data from 8 previously published experiments examining (a) specificity of training, (b) procedural reinstatement, (c) depth of processing, (d) phonological coding

Sent to Gonzalez & Laughery data from 2 newly completed experiments examining (a) cognitive and motoric fatigue, (b) feedback and cognitive load

Data Entry Experiments

Task: Subjects see a 4-digit number, and they type it on a

computer keypad

Design: In each session half, subjects see and type 5 blocks of

64 numbers

Measures: Both typing accuracy (proportion correct) and

typing speed (total response time) are measured

1 2 3 4 50.86

0.87

0.88

0.89

0.90

0.91

2.58

2.60

2.62

2.64

2.66

2.68

2.70

Proportion Correct

Total Response Time

Healy, Kole, Buck-Gengler, & Bourne (2004) Experiment 1

Block

Proportion CorrectTotal Response Time (in s)

1 2 3 4 50.82

0.84

0.86

0.88

0.90

0.92

Suppression

SilentSuppression

Silent

Kole, Healy, and Bourne (2005) Experiment 1

Block

Proportion Correct

No Weight

Weight

1 2 3 4 50.82

0.84

0.86

0.88

0.90

0.92

Feedback

No FeedbackFeedback

No Feedback

Kole, Healy, and Bourne (2005) Experimernt 2

Block

Proportion Correct

Data Entry

Multiplication

CU Experiments: Communication with Modelers Planned for Next Year

(1) Data Entry: Mental Rehearsal

2 experiments on repetition priming and motor imagery

(2) Hand-Eye Coordination: Further Work on Specificity of Training

1 experiment assessing relative merits of specificity and variability of training

1 experiment on strategy instructions and gender effects

1 experiment on immediate testing and transfer

(3) Duration Estimation: Functional Task Principle

1 experiment varying presence of secondary task

2 experiments varying features of secondary and primary tasks

2 experiments varying difficulty and modality of secondary task

CU Experiments: Expanded Work on Complex Tasks

(1) RADAR Task from CMU

Test of Training Difficulty Principle

(2) First Responder Navigation Task with Emergencies from NSF SGER Grant

Test of Memory Constriction Hypothesis

Test of Look-Up Speed in Emergency Check-List

Summary of Response to Comments from Review

(1) Tighter integration of modeling effort and experimentationExperimenter-taxonomist-modeler interactions are on-going, facilitated by

meetings and WIKI

(2) Data and computational tractabilityAssessments of data-computation compatibility are on-going, facilitated by

meetings between Fornberg and research personnel

(3) Training in a complex networked environmentExperiments underway using complex and more naturalistic tasks, such as

RADAR tracking, emergency response teams, flight simulation

(4) More emphasis on software in the deliverablesACT-R and IMPRINT software products will be available at various points

in the future

(5) The multiple obligations of one senior MURI team memberPeriodic meetings between senior member and research associate, enabling

use of IMPRINT by other team members to model existing data

Present and Future Activities

(1) Activities

(a) Experiments

(b) Taxonomy

(c) Modeling

(2) How do we propose to get from the current state of knowledge to the final goal of predicting performance as a function of training