1

Automaticity development and decision making in complex, dynamic tasks

Dynamic Decision Making Laboratorywww.cmu.edu/DDMLab

Social and Decision Sciences DepartmentCarnegie Mellon University

Cleotilde GonzalezRickey ThomasPolina Vanyukov

2

Complex and dynamic tasks

Executing a battle, driving, air traffic controlling, managing of a production plan, piloting, managing inventory in a production chain, etc.

• Demand real-time decisions (time constraints)

• Demand attentional control

• Require multi-tasking: they are composed of multiple and interrelated subtasks

• Demand the identification of ‘targets’ defined by multi-attributes

• Demand multiple and possibly changing responses

3

Automaticity in dynamic, complex tasks

• targets and distractors are often inconsistently mapped to stimuli and responses

• Often, we bring pre-learned categories and mappings to a task

stimulus - category category - responseL ------------- letter button --------- click

• Are decision makers in dynamic situations operating in controlled processing continuously?

4

Proposed model of automaticity in DDM

Cues Categories Responses

sub-task Structure (Mapping)

CM/VM

coupling

Cues Categories Responses

sub-task Structure (Mapping)

CM/VM

coupling

Goals (Relevancy)

Task switching (resource allocation)

Cues Categories Responses

sub-task Structure (Mapping)

CM/VM

coupling

Cues Categories Responses

sub-task Structure (Mapping)

CM/VM

coupling

5

Experiments

• Automaticity develops with consistently mapped stimuli to targets, even when targets move and time is limited (Experiment 1)

• The consistency of target to response mapping also determines automaticity development (Experiment 2)

• Automaticity of a task component frees-up time and resources for high level decision-making (Experiment 3)

• Automaticity develops differently with different degrees of pre-learned categories (Experiment 4)

6

The Radar Task

7

8

General method

• Independent variableso stimulus mapping (CM or VM)

• CM = Search for Numbers in Letters• VM = Search for Letters in Letters

o cognitive load• Memory set size (MSS): Number of possible targets to remember (1 or 4)• frame size (FS): Number of blips present on the screen at a given time (1

or 4)o target present/absent (a target was present 75% of the trials)

• Dependent variableso Accuracy: proportion of correct detections or decision-making

responseso Time: mean target detection or decision-making time in msec

• From 18 to 30 hours of practice, 3 hours per day 6 to 10 days

9

Experiment 1: Consistency of stimuli

• Replicate major findings from the dual-process theory (Schneider & Shiffrin, 1977) in a dynamic task

• Automaticity is acquired with practice in consistent mapping conditions, and automatic performance is unaffected by workload

10

Experiment 1: Method

o CM vs. VM

o Cognitive Load Variables• Memory Set Size• Frame Size

o Only one possible response: pressing spacebar when target is detected

11

Experiment 1: Accuracy

CM

VM

0.6

0.7

0.8

0.9

1

FS=1 FS=4

MSS = 1

Ac

cu

rac

y

CM

VM

FS=1 FS=4

MSS = 4

12

Experiment 1: Detect Time

CMVM

600

800

1000

1200

1400

1600

FS=1 FS=4

MSS = 1

Ms

ec

CM

VM

FS=1 FS=4

MSS = 4

13

Experiment 1: Summary

• Radar’s manipulations of cognitive load interact with stimulus mapping in ways that parallel Schneider & Shiffrin’s results

• Automaticity develops with extended practice and consistently mapped stimuli even when targets move and time is limited

• Radar task can be used to study automaticity in dynamic stimulus environments

14

• There is some evidence that response mapping is not critical for automaticity to develop (Fisk & Schneider, 1984; Kramer, Strayer, & Buckley, 1991)

• In complex tasks mapping of targets to responses can be inconsistent

o Resulting in large processing costs, even when stimuli are consistently mapped to targets

Experiment 2: Response Consistency

15

Experiment 2: Method

o Only consistently mapped stimuli

o Cognitive Load Variables• Memory Set Size• Frame Size

o Response consistency varied in four levels

16

Mapped to Stimuli Fully Mapped to interface

Partial Mapping to interface Random Mapping

T T

T T

Response Mapping Conditions

17

Experiment 2: Accuracy

0.6

0.7

0.8

0.9

1

Stimulus Full Partial Random

Ac

cu

rac

y

18

Experiment 2 : detect time

600

800

1000

1200

1400

1600

Stimulus Full Partial Random

Ms

ec

19

Experiment 2: Summary

• A consistent response reduces processing requirements

• Total task consistency (both, consistency of stimuli and consistency of responses) matters

o There are processing costs if responses are not consistently mapped, even when stimuli are

• Implicationso Interface design: interface influences processing of

responses• Response selection using track-up vs. north-up displays• Make response selection intuitive• Interface design, decision support tools, training

o We can now systematically manipulate Radar to elucidate the effects of automaticity on high-level dynamic decision-making

20

Experiment 3: Automatic detection & high-level decision making

• How would automatic detection of a component help decision-making?

• Decision-making component required operators to analyze a sensor array of detected aircraft

• Sensor and weapon information changed dynamically

21

Experiment 3: Method

• Sensor Reading Task

• Determine if Target is Hostileo Scan Sensorso > 13 (Hostile)o < 13 (Non-Hostile)

• Press Ignore (5-Key)

• Select Response (Weapon Systems)o Guns vs. Missileso > 10 Missiles (6-Key)o < 10 Guns (4-Key)

• Quiet Airspace Reporto No targets detectedo Click submit report with mouse key

22

Experiment 3: Detect Accuracy

CM

VM

0.6

0.7

0.8

0.9

1

FS=1 FS=4

MSS = 1

Ac

cu

rac

y

CM

VM

FS=1 FS=4

MSS = 4

23

Experiment 3: Decision-making Accuracy

CMVM

0.6

0.7

0.8

0.9

1

FS=1 FS=4

MSS = 1

Ac

cu

rac

y CM

VM

FS=1 FS=4

MSS = 4

24

Experiment 3: Detect Time

CM

VM

FS=1 FS=4

MSS = 4

CM

VM

600

800

1000

1200

1400

1600

FS=1 FS=4

MSS = 1

Ms

ec

25

Experiment 3: Decision-making Time

CMVM

FS=1 FS=4

MSS = 4

CM

VM

600

800

1000

1200

1400

1600

FS=1 FS=4

MSS = 1

Ms

ec

26

Experiment 3: Summary

• Consistent mapping of targets improved he accuracy of the decision-making of the task

• Detect time, detect accuracy, and whole-task performance are sensitive to workload manipulations

• Implicationso Consistent mapping actually improved whole-task

performance by freeing up time for the controlled sensor-reading tasks to run to completion

o Thus, processing speed-up associated with automatic detection can have a large impact on whole-task performance

27

But…?

• Is accuracy of decision-making improved simply because there is more time to process?

• Effect of detection on high-level decision-making in the presence of a dual-task

28

Experiment 3b: Method

• Secondary tone task: enter count of number of non-standard tones

o Calibrated to standard tone at beginning of session for each participant

o Non-standard tones higher/lower pitch than standard

29

Experiment 3b: results

• In fact the Radar task performance was the same with and without the tone task!

• Detect Timeo No Effect of secondary task

• Detect Accuracyo No Effect of secondary task

• Decision-Making Timeo No Effect of secondary task

• Decision-Making Accuracyo No Effect of secondary task

Performance on Tone Task

0.65

0.67

0.69

0.71

0.73

0.75

0.77

0.79

0.81

0.83

0.85

MSS = 1 MSS = 4

Ac

cu

rac

y

CM

VM

30

Experiment 3b: Implications

• No effect of dual task on RADAR performance

• Operators are allocating resources away from tone task to maintain RADAR performance

• Implicationso Finding supports the hypothesis that consistent

mapping improves decision-making performance by freeing up resources for other tasks

o Thus, processing speed-up and low resource requirement associated with consistent mapping can have a large impact on performance in complex task

31

Experiment 4: Categorization

• Since consistent mapping is the search for numbers in letters, it is possible that load-free processing is due to categorization (Cheng, 1985)

• Purpose of this experiment is to establish the presence of load-free processing without categorization

32

Experiment 4: Method

• Incorporate memory ensembles where no possible categorization can take place either a priori or with learning

• CM vs. VM with toneo CM = {C, G, H, M, Q, X, Z, R, S}o VM = {B, D, F, J, K, N, W, P, L}

• Memory ensembles were equatedo Angular {H,M,X,Z,F,K,N,W} vs. Round {C,B,D,G,Q,P,R,J}o Beginning {B,C,D,F,G,H,J,K} vs. End {M,N,P,Q,R,W,X,Z}

• Cognitive Load Variableso Memory Set Size (1 or 4)o Frame Size (1 or 4)

• Indicated detection of target by pressing spacebaro Detect Performanceo Detect Response Time

33

Experiment 4: Detect accuracy

CMVM

0.6

0.7

0.8

0.9

1

FS=1 FS=4

MSS = 1

Ac

cu

rac

y

CM

VM

FS=1 FS=4

MSS = 4

34

Experiment 4: Decision-making accuracy

CM

VM

0.6

0.7

0.8

0.9

1

FS=1 FS=4

MSS = 1

Ac

cu

rac

y

CM

VM

FS=1 FS=4

MSS = 4

35

Experiment 4: Detect time

CM

VM

FS=1 FS=4

MSS = 4

CMVM

600

800

1000

1200

1400

1600

FS=1 FS=4

MSS = 1

Ms

ec

36

Experiment 4: Decision-making time

CMVM

FS=1 FS=4

MSS = 4

CMVM

600

800

1000

1200

1400

1600

FS=1 FS=4

MSS = 1

Ms

ec

37

Experiment 4: Implications

• Varied mapped performance is more sensitive to load than consistently mapped performance

• Individuals performed better in the high-level decision-making component of Radar when stimulus mapping was consistently mapped

• Implicationso Categorization is NOT a necessary requirement

for automaticity developmento Consistent stimulus mapping is a necessary

condition for the development of automatic detection

38

Summary of accomplishments

• Developed Radar, a dynamic simulation where it is possible to study (i.e., to measure) automaticity

• In Radar it is possible to elucidate the effects of automaticity on high-level dynamic decision-making

• Established the usefulness and applications of the dual-process theory of automaticity

• Deepen our understanding of the implications of automaticity development for practical real-world tasks

• Brought together two main theories of automaticity: instance-based theory and dual-process theory

39

Future research

• Consistency of mapping and responding is relative to the categories (i.e., similarity) that a user can form

• Thus, consistent mapping can lead to automatic responses for high-level decision-making after extended practice

40

Looking towards applications

• Test these hypotheses in airport luggage screening

• Decide whether to hand search the luggage

• There is no consistency but rather just similarity (relative to a ‘knife’ category)