PSYC 2220 – HUMAN FACTORS IN DESIGN d e c i s i o n m a k i n g1 of 49 Decision Making 1 Department of Cognitive Science Michael J. Kalsher Human Factors

PSYC 2220 – HUMAN FACTORS IN DESIGN

d e c i s i o n m a k i n g 1 of 49

Decision Making

1

Department of Cognitive Science

Michael J. Kalsher

Human FactorsPSYC 2200

1



What is a Decision-Making Task?One in which:

– a person must select among more than one choice alternative.

– some (but not all) information is available for each alternative.

– the time frame is relatively long (more than a second)

– the best choice is not necessarily obvious (uncertainty and risk present).

2



The 3 Phases of Decision-making

1. Acquiring/perceiving relevant information (decision cues)

2. Assessing the situation to determine how the information we have relates to the decision at hand.

3. Planning and selecting choices (based on perceived costs and

benefits of each choice)

Note: Decision-making and problem-solving often go hand-in-hand -- not easily separated.

Controlled vs. Automatic Decision-making

– Quick/automatic = “Intuitive decision-making”

– Slow/deliberate = “Analytic decision-making”

3



Classical Decision Theory (also termed Rational Decision Theory)

Assumes that if researchers could specify values

for the costs and benefits associated with

different choices, then mathematical models

could be applied to those values, yielding an

optimal choice (i.e., the one that maximizes benefit

and minimizes cost).

4



Major Models of Decision-making

–Normative models• Revolve around the concept of utility, or the overall

value of a choice to the decision-maker.• Prescriptive -- they specify what people ideally

should do.• Do not describe how people actually perform

decision making tasks.

–Descriptive models• Attempt to describe and model actual human

decision-making.5



Normative Decision Models

• Translate the multi-dimensional space of options into a single dimension reflecting the overall utility (or value) of each option.

• Assume the overall value of a decision option is the sum of the magnitude of each attribute multiplied by the utility of each attribute.

6



Multi-attribute Utility Theory

U(v) = a(i)u(i)

U(v) = overall utility of an optiona(i) = magnitude of the option on the ith attributeu(i) = utility (importance) of the ith attribute, n = number of attributes.

n

i = 1

Attributes

A1Price A2Mileage A3Insurance A4Stereo A5Repairs

4 5 2 1 8

01Model 1 3 (12) 3 (15) 9 (18) 3 (3) 1 (8) 56

Options 02Model 2 3 (12) 3 (15) 3 (6) 3 (3) 3 (12) 60

03Model 3 9 (36) 5 (25) 3 (6) 1 (1) 9 (72) 140*

04Model 4 1 (4) 3 (15) 9 (18) 9 (9) 9 (72) 118

•= Highest overall utility among the options available

Utility of each attribute (u)

U(v) or overall utility or value of each option

7



Normative Decision Models Expected Value Theory (addresses outcome uncertainty)

Applies to any decision that involves a “gamble” type decision.– Each choice has one or more outcomes with an associated worth and

probability.

• A .20 probability of winning $50 vs.

• A .60 probability of winning $20

Assumes that overall value of a choice is the sum of the worth of each outcome multiplied by its probability.

– E(v) is the expected value of the choice

– p(i) is the probability of the ith outcome

– v(i) is the value of the ith outcomeE(v) = ∑ p(i)v(i)

i = 1

n

8



Normative Decision Models: Subjective Expected Utility Theory (SEU)

The worth component is considered to be subjective and determined individually for each person.

– Assumes a person will select the action with the highest overall subjective expected utility value.

– SEU useful for studying conditions in which humans make decisions and for developing training and decision aids.

Worth Component

E(v) = ∑ p(i)v(i)i = 1

n

9

E(v) is the expected value of the choice p(i) is the probability of the ith outcomev(i) is the value of the ith outcome



Descriptive Decision Models

• Developed because human decision-making frequently violates key assumptions of

normative models.

• Descriptive models attempt to capture how humans actually make decisions.– People tend to rely on simpler and less-complete means

of selecting among choices termed “heuristics.”

10



Effects of Framing: Prospect Theory

How a problem or situation is “framed” can affect the outcome of choice problems in a way that violates assumptions of rational choice theory (Tversky and Kahneman ,1981).

Gain-frame vs. Loss-frame

Example: If asked to choose between getting $1000 with certainty or having a 50% chance of getting $2500, which would you choose?

• Many people choose the certain $1000 instead of the uncertain chance of getting $2500 even though the mathematical expectation of the uncertain option is $1250. This attitude is described as risk-aversion.

• Research also shows that the same people when confronted with a certain loss of $1000 versus a 50% chance of no loss or a $2500 loss often choose the risky alternative. This is called risk-seeking behavior.

11



Examples of Framing: The Asian Disease Problem

– Participants are asked to "imagine that the U.S. is preparing for the outbreak of an unusual Asian disease, which is expected to kill 600 people.”

– They are then asked to choose between two alternative programs designed to combat the disease.

– The consequences of the choices are posed as probability statements.

12



Effects of Framing: The Asian Disease Problem

One group of participants are presented with a choice between:Program A: "200 people will be saved"Program B: "there is a one-third probability that 600 people will be saved, and a two-thirds probability that no people will be saved"

72 percent preferred program A28 percent preferred program B

• --------------------------------------------------------------------------------------Another group of participants are presented with the choice between:

Program C: "400 people will die"Program D: "there is a one-third probability that nobody will die, and a two-thirds probability that 600 people will die"

22 percent preferred program C 78 percent opting for program D

13



Descriptive Models: Satisficing (Simon, 1957)

Major Assumption People don’t usually meet the goal of making absolutely best/optimal decisions, but instead opt for a choice that is “good enough.”

The rationale Going beyond “good enough” has too little advantage to be

worth the effort.

Does it work?- Reasonable approach given that people have limited

cognitive capacities and limited time.- Sampling procedures are key to its success.

14



Decision Models: A Summary

• Careful analysis of choices and their respective utilities is desirable if:– time is unlimited.– the amount of available information is limited.

• Given limited time, too much information, and/or stress, people tend to shift to simplifying heuristics.

• Research shows that people can shift between analytical and heuristic decision-making as circumstances dictate.

15



Heuristics and Biases: What are they?

• Cognitive heuristics are usually very powerful and efficient decision tools, but their use does not guarantee the best solution.

• Because they are simplifications, heuristics sometimes lead to biases or misperceptions.

16



Heuristics and Biases: Information-Processing Limitations in Decision Making

Figure 7.1 An Info Processing Model of Decision-making. The model highlights several cognitive limits to effortful decision making that lead us to rely on heuristics, including: selective attention, working memory, and retrieval from LTM. 17



Information Processing: Factors and

Limitations that Influence Decision-Making Quality

• Amount and quality of information brought into WM (e.g., workload; attentional resources)

• Working memory capacity limitations• Amount of time available for decision making (e.g., medical

emergency; system failure).

• Amount and quality of knowledge a person holds in LTM relevant to activity (“knowledge-in-the-head”).

• Ability to retrieve relevant information, hypotheses or actions from LTM at the critical moment (Problem of inert knowledge)

People have the most difficulty with decisions made with too little or erroneous information, extreme time stress, high cognitive workload, changing dynamic informational cues, conflicting goals, and novel/unusual circumstances--factors common in high-risk environments.

18



Heuristics and Biases: Information Processing Limits in Decision Making

• Cue Reception and Integration– Relevant cues (pieces of info) are retrieved from environment and go into Working

Memory. Limits on the number of cues that can be considered.

• Hypothesis Generation, Evaluation and Selection– Decision-makers make educated guesses as to the cues’ meaning.– Meaning is derived by retrieving information from LTM and comparing it to the cues;

hypotheses are brought into WM and evaluated with respect to how likely they are to be correct.

– Revise or generate a new hypothesis.– Chosen hypothesis serves as the basis for course of action.

• Generating and Selecting Actions– One or more possible actions generated in WM by retrieving possibilities from LTM.– Action selection is achieved by evaluating possible outcomes, likelihood of each

outcome, and the positive/negative factors associated with each.

19



Heuristics and Biases: Heuristics and Biases in Receiving and Using Cues

– Attention to a limited number of cues (limited by constraints on working memory).

– Cue Primacy and Anchoring (first impressions are lasting).

– Inattention to later cues (cues occurring later in time or ones that change over time are ignored; attributable to attentional factors).

– Cue Salience (Loudest, brightest cues are more likely to attract attention and are given more weight. The most salient cues aren’t always the most diagnostic ones).

– Overweighting of unreliable cues (relative to more reliable information).

20



Heuristics and Biases: Heuristics and Biases in Hypothesis Generation

After a set of cues are processed in WM, decision-makers generate hypotheses by retrieving one or more from LTM: The following heuristics/biases affect this process:

1. Generation of a limited number of hypotheses– People generate only a small subset of hypotheses (1-4) due to WM constraints and

never consider all relevant ones. Stress exacerbates the problem.– The first option considered by experts is likely to be reasonable, but not for novices.

2. Availability heuristic– People more easily retrieve hypotheses that have been considered recently or that

have been considered frequently; If something comes to mind easily, people assume it is relatively common and therefore a good hypothesis

3. Representativeness heuristic– Tendency to judge an event as likely if it represents features typical of its category.

4. Overconfidence– People believe that they are more correct than they actually are; less likely to seek

out evidence for alternative hypotheses or prepare for the possibility they are wrong.

21



Heuristics and Biases: Heuristics and

Biases in Hypothesis Evaluation and Selection

Once the hypotheses have been brought into WM, additional cues are sought to evaluate them. The process of considering additional cue information is affected by the following cognitive limitations:

1. Cognitive Tunneling (functional fixedness; mental set).– People tend to adopt and fixate on a single hypothesis, assume that it is true,

and then proceed with a solution consistent with the hypothesis. – Fail to utilize subsequent cues.

2. Confirmation Bias– People have a hypothesis they are trying to evaluate and seek only

confirming information in evaluating the hypothesis.

These limitations are exacerbated by high stress & mental workload

22



Heuristics and Biases in Action Selection

1. Retrieve only a small number of actions– Limited number of plans can actually be retrieved and kept in

WM

2. Availability heuristic for actions– People retrieve most “available” actions from LTM– Availability of items in LTM is a function of recency, frequency,

and how strongly they are associated with hypothesis

3. Availability of possible outcomes– When more than one action is retrieved, must select based on

how well action will yield desirable outcomes– Evident when people make decisions and fail to foresee

outcomes that are readily apparent in hindsight

4. Framing bias.-- Decisions affected by the way the situation is presented.

23



Framing Bias: Additional Examples

1. People are asked the price they would pay for a pound of ground meat that is 10 percent fat or a pound that is 90 percent lean. People tend to pay 8.2 cents per pound more for the option presented as 90 percent lean.

2. Students told either that they answered 80 percent of the questions on the exam correctly or that they answered 20 percent of the questions incorrectly.Students more likely to feel they are performing better if they are told the former.

3. People are told there is a 20 percent mortality rate associated with a particular treatment or they are told there is an 80 percent chance the treatment will save their life. People less likely to choose the treatment when expressed in terms of mortality.

Sunk Cost Bias: The tendency to choose the riskier of two options when the framed in terms of loss. (Hesitation to sell losing stocks, but willingness to sell winning stocks to lock in a gain). Decisions should be framed in terms of gains to counteract risk-seeking tendencies.

24



Naturalistic Decision Making: Making decisions in the real world

Real world decision making tasks tend to have characteristics such as:– Ill-structured problems– Uncertain, dynamic environments– Information-rich environments in which situational cues

change rapidly– Cognitive processing that proceeds in iterative

action/feedback loops– Multiple shifting and/or competing individual and

organizational goals– High risk– Time constraints & stress– Many people involved in the decision

25



The Decision Making Context

Everyday decision-making is characterized by cognitive complexity– Multitude of factors affect everyday decisions– Heuristics are accurate much of the time, but depends on

people having the appropriate information resources and an ability to adapt them.

Traditional decision making research and more recent naturalistic decision making are complementary--not mutually exclusive– Heuristics and biases discovered in laboratory research have

been validated in the “real world.”26



Research on Decision-Making in Complex Environments: Skill, Rule, Knowledge

Rasmussen’s SRK model describes 3 levels of cognitive control that might be used during task performance: Skill-based behavior, Rule-based behavior, & Knowledge-based behavior 27



Rasmussen’s SRK Model: Skill-Based Behavior

• If extremely experienced with task, information is processed at the “skill-based level.”– React to perceptual elements at automatic or subconscious

level.– Performance governed by stimulus-response associations

developed at the neurological level.

• Errors in skill-based behavior usually caused by:– Misdirected attention– Paying too close attention to the task, which then may interrupt

an automated sequence of behavior (scripts).

28



• If familiar with task, but limited experience, information processed at the “rule-based level.”– Meaningful cues (signs) can trigger rules accumulated from

past experience.– Rules are If-Then associations between cue sets and the

appropriate actions.

• Errors tend to result from misclassification of the situation and subsequent application of the wrong rule.

Rasmussen’s SRK Model: Rule-Based Behavior

29



• If situation is novel, person operates on knowledge-based level.– Analytical processing using conceptual information.– Person assigns meaning to cues and integrates this information

into a coherent “story” that describes what is happening.– Information is processed with respect to goals in WM.

• Errors at knowledge-based level tend to result from factors associated with analytical thinking – Limited WM– Biases in generating hypotheses/actions– Cognitive fixation– Incorrect mental models

Rasmussen’s SRK Model: Knowledge-Based Behavior

30



Cognitive Continuum Theory• Decision making process assumed to occur along a

continuum from intuition to analysis– Intuitive process—characterized by low levels of control and conscious

awareness, rapid processing, and high confidence in the answer.– Analytical process—characterized by higher levels of cognitive control, slow

processing and lower confidence in answer.

• Use of Intuitive vs. Analytical processes is determined by the nature of the task:

• Intuitive processing induced by tasks having a large number of cues, simultaneous and brief display of cues, strong relationships among the cues, and short time-frame for the decision.

• Analytical processing induced by tasks having fewer cues, high confidence in the task, and long sequential availability of cues.

– Failure in the use of one type causes switching to the other.

31



Situation Awareness• Perception of the elements in the environment within

a volume of time and space, the comprehension of their meaning and the projection of their status in the near future (Micah Endsley, 1988).

• Levels of situation awareness (SA) and cognitive complexity:– Level I: Perceiving the status, attributes, and

dynamics of relevant elements in the environment– Level II: Comprehending relevant cues in light of one’s

goals– Level III: Projecting the future activity of the elements in the

environment

32



Situation Awareness: Cognitive Processing Requirements

– The integration of cues into complex mental representations of a system accomplished by using pre-existing knowledge to interpret and give meaning to cues.

– SA may also require evaluation of factors such as risk and time available for decision.

In times of high mental workload and stress, people seem to “lose” situation awareness.

33



Recognition-Primed Decision Making: Studying Experts Making Decisions Under Time Stress

• According to Klein (1989), in most instances, experts simply recognize a pattern of cues and recall a single course of action which is then implemented.

• 3 Assumptions of RPDM:– People use experience to generate a plausible option the first

time around.– If the decision makers are experts, time pressure should not

cripple performance because of rapid pattern matching.– Experienced decision makers can adopt a course of action

without comparing and contrasting possible courses of action.

34



Schemas, Stories & Mental Models• Schemas

People use previous knowledge to comprehend and integrate the situational cues into a dynamic model of situations they are trying to evaluate.

• Explanation-based decision making (Pennington & Hastie, 1988; 1993)

– Consists of 3 activities• Receiving info & constructing a causal story that can account for

the information• Generating possible actions • Determining actions that best fit the story via a matching process

– Constructing a causal explanation is pivotal!

35



Mental ModelsWhen applied to decision-making, suggests that people construct mental models of the relevant system or environment and use it to run simulations throughout the decision-making process.

Simulation is used to:1. generate expectations for other cues not previously considered.2. guide observation of changes in system variables.3. evaluate goals, actions & plans and to make predictions useful in monitoring actions & consequences in the system or environment

36



Integrated Model: Adaptive Decision Making

37



Integrated Model

• Information enters system and is processed at one of three levels:– Automatic skill-based– Intuitive rule-based– Analytical knowledge-based

• If situation is difficult or complex and time allows, decision maker utilizes more complex evaluative processes

38



Improving Human Decision Making

• “Human Error” focuses on cognitive errors (poor decision making) rather than behavioral errors (hand slipped)

• Possibilities for improvement– Redesign for performance support– Training– Decision Aids

39



Redesign for Performance Support

Improve the quality of information provided by the external environment (e.g., focus on improving the quality of “knowledge-in-the-world”).

Doing so supports better decision making, thereby eliminating the need to change the person making the decisions

40



Training• Train people to overcome heuristics/biases

– Focus on counteracting specific types of biases– Allow natural use of varying strategies, but teach people when to

use them and the shortcoming of each

• Highlight the value of metacognition by training people to:– Recognize and use appropriate/adequate cues that facilitate

situation awareness– Check situation assessments or explanations for completeness

and consistency with cues– Analyze data that conflict with situation assessment– Recognize when too much conflict exists between the

explanation or assessments and the cues

41



Training

• At intuition rule-based level, provide training to enhance perceptual and pattern recognition skills

• Focus on situation assessment– Trainees learn to recognize critical situational cues

& to improve accuracy of their time available & risk judgments

• At automatic level, focus relevant cues in raw data form– Works only for situations where a cue set

consistently maps onto a particular action

42



Training

• Decision maker should receive feedback, preferably for each cognitive step

• Training does not overcome memory limitation problems

• Large amount of knowledge may remain inert and unretrieved

43



Decision Aids

Decision tables– Used to list possible outcomes, probabilities &

utilities of action alternatives– Deflects the load placed on WM– Similar to a decision tree used for representing

decisions that involve sequence of decisions & possible outcomes

• Branching point used for possible consequences & associated probabilities

44



Decision Aids

Expert systems– Computer programs designed to capture one or more

experts’ knowledge and provide answers in a consulting type role

– Developed to help with wide variety of tasks– Take situational cues as input and provide either a

diagnosis or suggested action as output– Have not yet been successful in complex decision

environments• Limited ability to collaborate/communicate with expert system

45



Cognitive Support: Decision Support Systems

– Designed to improve decision making by extending the user’s cognitive decision making abilities

• e.g., utilize computers to support WM and perform calculations

– Front-end analysis of the task is critical to determine what information should be provided or what calculations/modeling needs to be performed

– Usability testing is also needed, especially for advanced features

– Success depends upon:• Users’ ability to control and/or redirect the subsystem• The extent to which the user/subsystem have common or

shared representations of the state and problem status

46



Problem Solving: Characteristics

• Occurs when the “problem-solver” does not have the method to do a task stored in memory.

• Problem solving is difficult because:

– Limitations on WM; Lack of sufficient relevant system knowledge to solve problem; Person has sufficient system knowledge, but is disconnected/disorganized and cannot access it from LTM

• Solution path usually a set of “subroutines” that are combined to solve the problem.

• Knowledge based decision-making shares similar cognitive processes as problem solving.

47



Problem Solving: Requirements

• Relies extensively on generating actions and planning.

• Requires large amounts of relevant knowledge, good strategies for generating solutions, and effective mental models to offset memory limitations, lack of knowledge in LTM and a lack of good memory retrieval strategies.

48



Problem Solving: Errors and Biases

• First type of difficulty caused by the way in which people represent the problem- If overly constrained, omits constraints, or only

allows one view of things, a solution will be less likely to be generated

• Failure to generate correct solution plan- Due to fixation on previous plans that worked in

the past- Prone to functional fixedness

49



Errors in Problem Solving

• Failure to develop solution caused by limitations of WM– Often a long sequence of action “packets”

must be composed into a “plan”– Cognitive simulation must be carried out to

evaluate the plan• Frequently involves too many bits of info to be

handled in working memory

50

Documents

PSYC 2220 – HUMAN FACTORS IN DESIGN d e c i s i o n m a k i n g1 of 49 Decision Making 1 Department of Cognitive Science Michael J. Kalsher Human Factors