Design and Evaluation Human Factors Psychology Dr. Steve

Design and Design and EvaluationEvaluation

Human Factors PsychologyHuman Factors Psychology

Dr. SteveDr. Steve

Human Factors Design Human Factors Design ActivitiesActivities

Design product/system interfacesDesign product/system interfaces Modify designs/correct design Modify designs/correct design

flawsflaws Safety activities (warnings, Safety activities (warnings,

analyses)analyses) Develop training Develop training

programs/materialsprograms/materials

Cost/Benefit AnalysisCost/Benefit Analysis

Costs• HF personnel salaries • Extended time to product release• Analysis tools (software, video)• Prototyping • Increased development cost

Benefits• More effective product• Safer product• Cheaper to change design early in development• Higher sales• Fewer liability suits

What other costs and benefits of HF activity in product/system design can you think of?

System Development and System Development and HF ActivitiesHF Activities

• Stage 1. Front-End Analysis*• user analysis• function analysis• preliminary task analysis• environmental analysis• system specifications

• Stage 2. Conceptual Design*• function allocation• support conceptual design

• Stage 3. Iterative Design and Testing*• task analysis• interface design & prototyping• heuristic evaluation• cost/benefit analysis of alternatives• workload analysis (simulation & modeling)• safety analysis• usability testing

• Stage 4. Design of Support Materials• develop manuals

• Stage 5. System Production• Stage 6. Implementation and Evaluation

• experiments to evaluate system• Stage 7. System Operation/Maintenance

• monitor system performance• Stage 8. System Disposal

* Topics covered in class

User-Centered DesignUser-Centered Design

1. Early focus on the user and tasks2. Empirical measurements

• questionnaires, usability studies, data collection3. Iterative design

• prototyping of interface4. Participatory design

• user input to design, preferences, SLUGs

Sources for Design DataSources for Design Data

• Data Compendiums•Engineering Data Compendium: Human Perception and Performance

• Human Factors Design Standards• MIL-STD-1472D (military anthropometry, ergonomic specs)• ANSI/HFES-100 (VDT specs)• ANSI/HFES-200 (software specs)• OSHA standards

• General HF Principles/Guidelines• Textbooks (by topic: interface design, aviation, software)• Research Journals (Human Factors, Ergonomics, Applied Ergonomics)• Handbooks (Handbook of Human Factors)• Conferences (CHI, HFES, IEA)• Workshops (U of Mich)

Front-End AnalysisFront-End AnalysisAnswers the following questions:

1. Who are the users?• Consumers, trainers, maintainers, etc.

2. What functions are performed by system?• Allocate functions to machine or person

3. What are the environmental conditions where system will be used?• Noise, temperature, pressures, etc.

4. What are users’ preferences/requirements?• Color, functions, shape, etc.

5. What are constraints to design of the system?• Space, regulations, costs, resources

6. What are the HF criteria for design solutions?• Shorter training time, less accidents, higher performance

Front-End AnalysisFront-End AnalysisUser AnalysisUser Analysis

Who are the intended users?• Age• Experience level• Intellectual level• Anthropometrics (size & shape)• Strength• Culture & Language• Disabilities (sight, hearing, mobility)

Why does the drive- thru ATM have Braille on its controls? What are the characteristics of DVD player consumers?

Front-End AnalysisFront-End AnalysisFunction/Preliminary Task Function/Preliminary Task

AnalysisAnalysis

Function Analysis: What are the basic functions of the system?Example: The function of a wheel barrow is to move objects to the work site.

Preliminary Task Analysis: What are the jobs, duties, tasks, and actions that user will be performing?Example: Tasks to be performed with a wheel barrow include loading it with dirt/building materials, transporting, mixing, and dumping materials, cleaning, maintaining, and storing wheelbarrow.

Front-End AnalysisFront-End Analysis Preliminary Task AnalysisPreliminary Task Analysis

Preliminary Task Analysis Methods:• Interviews• Focus Groups• Observation• Questionnaires

Used prior to design to learn how intended users perform their jobs.

What method(s) would you use if you were designing a new library cataloguing system?

Front-End AnalysisFront-End AnalysisEnvironment AnalysisEnvironment Analysis

Where will the tasks be completed and under what conditions?• Indoors/outdoors• Temperature/humidity• Lighting levels• Noise levels• Vibration• Gravity• Special clothing/gear requirements

• How would the design of a radio transmitter differ if the intended users were researchers in Antarctica?• Would a standard vending machine work on the space station? If not, how would you design a vending machine for that purpose?

Front-End AnalysisFront-End AnalysisUser Preferences & Req’sUser Preferences & Req’s

When different design options are available, must determine the intended users preferences and needs (perhaps determined through focus groups).

User preference issues include:• What components are automated• Color features/aesthetics• Balance between performance and preference• What makes other products popular/unpopular• What can be made adjustable and what cannot• Feasibility of user’s demands

Conceptual Design Conceptual Design ActivitiesActivities

Function AllocationFunction Allocation

Things Humans Do Well Ability to:• detect small amounts of visual or acoustic energy• perceive patterns of light or sound• improvise and use flexible procedures• store very large amounts of information for long periods and to recall relevant facts at the appropriate time• reason inductively• exercise judgment

Things Machines Do WellAbility to:• respond quickly to control signals, and to apply great force smoothly and precisely• perform repetitive, routine tasks• store information briefly and then to erase it completely• reason deductively, including computational ability• handle highly complex operations, I.e., to do many different things at once

Fitt’s List (1951)

Function Allocation: the assigning of tasks to humans or machine taking into account human vs. machine capabilities and limitations

Is Fitt’s list still appropriate 50+ years later?

Conceptual Design ActivitiesConceptual Design ActivitiesDecision/Action & Functional Flow Decision/Action & Functional Flow

AnalysesAnalysesDecision/Action and Functional Flow Analyses: Procedures for identifying and depicting the sequence of functions, actions, and decisions made by the system (user included as part of system).

Example: Programming your VCR to record a show Thursday night at 7:30 on Fox Network

Press “Program”

“use up & downarrows to

indicate selection”

Accessinputmenu

Determine When show

airs

Hit arrowto indicate

day

VCR acceptsInput, moves to next field

Press“Enter” button


time

VCR acceptsInput, moves to next field

Press “Enter” button


channel

VCR acceptsInput

Press“Enter” button

“Program completeSelect another

program or pressEnd to finish”

Yes or No

Press“End”

Yes

No

(See homework assignment)

Conceptual Design Conceptual Design ActivitiesActivities

Typical Design MeetingTypical Design Meeting

BrainstormingStory-Boarding

Iterative Design and Iterative Design and TestingTesting


Task Analysis (detailed)Task Analysis (detailed)

Purpose of Detailed Task Analysis is to identify:• User goals and activities• Tasks required to achieve goals• Conditions under which task is performed• Expected outcomes of tasks• KSA’s needed to perform tasks• Equipment needed to perform tasks• Complex decision making, problem solving, diagnoses*• Complex rule structures dependent on situation*

* Part of Cognitive Task Analysis

Task Analysis MethodsTask Analysis MethodsInterviewsInterviews

Interviews: Questioning subject matter experts to get them to explain what they do on the job.

Tries to answer questions such as:• How do you perform the task?• Why do you perform the task?• Under what conditions do you perform the task?• What happens before and after performing the task?• What are the consequences of performing or not performing the task?

Advantage: simple to useDisadvantage: SMEs may not be good at describing tasks

Task Analysis MethodsTask Analysis MethodsObservationObservation

Observation: Watch and take notes of people performing the task

Suggestions: • Be unobtrusive• Use video if possible and with approval• Review observations with SMEs

Advantages: See what workers do, not what they say they do.Disadvantages: Does not capture cognitive processes.

Task Analysis MethodsTask Analysis MethodsThink-Aloud Verbal ProtocolThink-Aloud Verbal Protocol

Think, think, think!

Think-Aloud Verbal Protocol: Worker state what they are thinking as they perform the task

• Concurrent – verbalize as performing the task• Retrospective – verbalize what they were doing as they view a videotape of themselves• Prospective – verbalize as they imagine performing a hypothetical task

Advantage: Can get info regarding thought processes and decisions of experts performing the task.

Disadvantage: Doing concurrently can interfere with the task, and doing retrospectively relies too heavily on memory

Task Analysis MethodsTask Analysis MethodsHierarchical Task AnalysisHierarchical Task Analysis

Hierarchical Task Analysis: Graphical (chart or diagram) depiction of task analysis

Advantage: Allows to conceptualize task sequencesDisadvantage: May not accurately represent true parallel nature of task

Example: Table 3.5 from Wickens, Gordon, & LiuHierarchical task analysis for using a lawnmowerStep 1. Examine Lawn a. Make sure grass is dry

b. Look for any objects laying in the grassStep 2. Inspect Lawnmower

a. Check components for tightness1. Make sure grass bag handle is securely fastened to grass bag support2. Make sure grass bag connector is securely fastened to bag adapter3. Make sure the deck cover is in place4. Check for any loose parts (such as oil cap) and blade


Link AnalysisLink Analysis

Link Analysis: Shows the relationships between components of a system (e.g., communication, control, movements)

For determining:• layout of related workstations• placement of controls• communication patterns• frequency of equipment use

Advantage: Graphic depiction of relationships among system featuresDisadvantage: Shows frequency of use, not importance, and may vary by situation

Link Analysis of CIC of USS Louisville


Flow DiagramsFlow Diagrams

Operational Sequence Diagram: Graphically depicts the sequence and timing of activity, and shows the interactions between individuals and equipment

Advantage: Connects actions to time and componentDisadvantage: Does not lend itself easily to interface design

Diagram for a collision avoidance system


Interface Design PrinciplesInterface Design Principles

1. Provide a good conceptual model2. Make things visible3. Use natural mappings4. Provide feedback



Provide a good conceptual model: when things are designed to be consistent with our mental models of how they should work (expectancies) then it is easier to predict what will happen when we use them.

In this metronome, the tempos are arranged around a circle with a knob and arrow in the middle. Most people would expect to twist the knob until the arrow points to the desired tempo. Actually the arrow is a button that you push if you want the tempo to increase and the knob is not a knob at all.



Make things visible: It is easier to operate a system when the features are clearly visible

Starting this gas pump is more difficult than it sounds because the start button is hidden among the other sea of information and does not “pop out” at you as a start button either figuratively or literally. Additionally, the use of red usually signals something other than start.



Use natural mappings:Actions should correspond to

the system’s state.To make something go up,

the action should be up, to make it go right, the action should be to the right

This stovetop makes good use of natural mappings. The burners are arranged in the same pattern as the controls.



Provide feedback: Feedback is important because it allows one to see whether

the intended actions have taken place. (like hourglass icon on computer software)

(read “Set Phasers to Stun” in Casey book for another example).

In this example the feedback is confusing. The light on the bottom signifies that the smaller quantity of coffee was requested. The light does not go on when the larger quantity is requested.



Other Design Principles (Norman, 1992):• Simplify the structure of tasks• Make alternative actions and knowledge of results visible• Make it easy to determine what actions are possible• Make it easy to evaluate current system state• Exploit the power of constraints (remove possibility of error)• Design for easy error recovery (undo)• When all else fails, standardize.


PrototypingPrototyping

Prototypes or Mockups:Used to test features of the interface prior to production - may be anything from crude cardboard cut-outs to realistic models

Example of Rapid Prototyping Software used to model different phone interfaces


Evaluative StudiesEvaluative Studies

Types of evaluative studies:• Cost/Benefit Analysis for Design Alternatives• Trade-Off Analyses• Safety Analysis• Workload Analysis• Simulation & Modeling

Mathematical Modeling

Evaluative Studies Evaluative Studies Simulation & ModelingSimulation & Modeling

CognitiveModeling

Biomechanical Modeling

Modeling programs:•MicroSAINT•Mannequin•COMBIMAN•HUMANCAD

Evaluative Studies Evaluative Studies Usability TestingUsability Testing

Usability Testing: Ease of use (user-friendly)

May test for:• Learnability• Efficiency• Memorability• Errors• Satisfaction

Final Test and EvaluationFinal Test and EvaluationField StudiesField Studies

Design: between vs. within subjectsTest Participants: user populationProximal Measures:

• satisfaction• usability• performance• errors

Distal Measures:• manufacturing costs• personnel costs• number of accidents/disability claims

Alternatives to HF Design and Alternatives to HF Design and DevelopmentDevelopment

TrainingTo use system capabilitiesMethods to make task safer

Modifying Current EquipmentEquipment ReplacementErgonomic Aids

Wrist padsFoot stoolsAnti-fatigue mats

Job Rotation (and other admin approaches)Vary tasksRest periods

Cow Factors in DesignCow Factors in Design

This is what happens when you fail to consider the user in the design

Documents

Design and Evaluation Human Factors Psychology Dr. Steve