The Influence of Fatigue on Human The Influence of Fatigue on Human Performance: Performance: A MetaA Meta--Analysis of Sleep Analysis of Sleep Deprivation on Performance Measures Deprivation on Performance Measures

(Reaction Time and Accuracy).(Reaction Time and Accuracy).

Candi Griffith and Candi Griffith and SankaranSankaran MahadevanMahadevan

August 27, 2007August 27, 2007

DisclaimerDisclaimer

►►The contents of this research do not The contents of this research do not necessarily reflect the views or policies of necessarily reflect the views or policies of any government office or agency.any government office or agency.

Outline for TodayOutline for Today

►► What is MetaWhat is Meta--Analysis (MA)Analysis (MA)►► Application of MetaApplication of Meta--Analysis Analysis

to fatigue literatureto fatigue literature►► ResultsResults►► Next stepsNext steps

My Research TopicMy Research Topic

►► Improve stateImprove state--ofof--art of HRA art of HRA (Human Reliability Analysis) (Human Reliability Analysis)

►► Establish framework for psychological data input Establish framework for psychological data input to HRAto HRA

►► Conduct a MetaConduct a Meta--Analysis to determine fatigue Analysis to determine fatigue effects on performanceeffects on performance

►► Develop methodology for effect size information to Develop methodology for effect size information to be used in HRAbe used in HRA

What is MetaWhat is Meta--analysis?analysis?

►► What it is: What it is: ►► Synthesis of available Synthesis of available

literature about a topicliterature about a topic►► Structured format to Structured format to

extract information from extract information from selected studiesselected studies

►► Compiles data for overall Compiles data for overall effecteffect

►► 11stst done by Smith and done by Smith and Glass in 1977Glass in 1977

►► Idea introduced by Karl Idea introduced by Karl PearsonPearson

►► What it is not: What it is not: ♣♣ A vote counting methodA vote counting method♣♣ Literature reviewLiterature review♣♣ Primary or Secondary Primary or Secondary

analysisanalysis

How MA works on literatureHow MA works on literature

Fatigue Literature

Info used in HRA

Meta-

Analysis

Psych Literature

Medical Literature

Military Literature

5 Step Plan for Meta5 Step Plan for Meta--analysisanalysis

►►Formulate the problemFormulate the problem►►Collect the dataCollect the data►►Evaluate the data (coding)Evaluate the data (coding)►►Analyze the data (calculate Effect size)Analyze the data (calculate Effect size)►►Report the findingsReport the findings

The Problem:The Problem: Step 1 of 5Step 1 of 5

►► The hypothesis:The hypothesis:♣♣ Fatigue degrades human performance.Fatigue degrades human performance.

►► To what extent is performance decremented?To what extent is performance decremented?►► Simplified Model of FatigueSimplified Model of Fatigue

♣♣ 1 input Sleep1 input Sleep--DeprivationDeprivation♣♣ 1 output Reaction Time (speed and accuracy)1 output Reaction Time (speed and accuracy)

Reaction Time

Fatigue Sleep Deprivation

Data CollectionData Collection Step 2 of 5Step 2 of 5

Invisible College

Ancestry Approach

Descendent Approach

Computer Databases

Hand Search

Search Strategy

Study Criteria

Empirical Findings

Published in English

Post WWII - modern

Quality Study

Published Study

Data EvaluationData Evaluation Step 3 of 5Step 3 of 5

►►Develop coding manualDevelop coding manual♣♣Study descriptorsStudy descriptors♣♣Effect size (statistical info.)Effect size (statistical info.)♣♣Determine what information needs to be Determine what information needs to be

recordedrecorded

►►Apply to coding manual to the selected Apply to coding manual to the selected studiesstudies

Analyze DataAnalyze Data Step 4 of 5Step 4 of 5

►► Derive Effect Size from: Derive Effect Size from: ♣♣ Significance dataSignificance data♣♣ MeansMeans♣♣ Sample sizesSample sizes

►► Effect Size metrics:Effect Size metrics:♣♣ dd--indexindex♣♣ Other ES representations:Other ES representations:

►►OddsOdds--ratioratio►►Percentage of variance (PV)Percentage of variance (PV)►► rr--index index –– 2 continuous variables2 continuous variables

What is an Effect Size?What is an Effect Size?

►►Term often used psychological and Term often used psychological and biological studiesbiological studies

►►Describes the degree of change from one Describes the degree of change from one state to anotherstate to another

►►Standardized Mean Effect SizeStandardized Mean Effect Size♣♣The degree of change in the mean performance The degree of change in the mean performance

levellevel♣♣Nominal performance vs. test conditionNominal performance vs. test condition

Report FindingsReport Findings Step 5 of 5Step 5 of 5

►►Inform HRA in generalInform HRA in general►►HERA HERA -- should be able to input data from should be able to input data from

research coding into HERAresearch coding into HERA►►SPARSPAR--H H PSFsPSFs category selectioncategory selection

Fatigue: The Little Black Box on Fatigue: The Little Black Box on PerformancePerformance

FatigueNormal

PerformancePerformance after Fatigue

►► What are the effects?What are the effects?►► What Magnitude?What Magnitude?►► What direction?What direction?

Application of MetaApplication of Meta--Analysis to Analysis to Fatigue literatureFatigue literature►►19 studies used for Effect Size calculation19 studies used for Effect Size calculation

♣♣31 data points31 data points►►Multiple measures of reaction timeMultiple measures of reaction time

♣♣Speed: 15 points from 12 studiesSpeed: 15 points from 12 studies♣♣Accuracy: 16 points from 13 studiesAccuracy: 16 points from 13 studies

►►% correct% correct►►# misses# misses

►►Multiple disciplinesMultiple disciplines♣♣Medical, psychological, military, etc.Medical, psychological, military, etc.

Effect Size Calculation Equations:Effect Size Calculation Equations:

pooledsXXES )( 21 −=

►►Effect Size for Effect Size for sample meanssample means

1))1()1(

21

22

21 21

−+

−+−=

nnsnsn

spooled

►►Effect Size for ChiEffect Size for Chi--SquaredSquared 2

2

2χ

χ−

=N

ES

Calculate Effect Size for Calculate Effect Size for tt--test & test & F F ––value:value:

21

21 )(nn

nnFES +=

21

21 )(nnnntES +

= ►►Effect Size for Effect Size for tt --testtest

►►Effect Size for Effect Size for FF --valuevalue

Results summaryResults summary

Summary Overall ES ES (speed) ES (accuracy)

N for ES 31 15 16

Mean -1.5165 -1.9706 -1.0625

# +, # - 3+, 28- 0+, 15- 3+, 13-

σ 1.527 1.641 1.316

95% CIN [-2.38, -0.65] [-2.89, -1.04] [-1.81, -.32]

Range (min, max) (-6.08, -0.37) (-3.7, +0.58) (-6.08, +.576)

Stand Error of ES 0.3116 0.4737 0.3798

What We Did See What We Did See ……

►► The application of metaThe application of meta--analysis to fatigue analysis to fatigue literature for HRA improvements is feasible literature for HRA improvements is feasible

►► Significant negative relationship between Reaction Significant negative relationship between Reaction Time and Sleep Deprivation on mean performanceTime and Sleep Deprivation on mean performance

--1.5165 drop in means1.5165 drop in means

Time

Perf

orm

ance

: Rea

ctio

n Ti

me

Fatigue

What comes next?What comes next?

►►Conduct multiple small metaConduct multiple small meta--analysis on analysis on specific variablesspecific variables

►►More inputsMore inputs♣♣Sleep Deprivation and/orSleep Deprivation and/or♣♣Circadian Rhythm influence and/orCircadian Rhythm influence and/or♣♣Hours workedHours worked

►►Develop method for Effect Size results to be Develop method for Effect Size results to be useful for HRAuseful for HRA