Experimental Design

John VanMeter, Ph.D.

Center for Functional and Molecular ImagingGeorgetown University Medical Center

Development of an fMRI Experiment

Independent and Dependent Variables

• Independent variables are the parameters that are controlled by the experimenter

• Dependent variables are the data measured by the experiment

• One or more independent variables is manipulated in an experiment the effect of which will be measured by the dependent variables

• In most fMRI studies the dependent variable is the change in the fMRI signal

Types of Conditions

• Two basic types of conditions are used in fMRI: – Experimental condition is the condition or

task of interest– Control condition is the task that is

subtracted from the experimental condition– Recall that BOLD contrast is non-

quantitative

Possible Control Conditions for a Face Processing Study

Confounding Factors

• Control condition should in general match the experimental condition as much as possible

• Confounding factor is any parameter that varies with the independent variable

• Selection of a good control condition is important to getting meaningful results

Alcohol Example

• Suppose one found that there was a decrease in fMRI activation for a motor task when subjects drank alcohol as opposed to water

• Possible conclusion is that alcohol reduces neuronal activity

• However, should consider other possibilities such as whether the effect of alcohol caused these subject to perform the motor task at the wrong times or less frequently

Subtraction Method

• Basic analysis is based on comparing fMRI signal between two conditions

• Assumption is that cognitive process of interest is the only difference between the two conditions

Petersen, et al., 1988

Pure Insertion Assumption

• Insertion of a single cognitive process does affect any other processes

• Interactions between two cognitive processes would invalidate subtraction analysis

• Violation of Pure Insertion would mean results uninterruptible

Example of Failure of Pure Insertion Assumption

• Comparison of semantic and letter judgment tasks using three different modalities: mouse, vocal, and covert (silent/mental)

• Interaction between modality and task in left prefrontal cortex

• Cannot distinguish whether change due to modality or task

Jennings, et al., 1997

Analysis and Pure Insertion Assumption

• Subtraction analysis assumes pure insertion holds - baseline/control task does not engage any other processes

• Example– Subtraction of word naming from verb

generation– Word naming does not require semantic

processes– What if this control condition automatically

engages these processes anyways

Main Design Models

• Common Baseline• Parallel Comparisons• Tailored Baselines• Hierarchical• Parametric• Selective Attention• Adaptation

Common Baseline

• Comparison of two experimental conditions to same control– Ex A > Ctrl– Ex B > Ctrl

• Detects areas common to both conditions

• Assumes both experimental conditions have similar psychometric properties (ie, task difficulty, equivalent degree of activation across subjects)

Parallel Comparisons

• Compare both experimental tasks to each other (seeing vs hearing words)– Ex A > Ex B– Ex B > Ex A

• Compliments Common Baseline• Assumes similar psychometric

properties in both A and B

Tailored Baseline

• Use different control tasks unique to each experimental condition– Ex A > Ctrl A– Ex B > Ctrl B– Example:

• visual display of words vs. false font text• hearing words vs.reverse speech

• Assumes each control task equally removes modality specifics

• Assumes similar psychometric properties for all conditions - unlikely in most cases

• Good to include a common baseline

Hierarchical Subtraction

• Three or more task conditions that progressively include additional factors– Ex A > Rest– Ex B > Ex A– Ex C > Ex B

• Example:– Ex A = words, no response– Ex B = repeat words verbally– Ex C = generate verb associated with word

• Pure Insertion must hold at all levels

Sensory

Motor

Semantic

Parametric

• Increasing level of difficulty or intensity of task• Variation along a single dimension

–A > A > A > A

• Example - working memory load• Useful for determining function in addition to

“where”• Assumes Pure Modulation -

– Different levels produce quantitative differences in level of engagement

– Must be able to define magnitude of differences across levels

Variation of Rate of Extension and Flexion of

Wrist

Step function – fixed increase in activity irrespective of tapping rate

Linear function – linear increase in activity with tapping rate VanMeter, et al., 1995

Differential Response

Premotor Primary Motor (M1)

Selective Attention

• Present same stimuli in all conditions but instruct subject to attend to different features– A B C– A B C– A B C

• Can be done implicitly or explicitly• Assumes cognitive process is modified by what is

attended to• Assumes variables of interest are modulated by

selective attention• Assumes passive processing of unattended features

does not include cognitive processes of attended feature

Selective Attention: Visual Processing

• Corbetta, et al. presented squares, circles, and triangles that changed in color and moved

• On each trial all three parameters were varied

• By instructing subjects to attend to different features able to identify areas that respond uniquely to shape, color, and motion

Trial 1

Trial 2

Trial 3

Selective Attention Example

• Directed attention to specific features elicited selective activation in corresponding form, color, motion centers– Attention to motion -> V5/MT– Attention to color -> V2– Attention to shape -> V1

Adaptation/Repetition Suppression

• Repetitive presentation of same stimulus that produces change in level of activity (typically decreased)

• Inference is areas with diminished response are sensitive to stimulus features

• Also used to diminish response using one type of stimulus to identify response to a novel stimulus

• Pure Modulation Assumption - specific features of stimuli that produce reduction are qualitatively the same

Adaptation

Selectivity Invariance for B Stimuli between A & B

Stimuli

Adaptation in Visual Cortex

Rebound Index = (% signal change per condition) / (% signal change for identical stimuli)

Altmann et al., 2003

Main fMRI Designs for Task Presentation

• Block Design– Multiple trials of the same condition are

presented consecutively– Switch back and forth between blocks of

experimental and control conditions

• Event Related– Trials are presented separately and in

random order with respect to experimental and control conditions

Reasons for Using Block or Event Related Designs

• Block Designs– Better at detecting differences between

conditions (detection)– Some experimental factors take time to

occur (e.g. vigilance or sustained attention)

• Event Related Designs– Better at detecting differences in HRF

(estimation)– Some experimental factors are transient or

infrequent events by nature (e.g. oddball or n-back tasks)

Considerations for Block Designs

• Alternating between experimental and control conditions has limitations (e.g. noun vs verb reading)

• Generally good idea to include null-task blocks - blocks where subjects do “nothing”; fixation on a cross preferred to “nothing”

• Consider including a progression of blocks in which additional factors are added

Analysis of Block Designs

• Subtraction of two conditions only statistical analysis possible of block designs*

• Thus, baseline/ control events equal in importance to experimental condition

• Lengths of block types should be equal

Block Length and Frequency

• Short block lengths presented close together can limit return to baseline of HRF

• Longer blocks maximize difference in signal between conditions

• Best to use many blocks to minimize noise aliased at frequency of task presentation

• Frequency of task should be relatively high to minimize low frequency noise such as scanner drift

Superposition, HRF Model Block Design Indifference

Activations and Deactivations

• Deactivation - decrease in hemodynamic response in task condition relative to control condition

Event Related (ER) Designs

• Trials (aka events) are presented briefly in a random order

• ISI (interstimulus interval) is the separation between events and is also randomized

Analysis of ER Designs

• Average fMRI signal across all of the presentations of the same event type beginning from onset time of the event

• Similar to ERP (event-related potential) analysis used in analysis of EEG data

Comparison of Block and ER Designs - Detection

ER Designs - Estimation

Principles of ER Designs

• Boynton (1996) showed that amplitude and timing of hemodynamic response depends on both intensity and duration of stimulus

• Dale and Buckner (1997) showed that it was possible to extract hemodynamic response function of two different events presented only 1-2 seconds apart

Overlap - Rapid ER

• Difference in degree of activity due to reduced number of events as run length was kept constant

Overlap

• Overlap of events possible due to “jitter”

• Jitter is the randomization of ISI between events

• Without jitter the 1-2 sec ISI will become equivalent to block design

ER Design Advantages

• Flexibility in design• Not every experiment

can be turned into block design

• Flexibility in analysis as same event type can be treated differently

• Trial sorting - choosing events to use in an analysis based on some other parameter such as correctness or reaction time

Semirandom Design

• Slight reduction in detection power• But major increase in estimation

efficiency

Mixed Designs

• Uses a block-design presentation

• Mix– Analysis is done

using trial sorting (e.g. examining only trials with correct response)

– Within a block presented more than one event type

Mixed Design Example - Alzheimer’s Disease

• Two separate runs performed• Run1 (Encoding)

– single words nouns presented– instructed to identify if animate or inanimate

• Run2 (Retrieval)– 8 minutes later present nouns; half old half new– instructed to identify old vs new words

• Analysis examined words in Run1 based on whether they were correctly remembered in Run2

Mixed Design Example - Alzheimer Study

Remembered Trials > Forgotten Trialsin the Encoding run

VanMeter, et al. unpublished

Good Practices

• Simple methods for reducing confounding factors:– Randomization: randomize the order in which

conditions presented• Could also be applied to experimenters; don’t have one

person run all subjects from one group and a second person run all subjects from the other group

– Counterbalancing: switch the order in which conditions are presented across subjects

• Study with subjects assigned to one of two groups; try to ensure equal number of men and women in each group in case there are gender effects

• Randomize order of runs across subjects; limits practice and order effects

Questions to Ask When Designing an Experiment

Good Practices of fMRI Experimental Design

• Evoke the cognitive or other process of interest• Collect as much (fMRI) data as possible• Collect data on as many subjects as possible• Choose stimulus and timing to create maximal

change in cognitive process of interest• Time stimuli presentation of different

conditions to minimize overlap in signal– Use software to optimize design efficiency for ER

designs

• Get measure of subject behavior in the scanner (ideally related to task)

Put Thought into Experimental Design

• Avoid simple comparison of two conditions with minimal thought of what cognitive processes are being compared– Discussion section of these types of papers

come up with a post-hoc “just so story” as to the meaning of results

• Ideally want to test some model• Have hypotheses that can be confirmed

or repudiated

Example of Misuse of fMRI:“This is Your Brain on Politics”

NY Times Op-Ed• Iacoboni, et al. wrote an Op-Ed piece (Nov.

2007) on an experiment designed “to watch the brains of a group of swing voters as they responded to the leading presidential candidates”

• Never published results in any journal• Experimental design consisted of showing 20

subjects (1/2 male & 1/2 female) still photos and videos of speeches from candidates running for presidency at the time

• Compared brain activity with response to questionnaires outside scanner

Clinton’s Results

• Voters who had unfavorable opinions about Sen. Clinton had strong activation of ACC

• Therefore “an emotional center of the brain that is aroused when a person feels compelled to act in two different ways but must choose one. It looked as if they were battling unacknowledged impulses to like Mrs. Clinton.”

Male vs Female Response to Clinton and Giuliani

• “Men show little interest in Mrs. Clinton initially but after watching her video they react positively. Women respond to her strongly at first, but their interest wanes after they watch her video.”

• “With Mr. Giuliani, the reactions are reversed. Men respond strongly to his initial still photos, but this fades after they see his video. Women grow more engaged after watching his video.”

• “For men, Mrs. Clinton is a pleasant surprise. For women, Mr. Giuliani has unexpected appeal.”

Obama and McCain

• “Barack Obama and John McCain have work to do. The scans taken while subjects viewed the first set of photos and the videos of Mr. McCain and Mr. Obama indicated a notable lack of any powerful reactions, positive or negative.”

Is fMRI Simply Correlational or an Epiphenomenon?

• Epiphenomenon - a secondary effect or consequence not directly related to the process of interest

• A major critique of fMRI - we can not state that hemodynamic changes are directly related to neuronal activity given that we don’t fully understand the relationship between the two

Reasons why fMRI is not an Epiphenomenon

• All experiments examine the effect manipulation of the experimental has on the dependent variable

• Same critique could be applied to most types of studies (e.g. drug studies)

• BOLD contrast has been consistently shown to be a reliable predictor of neuronal activity

• Most research also relies on convergent evidence such as what is known from animal studies, other imaging techniques (e.g. MEG, EEG), or deficits that arise from a stroke or tumor

• Logothetis’ studies demonstrating relationship between BOLD and neuronal activity