Frank Hodge, 1
Frank HodgeUniversity of Washington
Computer-Based Computer-Based Instruments:Instruments:
Design Issues and Benefits
Frank Hodge, 2
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
“An expert is a person who has made all the mistakes that can be made in a very narrow field.”
--Niels Bohr
Frank Hodge, 3
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Instrument types─ Attachments─ Surveys─ Basic experiments─ Complex experiments
Frank Hodge, 4
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Coding─ You
─ Student helper
─ A professional─ In house─ Outside
Frank Hodge, 5
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Interface─ Basic─ Adding bells and whistles
Data collection─ Format
Frank Hodge, 6
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Attachments─ Example
─ Sedor (The Accounting Review, 2002)
Frank Hodge, 7
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Surveys─ Front end
─ Back end
─ Examples─ Hodge (Accounting Horizons,
2003)─ U of Washington’s WebQ─ External resources (e.g.,
SurveyMonkey.com)
Frank Hodge, 8
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Basic experiments─ Random assignment
─ Front end
─ Back end
─ Example─ Elliott WP─ External resources
Frank Hodge, 9
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Complex experiments─ Random assignment
─ Front end
─ Back end
Frank Hodge, 10
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Complex experiments─ Examples
─ Hodge, Kennedy & Maines (The Accounting Review, 2004)
─ Elliott, Hodge, Kennedy & Pronk (WP)
─ Hodge & Pronk (WP)
Frank Hodge, 11
Frank Hodge, 12
Frank Hodge, 13
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Questions to ask yourself─ Who will do the coding?
─ What do I want the front end to look like?
─ What data do I want to collect?
─ ∑BENEFITS > ∑COSTS
Frank Hodge, 14
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
“Potential” benefits─ Not geographically constricted─ Not restricted to “normal” hours─ Ability to collect larger samples─ Experiment “reaches out” to
subjects─ Potential to reduce costs─ Making changes can be easier
Frank Hodge, 15
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
“Potential” benefits─ Validity benefits─ Ability to gather “behind the
scenes” data─ Being asked to be on a panel…
Frank Hodge, 16
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Before deciding to use a computer-based experiment do a cost-benefit analysis.
∑BENEFITS*1/2 > ∑COSTS x 2
Frank Hodge, 17
OvervieOvervieww
DesignDesign BenefitBenefitss
ConcluConcludede
Will computer-based experiments become the norm in accounting?
“Prediction is very difficult, especially of the future.”
--Niels Bohr