Why Do Auditors Over-rely on Weak Analytical Procedures? The Role of
Outcome Bias and Insensitivity to Precision
Steve GloverDoug Prawitt
Jeff Wilks
Brigham Young University
Importance of Analytical Procedures
Frequent and increasing use of analytical procedures (Ameen & Stawser 1994; McDaniel & Kinney 1995; Hirst & Koonce 1996; Kinney & McDaniel 1996; Bell et al. 1997)
Findings of the POB (2000)
– Analytical procedures used for substantive testing provided insufficient assurance 20% of the time.
– Insufficient procedures typically related to income statement accounts.
Reliance on Audit Evidence
Auditors sensitive to– Source competence (Bamber 1983)
– Objectivity (Hirst 1994, Caster & Pincus 1996)
– Amount, composition, directness, and deviations from expectations (Caster & Pincus 1996)
What might explain over-reliance on audit evidence, and analytical procedures in particular?
Favorable Outcome Effect
When analytical procedures produce expected or even desired results, auditors are less critical of the procedure.– Motivated reasoning (Kunda 1990)
– Inherited hypotheses (Swann & Giuliano 1987)
– Biased hypothesis testing (Pyszczynski & Greenberg 1987)
Sensitivity to Precision Precision
– The expected closeness of an observed statistic to the “correct” amount
– A critical determinant of the quality of evidence produced by substantive analytical procedures
– SAS No. 56 suggests that• Disaggregated data produces more precise expectations than
aggregated data
• Precision also affected by reliability of source data, predictability of account, and type of procedure
Sensitivity (continued)
Contingent auditor sensitivity to precision– Auditors sensitive to precision when testing a
hard account, but not estimates (McDaniel & Simmons 2003)
– We argue “no-significant-difference” outcomes diminish auditor sensitivity to precision
Experiment 1
No Significant Difference
Significant Difference
Aggregate Procedure
Assess Evidential Strength
H1: Outcome Bias
The assessed strength of audit evidence obtained from an aggregate analytical procedure will be greater for auditors who observe no significant difference (a favorable outcome) than for auditors who observe a significant difference.
Experiment 1
No Significant Difference
Significant Difference
Aggregate Procedure
Assess Evidential Strength
Significant Difference
Disaggregate Procedure
Assess Evidential Strength
Reassess Strength of
Part A
After evaluating the disaggregated procedure, auditors who initially observe no significant difference will lower their assessments of the aggregate procedure’s evidential strength more than will auditors who initially observed a significant difference.
H2: Sensitivity to Precision
Expected Results
Evi
dent
ial S
tren
gth
No
Yes
Initial Assessment of
Aggregate Procedure
Reassessment of Aggregate
Procedure
Originally Observed
Significant Difference
Experiment 1: Method Task
– Determine appropriateness of interest income– Compute expectation for current year based on
annual loan balances and rates– Compute new expectation based on quarterly and
categorized loan balances and rates
Participants– 67 supervising seniors from one Big Four firm
participated during national training sessions– 34.5 months of audit experience (standard deviation
12.8)
Experiment 1: Method
Design (2x2 mixed design)– Favorable outcome: Yes or no (between
subjects)– Precision of analytic: Aggregate or
disaggregate (within-subject)
Primary dependent measure– Evidential strength: 1 (extremely weak) to 7
(extremely strong)
Results (Table 1)
Estimates of Evidential Strength for
Outcome of Aggregate Analytical Procedure
Part A-Aggregate Procedure (Initial
Assessment)
Part B-Disaggregated Procedure – Suggests Material Misstatement
Aggregate Procedure
(Reassessment)
No Significant Difference
3.9 (0.81) [29]
3.7 (0.99) [29]
2.7 (1.16) [29]
Significant Difference 2.9
(1.02) [36]
3.8 (1.03) [36]
2.8 (1.21) [36]
Combined 3.4 3.8 2.8
Control Conditions
Evi
dent
ial S
tren
gth
No
Yes
Initial Assessment of
Aggregate Procedure
Reassessment of Aggregate
Procedure
Originally Observed
Significant Difference
Cueing Sensitivity to Precision
Ex ante warning to consider weaknesses can lead to more careful scrutiny (Petty & Cacioppo 1986)
– Evidenced by lower assessments of strength– Possible demand effect
Include two types of participants– Auditors with relevant task knowledge– Students without such knowledge
H3: Cueing Sensitivity to Precision
Part 1: The perceived strength of evidence obtained from a weak, high-level analytical procedure will be lower for auditors who are cued ex ante to critically evaluate the quality of the analytical procedure than for auditors who are not cued to critically evaluate the procedure.
Part 2: An ex ante cue to critically evaluate the quality of a procedure will make no difference in students’ assessments of evidential strength.
Expected Results
Evi
dent
ial S
tren
gth
Students
Auditors
No Cue Cued to Critically Evaluate
Experiment 2: Method Participants
– 84 supervising seniors from one Big Four firm participate during national training sessions (34.5 months of audit experience, standard deviation 12.8 months)
– 90 graduate students in a second auditing course Design (2x2 between subjects) Task and materials same as Part A, Exp. 1 (ND) Dependent measures same as Part A, Exp. 1 (ND)
Experiment 2: Method The text of the cue read as follows:
Before computing an expectation of interest income for the current year, please consider the strength (quality and sufficiency) of the evidence provided by the interest income analytical procedure used by the audit team last year. Please list in the space provided below one or more weaknesses of this analytical procedure.
Results
Participant
Explicit Guidance to Consider Weaknesses
Professional
Participants
Student
Participants Combined
No 4.1
(0.64) [42]
2.9 (1.27) [45]
3.5 (1.18) [87]
Yes 3.3
(1.05) [42]
2.8 (1.13) [45]
3.0 (1.11) [87]
Combined 3.7 2.9 3.3
Conclusions & Implications
An outcome effect can explain in part why auditors may over rely on weak analytical procedures.– Hinders consideration of precision– May also hinder consideration of…?
• Source competence (Bamber 1983)
• Objectivity (Hirst 1994, Caster & Pincus 1996)
• Amount, composition, directness, and deviations from expectations (Caster & Pincus 1996)
Conclusions & Implications
Ex ante prompt to consider weaknesses of an analytical procedure appears to– Produce no demand effect – Lead to more careful scrutiny by those who
have relevant task knowledge
Limitations & Future Research
All auditors from one Big Four firm The impact of teamwork, the review
process, and accountability not examined