1. An Interactive Discussion: Contemporary Research on IS Auditors and Automated Controls Dr. Daniel Selby University Of Richmond ISACA VA January 20, 2011

2. Presentation Roadmap Interactive Judgment Exercises (20 minutes)
Interactive Presentation of An Actual Research Study (20 minutes)
Debriefing (10 minutes)
Closing Comments/more Q&A

3. Key Terms
Relevant: information that is useful for a specific judgment outcome (Nisbett et al. 1981; Hilton & Fein 1989;Macrae et al. 1992; and Young et al. 2001)
Irrelevant: information that is of little value for a specific judgment outcome (Nisbett et al. 1981; Hilton & Fein 1989;Macrae et al. 1992; and Young et al. 2001)
Key Risk Factor (KRF): factor that directly effects the successful completion of an objective
i.e., KRFs affect the likelihood of a deficiency or material misstatement of a financial statement amount or disclosure

4. Motivation
IS auditors’ sift through numerous pieces of information to target items that are relevant to understanding KRFs (Singleton 2007).
Individuals have been found to lower their predictions of future outcome predictions when they are exposed to irrelevant information.
Individuals tend to dilute their predictions by unintentionally overlapping characteristics of the irrelevant information with characteristics of the relevant information.
Irrelevant information may influence IS auditors to reduce their assessments of KRFs when higher assessments would be more appropriate.

5. Research Question
Can IS auditors ignore irrelevant information when they assess key risk factors?

6. Why is this study important?
Irrelevant information in a KRF assessment setting should not be used to assess the KRF.
If IS auditors decreased their KRF assessments during a financial statement audit as a result of their exposure to irrelevant information, too few resources may be allocated towards better understanding of the KRFs.
Audit failure may occur

7. So What??? Investors Lose $$$
Litigation for IS auditors and their employing firms
Diminished professional reputations
i.e., window Lehman Brothers

8. Theory
People subconsciously use available information whether relevant or irrelevant, to minimize their cognitive effort during unstructured tasks (Kahneman & Tversky 1972, 1974; Zukier & Jennings 1984).
Salient features of irrelevant information distract attention away from diagnostic features between the target and the outcome to influence perceptions of similarity for the all available information [i.e., “representativeness” Tversky (1977); Nisbett et al. (1981)].
Irrelevant information weakens the relationship between the diagnostic information and outcomes suggested by diagnostic information (Young et al. 2001).

9. Hypotheses
Ha: Irrelevant information will influence auditors to reduce their:
KRF effectiveness rating
likelihood estimate of the risk of material misstatement
audit plan revision relative to the prior year’s clean audit

10. Methodology 37 IS audit specialists 38 minutes on average to complete
Hypothetical integrated audit engagement of a large publicly-traded financial institution
Pretested Cues (Hackenbrack 1992; Waller and Zimbelman 2003)
Randomized order (Shaft & Vessey 1998; LaBella & Koehler 2004)
Manipulation checks: separately rate the diagnosticity (abatement condition in Young et al. 2001)
Knowledge Test/Background demographics

11. Participant Demographics (Std. dev.)
number 37
Avg. # months of experience
49.43 (39.66)
Avg. # of engagements
20.76 ( 26.82)
Avg. # of Professional IS Courses
7.92 (8.14)
Avg. # of IS courses in college
2.95 (3.64)
% professionally Licensed as CISA, CPA, or both
73%

12. Means (Std. Deviations)
KRF Effectiveness Rating
(-3 to +3)
Likelihood Estimate of the Risk of Material Misstatement
(0 to 100)
Audit Plan Revision Relative to the Prior Year’s Clean Audit
(0 to 10)
Irrelevant Information and Relevant KRF
-0.3
(1.27)
43.19
(27.54)
7.86
(1.32)
Relevant KRF only
0.97
(1.19)
57.7
(27.22)
8.46
(1.41)

13. Statistical Results based on one-tail t-tests
KRF Effectiveness Rating (t = 6.473, p = <0.0001)
Likelihood Estimate of the Risk of Material Misstatement (t = 3.948, p = )
Audit plan revisions relative to the prior year’s clean audit (t = 2.227; p = )

14. Knowledge of Automated Controls
Mitigates the influence of irrelevant evidence
Partitioned experimental participants based on knowledge test scores
KRF Effectiveness Rating (p = 0.009)
Likelihood Estimate of the Risk of Material Misstatement (p = 0.002)
Audit plan revisions relative to the prior year’s clean audit (p = 0.019)

15. Conclusion
If IS auditors are influenced by irrelevant evidence when they assess KRFs as they are in the highly simplified setting presented in this study, professional service firms may want to monitor this as a potential problem so that they can minimize the likelihood of audit failure. Next steps - Debriefing

16. An Interactive Discussion: Contemporary Research on IS Auditors and Automated Controls Dr. Daniel Selby