1. The Importance of the COBIT Framework IT Processes For Effective Internal Control over the Reliability of Financial Reporting: An International Survey UWCISA Symposium, October 11-13, 2007, Toronto, Canada Uday Murthy University of South Florida David S. Kerr University of North Carolina at Charlotte

2. 2 Introduction and Background Publicly held companies must have a system of internal controls, per regulatory requirements Internal controls are heavily “IT-dependent” Need for strong IT governance COBIT – a framework for IT governance  Specifies “best practices” for IT processes  Conformance to COBIT IT processes should result in better internal control

3. 3 Motivation To understand the extent to which the COBIT IT processes contribute to effective internal control over the reliability of financial reporting  Given limited resources, are there certain “key” processes that organizations should focus on from the viewpoint of reliability of financial reporting? To determine whether demographic variations in IT auditors explain differences in perceptions regarding the value of COBIT

4. 4 COBIT Control OBjectives for Information and related Technology Focus of COBIT is on the management and control of IT Comprises 34 IT processes organized into 4 domains  Plan and Organize (plan)  Acquire and Implement (build)  Deliver and Support (run)  Monitor and Evaluate (monitor)

5. 5 Figure 1: COBIT Framework

6. 6 Prior Work COBIT usage survey by Guldentops and De Haes (2002) Profile of COBIT adopters (n=182)  Almost half of the respondents were from the Americas  Most over 1,000 employees with 1/3 rd > 10,000 employees  90% of responding organizations used COBIT  Uses: audit planning and audit program development, validate current IT controls, to evaluate IT risks, to reduce IT risks, and as a framework for improving IT  ~ 40% of respondents indicated that their control framework and audit process was partly COBIT-based; less than 5% of respondents indicated that COBIT had been formally adopted and was enforced as corporate policy

7. 7 Research Questions RQ1: In the context of the reliability of financial reporting, what is the relative importance of each of the 34 IT control and security processes? RQ2: In the context of the reliability of financial reporting, to what extent does the relative importance of each of the 34 IT control and security processes vary as a function of characteristics of the IT professionals within the organization?

8. 8 Method Web survey of IT professionals ISACA members targeted through local chapters Sections of survey instrument  Demographics  Background information  COBIT familiarity  Importance rating for each process, top 10 processes

9. 9 Respondents 189 respondents from 21 countries Average age: 40.1 years Gender: 71% were male. Working in…  industry: 66%  public accounting: 18%  government: 16% Average time with current employer: 5.8 years Degrees: 38% masters; 57% bachelors Certifications: 58% CISAs

10. 10 Selected Demographics

11. 11 Table 2 COBIT Processes Sorted by Mean Importance Ratings COBIT Process* Description of process Mean importance rating DS5Ensure System Security4.661 AI6Manage Changes4.487 PO9Assess Risk4.413 DS11Manage Data4.333 M2Assess Internal Control Adequacy4.328 PO8Ensure Compliance with External Requirements4.222 DS10Manage Problems and Incidents4.101 AI4Develop and Maintain Procedures4.085 M1Monitor the Process4.079 PO11Manage Quality4.074 DS4Ensure Continuous Service4.048 M4Provide for Independent Audit4.021 DS7Educate and Train Users4.005 PO10Manage Projects3.952 M3Obtain Independent Assurance3.947 DS9Manage the Configuration3.931 PO2Define the Information Architecture3.884

12. 12 Table 2 (contd.) COBIT Processes Sorted by Mean Importance Ratings COBIT Process* Description of process Mean importance rating DS13Manage Operations3.884 PO1Define a strategic IT plan3.878 AI5Install and Accredit Systems3.873 PO6Communicate Management Aims and Directions3.825 AI3Acquire and Maintain Technology Infrastructure3.815 AI2Acquire and Maintain Application Software3.799 DS2Manage Third-party Services3.783 PO4Define the IT Organization and Relationship3.746 DS12Manage Facilities3.730 DS1Define and Manage Service Levels3.714 DS3Manage Performance and Capacity3.714 PO5Manage the Information Technology and Relationships3.709 PO7Manage Human Resources3.640 AI1Identify Automated Solutions3.566 PO3Determine the Technological Direction3.545 DS6Identify and Allocate Costs3.407 DS8Assist and Advise Consumers3.238

13. 13 Table 3 Number of times each IT process was selected as a “Top 10” process COBIT processDescription of processTop 10 count DS5Ensure System Security147 AI6Manage Changes133 PO9Assess Risk122 M2Assess Internal Control Adequacy98 DS11Manage Data97 PO1Define a strategic IT plan91 M1Monitor the Process81 AI4Develop and Maintain Procedures74 DS10Manage Problems and Incidents70 DS7Educate and Train Users66 PO8Ensure Compliance with External Requirements64 M4Provide for Independent Audit58 M3Obtain Independent Assurance55 DS4Ensure Continuous Service51 DS9Manage the Configuration50 PO10Manage Projects49 PO2Define the Information Architecture48

14. 14 Table 3 (contd.) Number of times each IT process was selected as a “Top 10” process COBIT processDescription of processTop 10 count AI2Acquire and Maintain Application Software46 PO11Manage Quality45 PO6Communicate Management Aims and Directions44 AI3Acquire and Maintain Technology Infrastructure39 PO4Define the IT Organization and Relationship38 DS1Define and Manage Service Levels38 DS13Manage Operations36 PO5Manage the Information Technology and Relationships35 AI5Install and Accredit Systems35 PO7Manage Human Resources34 DS2Manage Third-party Services31 DS3Manage Performance and Capacity29 PO3Determine the Technological Direction24 DS6Identify and Allocate Costs20 AI1Identify Automated Solutions19 DS12Manage Facilities17 DS8Assist and Advise Consumers6

15. 15 Table 4 Factor Analysis Results: Rotated Component Matrix COBIT process Factor 1Factor 2 Factor 3Factor 4Factor 5Factor 6 Key processes: General & application controls Planning and IT mgmt processes Organization and relationships processes Technology processes Operations and facilities processes Independent audit processes AI6: Manage Changes.787.121.104.120.117.098 DS5: Ensure System Security.755.196.145.175.130.114 DS11: Manage Data.734.197-.050.153.294.033 M2: Assess Internal Control Adequacy.673.067.239-.011.056.390 AI4: Develop and Maintain Procedures.643.041.176.348.050.143 PO9: Assess Risk.624.092.417.093.138.221 DS10: Manage Problems and Incidents.584.431.166.000.354.074 M1: Monitor the Process.551.254.244-.056.245.318 DS7: Educate and Train Users.533.326.171.274.102.081 AI5: Install and Accredit Systems.497.083.057.458.262.165 DS4: Ensure Continuous Service.280.764-.002.149.178-.023 DS3: Manage Performance and Capacity.186.711.249.132.269.131 DS1: Define and Manage Service Levels.202.693.242.066.235.197 PO3: Determine the Technological Direction -.011.649.511.251-.011-.004 PO1: Define a strategic IT plan.158.648.428-.011.036.040 DS8: Assist and Advise Consumers -.010.616.210.177.343.263 DS6: Identify and Allocate Costs -.015.590.252.148.217.478 PO10: Manage Projects.282.543.181.300.019.039 PO11: Manage Quality.484.485.164.159-.064.063

16. 16 Table 4 (continued) Factor Analysis Results: Rotated Component Matrix Factor 1Factor 2Factor 3Factor 4Factor 5Factor 6 COBIT process Key processes: General & application controls Planning and IT mgmt processes Organization and relationships processes Technology processes Operations and facilities processes Independent audit processes PO4: Define the IT Organization and Relationship.138.285.759.063.147.131 PO5: Manage the Information Technology and Relationships.148.209.711.091.171.180 PO6: Communicate Management Aims and Directions.133.510.587.071.083.086 PO7: Manage Human Resources.167.280.572.165.245.134 PO2: Define the Information Architecture.266.454.566.148-.095-.067 PO8: Ensure Compliance with External Requirements.388.032.520.179.263-.002 AI2: Acquire and Maintain Application Software.283.126.188.800.131-.023 AI3: Acquire and Maintain Technology Infrastructure.140.242.256.707.306-.012 AI1: Identify Automated Solutions.160.408-.008.656-.085.285 DS13: Manage Operations.444.185.272.185.606.252 DS2: Manage Third-party Services.247.351.223.084.577-.043 DS12: Manage Facilities.293.338.109.167.570.315 DS9: Manage the Configuration.409.042.393.277.546.075 M3: Obtain Independent Assurance.441.109.158.086.048.765 M4: Provide for Independent Audit.458.156.086.163.672

17. 17 Table 5 Ratings of Technology Processes by Employment Type Employment typeMean*Std. DeviationN Public accounting 4.0505.6776233 Industry 3.6640.83671124 Government 3.6022.6578331 Total 3.7216.79507188 Panel A: Descriptive Statistics * 1=Not at all important; 5=Very important. Source Type III Sum of SquaresdfMean SquareFSig. Corrected Model 4.424(a)22.2123.596.029 Intercept 1813.3111 2948.191.000 Employment 4.42422.2123.596.029 Error 113.786185.615 Total 2722.111188 Corrected Total 118.210187 Panel B: Tests of Between-Subjects Effects a R Squared =.037 (Adjusted R Squared =.027)

18. 18 Table 6 Ratings of Technology Processes: North America vs. Rest of the World Panel A: Descriptive Statistics * 1=Not at all important; 5=Very important. Panel B: Tests of Between-Subjects Effects a R Squared =.017 (Adjusted R Squared =.011) CountryMean*Std. DeviationN USA or Canada 3.8197.8025998 All other countries 3.6148.7771990 Total 3.7216.79507188 Source Type III Sum of SquaresdfMean SquareFSig. Corrected Model 1.970(a)11.9703.152.077 Intercept 2593.1001 4149.321.000 Country 1.9701 3.152.077 Error 116.240186.625 Total 2722.111188 Corrected Total 118.210187

19. 19 Table 7 Extent of COBIT familiarity by Audit Experience Panel A: Descriptive Statistics * 1=Not at all familiar; 5=Very familiar. Panel B: Tests of Between-Subjects Effects a R Squared =.048 (Adjusted R Squared =.043) Extent of audit experienceMean* Std. DeviationN Relatively less (four years or less) 3.65.90297 Relatively more (more than 4 years) 4.081.01491 Total 3.86.979188 Source Type III Sum of Squaresdf Mean SquareFSig. Corrected Model 8.578(a)18.5789.356.003 Intercept 2802.9191 3056.940.000 Audit Exp 8.5781 9.356.003 Error 170.544186.917 Total 2975.000188 Corrected Total 179.122187

20. 20 Table 8 Extent of work relating to task of reviewing/evaluating IT controls: North America vs. Rest of the World Panel A: Descriptive Statistics + Scale 1 = less than 10%; 2 = 10% - 25%; 3 = 26% - 50%; 4 = 51% - 75%; 5 = greater than 75%. Panel B: Tests of Between-Subjects Effects a R Squared =.028 (Adjusted R Squared =.022) CountryMean + Std. DeviationN USA or Canada 3.691.38098 All other countries 3.231.36690 Total 3.471.389188 Source Type III Sum of SquaresdfMean SquareFSig. Corrected Model 9.951(a)19.9515.274.023 Intercept 2251.2701 1193.265.000 Country 9.9511 5.274.023 Error 350.9161861.887 Total 2629.000188 Corrected Total 360.867187

21. 21 Table 9 Extent of Work Relating to Task of Reviewing/Evaluating IT Controls by Employment Type Panel A: Descriptive Statistics + Scale 1 = less than 10%; 2 = 10% - 25%; 3 = 26% - 50%; 4 = 51% - 75%; 5 = greater than 75%. Panel B: Crosstabulation Employment typeMean + Std. DeviationN Public accounting 3.941.34533 Industry 3.461.428124 Government 3.031.14031 Total 3.471.390188 Percentage of work relating to task of reviewing/evaluating IT controls Employment type Less than 10% 10% - 25% 26% - 50% 51% - 75% Greater than 75%Total Public accountingCount24631833 Expected Count3.26.76.05.811.433.0 IndustryCount1426152742124 Expected Count11.925.122.421.842.9124.0 GovernmentCount28133531 Expected Count3.06.35.65.410.731.0 TotalCount1838343365188 Expected Count18.038.034.033.065.0188.0

22. 22 Figure 2: Dendrogram

23. 23 Overview of Results Of the 34 IT processes, results reveal that some are more important than others from the viewpoint of the reliability of financial reporting In particular, five processes stood out as being critical: Ensure System Security (DS5); Manage Changes (AI6), Assess Risk (PO9), Assess Internal Control Adequacy (M2), and Manage Data (DS11) Factor analysis results revealed six distinct factors, with the “general and application controls” factor being the most prominent

24. 24 Limitations True response rate and hence extent of non-response bias is unknown Extent to which importance ratings were affected by the length of the instrument is unknown (the “fatigue factor”) Order of 34 processes was not randomized Despite instructions, it is possible that respondents were not attuned to the focus on the effect of the COBIT IT processes on the reliability of financial reporting Lack of a “reference point” or context for assessing importance of IT processes

25. 25 Conclusion and Future Research Some COBIT IT processes are deemed more critical than others from the standpoint of the reliability of financial reporting Internal and external auditors can focus their attention on the “Top 10” most critical COBIT processes Future research could focus on the why question – why some IT processes are deemed more critical than others Also worth investigating the extent to which COBIT processes contribute to other organizational objectives