1 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Sampling and small populations Ljubljana, October 2009 Henk W. H. Geilen RE RA CISA Senior auditmanager Dutch Audit Authority Henk W. H. Geilen RE RA CISA Senior auditmanager Dutch Audit Authority

2 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Outline 1.Introduction 2.Why Sampling 3.What is Sampling 4.“Small populations” 5.Conclusion

3 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Why sampling ? A true Story A long time ago……….

4 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009

5 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 What’s the problem ? Situation A (no errors) What to do? Stop? Seen Enough?

6 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 What is Sampling (2) Situation B (errors found) What to do? Take more items? Correcting only the errors found? ……

7 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 So….. You never know whether you have seen enough items So did you do “enough” work? “Enough” means: Not to few Not to much

8 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Mathematics ? Total number of balls Number of red balls Chance : Red /total Number of draws If you put the balls back after drawing Multiply chances for each draw

9 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Mathematics/Formula Red/(Red + White) * Red/( Red + White)… Number of draws: n Red / (Red +White) = Red / Total = p Chance: ß ß = p n

10 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 So What ? Audit We know “total” We don’t know “error” p We don’t want to know “Chance” ß …..

11 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Audit The “object” is good We mean the “object" is good ENOUGH So there can be a little error Let’s call it MATERIALITY Hé isn’t that p

12 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Chance 100% assurance Is this possible ? Is this necessary ? The auditor defines his risk This means: the chance that he accepts the population while the error is higher than acceptable

13 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Where are we ? We have a formula ß = p n Let’s say ß = 0,05 So the risk is 5% or the Probability = 95 % Say the materiality = 1 % (0,01) Then we can calculate n = the number of samples 0,05 = (1 - 0,01) n N = 300

14 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 This means If you want to accept 1 percent errors (max) You sample 300 times You find zero errors The risk is (less than) 5 % or If you take a sample of 300 items and find 0 errors you know 95 % sure that the maximum error in the population is 1 percent.

15 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Playing with n Different materiality: f.e. 2 percent n = 300/ 2 = 150 f.e. 0,5 percent n = 300 / 0,5 =600 Different risk: 10 percent : n * p = percent : n * p = 190

16 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Based on 0 errors Risk 5 percent 1 error : n * p = errors : n * p = errors : n * p = errors : n * p = errors : n * p = 1052

17 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Risk ß (Beta) Risk : The risk that you accept the population on the basis that the error is less or equal than the materiality α (Alpha) Risk : The risk that you don’t accept the population while in fact there is no error

18 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 What is important Random / A-select Meaning : each element has an equal change of being sampled The sampled set has the same characteristics as the population

19 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 First Conclusions Sampling : A method to calculate how much work to do Parameters : Materiality Confidence level Not size of the population

20 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Back to small populations The questions are/stay: How many items (operations) to audit What items (operations) to audit How to audit an item (operation) Non statistical sample

21 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 How can we do it (how much)? Step 1: Audit all “big” items (big means > materiality) Step 2: Calculate sample size for other items based on results system audit This defines the confidence level

22 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 In detail Total population (all operations) Minus Big operations Is Population to Sample Multiply Confidence Level Is “ temp result ” Devide (Population to sample) multiply Materiality Is Sample Size

23 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Assurance level from systems audit ReliabilityConfidence Level Factor Only minor improvements High60 %0,92 Some improvements Average70 %1,21 Substantial improvements Average80 %1,61 Does not workLow90 %2,31

24 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Example (393 operations) Minus ( 4 operations) Is Multiply 1,21 (70 %) Is Divide (0.02 * ) Is 60 Sample Size (plus 4)

25 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Which items (1) Random Verify Has sample same characteristics as population Average.. MA, Region etc Looks like risk model Commission

26 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 What items (2) Not the same characteristics = Draw again After a few draws …. Expand sample

27 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 After the audit Total error fractions Divide sample size Is average error fraction Multiply population to sample Is projected error Plus error in big operations Is Total error

28 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Evaluation Calculate materiality Compare with total error Example 0,02 * = Vs

29 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 What can you influence ? (393 operations) Minus ( 4 operations) Is Multiply 1,21 (70 %) Is Divide (0.02 * ) Is 60 Sample Size (plus 4)

30 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Reminder What was the goal? Sample? Audit? Audit Opinion?

31 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Conclusions No miracle solutions (Mathematic not Magic) basic principles equal to statistic sampling definition of the audit objective definition of the population definition of the characteristics to test define confidence level, materiality random selection

32 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 More conclusions non-statistical sample size rather large to support audit judgment consider increasing tests of controls to determine confidence level for sampling difficult to determine precision of error projection representativeness of the sample needs to be evaluated

33 Homologues Group Meeting Slovenia, October 2009 Republika SlovenijaEuropean Union Ljubljana, October 2009 Thank you for your attention! Henk Geilen