1. [ March 9, 2017] [ Bill Bowles, Audit Supervisor]
Data Analytics for Improved Decision Making
[ March 9, 2017]
[ Bill Bowles, Audit Supervisor]

2. Making Government Work Better
Workshop Objective
Review two different Sampling approaches, and the benefits of each
Use of Data Analytics with large data sets to make valid judgments
Become familiar with several key ACL commands
9/19/2018
Making Government Work Better

3. Making Government Work Better
The Challenge
9/19/2018
Making Government Work Better

4. Sampling is for all of us
Best Practices in Audit Sampling
Sampling is for all of us
Engineers
Health care providers
Data processing professionals
Puplic Safety professionals
Time card departments
Human Resource professionals
Manufacturing
Transportation professionals
Management

5. The Sampling Challenge
Best Practices in Audit Sampling
The Sampling Challenge
I know how to measure and test things, but I’m not a statistician.
Tools are available to let you leverage off of the knowledge needed to make fact based decisions
Use of these tools will make you a sought after individual

6. Best Practices in Audit Sampling
Some Sampling Basics
Population is large relative to the sample size
Must be Random; Each member of the population must have an equal chance of being chosen
Data values should be reasonably consistent
Two basic types of samples:
Statistical or Biased(Judgmental)

7. Best Practices in Audit Sampling
Some Sampling Basics
Statistical- You can make a scientifically valid projection of a attribute of a population, based on the results of a randomly selected sample test.
Biased(Judgmental- You can make a judgement on the non randomly selected sample, but CANNOT project the results to the general population. The results may point to an inference about the population, but it cannot be used as a scientifically valid projection of the population.

8. Best Practices in Audit Sampling
Binomial Sampling
Binomial sampling provides the most information about the characteristics of a population, but requires larger sample sizes. It provides:
Confidence level
Projection of a finite value of the population
Sampling Error
Example:
With a 95% confidence level, there are 75.5%, +/-5% residents who are registered to vote in this population of 68,000 residents.

9. Best Practices in Audit Sampling
Poisson Sampling
Poisson sampling provides support for an assumption about a population, such as the population is compliant: Its advantage is that it requires very small sample sizes, but only defines the maximum error rate of a population. It produces a
Confidence level
Maximum Error Rate of the population
Example:
With a 95% confidence level, there are no more than 5% of the 68,000 voters with bad addresses

10. Which Sampling method to choose
Best Practices in Audit Sampling
Which Sampling method to choose
Poisson: If you believe your population is compliant, use this method to validate your assumption. Good choice for compliance audits
Binomial: If you need to know the value of an attribute of a population with a sampling error. Good choice for compliance audits the large non compliance rates or effectiveness or efficiency assessments.

11. Which Sampling method to choose
Best Practices in Audit Sampling
Which Sampling method to choose
This presentation will focus on Binomial Sampling

12. Best Practices in Audit Sampling
Some Sampling Basics
Avg
1 Std Deviation

13. Picking your sample size
Best Practices in Audit Sampling
Picking your sample size
OR Use a tool

14. Picking your sample size
Best Practices in Audit Sampling
Picking your sample size

15. Best Practices in Audit Sampling
Your population data
When choosing a sample from the population, make sure
The data is reasonably consistent.
The sample is random. Every member of the population has an equal chance of selection

16. Reasonably consistent data
Best Practices in Audit Sampling
Reasonably consistent data

17. Reasonably consistent data
Best Practices in Audit Sampling
Reasonably consistent data
Small Std Deviation relative to the average
Data variation is small

18. Reasonably consistent data
Best Practices in Audit Sampling
Reasonably consistent data

19. Reasonably consistent data
Best Practices in Audit Sampling
Reasonably consistent data
Relatively Small Std Deviation relative to the average
Data variation is small

20. Best Practices in Audit Sampling
Our Challenge
You manage a healthcare office, and need to know what % of all your office visits are between $40 and $50.
You have 544,459 claims, with a total value of $20.2 million and each claim can involve multiple charges.
Can I use sampling to reliably answer this question?

21. Sample Size calculator
Best Practices in Audit Sampling
Sample Size calculator

22. Lets put the theory to work
Best Practices in Audit Sampling
Lets put the theory to work
Sample Metrics
$40 to $50
Count
Percent of Count
Percent of Field
Amount_Paid NO
294
76.56%
71.9%
10,351.62
YES 90
23.44%
28.1%
4,045.54
Totals
384
100%
14,397.16
With a 95% confidence level, we conclude that 28.1% of the $20.2 million in claims, +/- 5% is between $40 to $50 per claim.

23. Lets put the theory to work
Best Practices in Audit Sampling
Lets put the theory to work
Population Metrics
$40 to $50
Count
Percent of Count
Percent of Field
Amount_Paid NO
428,591
78.72%
74.32%
15,058,062.88
YES
115,868
21.28%
25.68%
5,202,967.30
Totals
544,459
100%
20,261,030.18
The actual % of claims between $40 and $50 is 25.7%, which falls within the projection of 28.1%, +\- 5%.
Sampling error is 2.4%

24. Best Practices in Audit Sampling
Tricky Data
My data is not uniform. It varies all over the place. Can I use sampling?
YES, but Stratify

25. Best Practices in Audit Sampling
Stratification
723,746 records
Range is from
$40 to $75K
Total revenue = $3.5 billion

26. Best Practices in Audit Sampling
Stratification
Std Deviation is twice the Average
Range is $40 to $75,000

27. Best Practices in Audit Sampling
Stratification

28. Best Practices in Audit Sampling
Stratification
CAUTION
If you don’t stratify your population where you have large variations in data values, your sampling results will NOT represent the population.

29. Without Stratification
Best Practices in Audit Sampling
Without Stratification
Population
Sample
Produces poor results

30. Best Practices in Audit Sampling
Stratification
Break the 384 samples into groups based on population in each group

31. Best Practices in Audit Sampling
Stratification
Group Sample Size = Tot Sample / Total Population * Group Population

32. Best Practices in Audit Sampling
Stratification
Relatively even sizes of total dollars for each group
139 78 51 54 38 23

33. Best Practices in Audit Sampling
Stratification
Perform the testing of the samples

34. Best Practices in Audit Sampling
The challenge
You own a cleaning business with 300 employees with revenues of $5.9 million per year, and a Gross profit of $2.3 million.
Your service does not clean bathrooms, and you are beginning to notice some customers are going to competitor’s who clean bathrooms.
You want to use data analytics to assess whether it would make sense to expand your business to include bathroom cleaning.

35. The challenge (Continued)
Best Practices in Audit Sampling
The challenge (Continued)
Each employee services an average of 11 customers per week, and spends 2 hours for each cleaning.
You receive $36 per cleaning, and you pay your employees 2 hours per cleaning at $10 to $12 per hour, based on experience.
You estimate that if you get into the bathroom cleaning business, your employees will need to spend 3 hours per cleaning, and you will need to provide cleaning supplies of approximately $6.00 per cleaning.

36. Best Practices in Audit Sampling
Key Questions
How big is the market for this option?
Will I maintain my profit margin
What is my risk?

37. Best Practices in Audit Sampling
Current State
We have gross sales of $5.9 million with payments of $3.6 million for cleaning services
We have 300 employees with a Gross Profit of $2.3 million.
We have 165,550 customer cleanings performed during the past year

38. Best Practices in Audit Sampling
Current State
165,400
3,597,445
5,936,400
300
2,334,183

39. Best Practices in Audit Sampling
Marketing Research
We were able to obtain a list of customer billings for the previous 12 months for 151,543 cleanings that totaled $10.7 million for cleanings that ranged from $60 to $80 per cleaning.
We know that this data includes some customers who have their bathroom included in the service. We also know that some of the cleaners are paid for cleaning supplies.
We would like to know how many of these cleanings include bathroom service.

40. Best Practices in Audit Sampling
Marketing Research
We will pick a random sample of the customer billings and attempt to project to the entire population:
The % of billings that include bathroom service
The % of total revenue that is for bathroom service
The % of billings for bathroom service where the provider had to pay for cleaning service
We want our confidence in the projection to be 95%, and we want our sampling error to be less than +/- 5%

41. Best Practices in Audit Sampling
Marketing Research
151,543
$10.7 million

42. Best Practices in Audit Sampling
Marketing Research
We will select 383 random samples and project thr results to the population

43. Best Practices in Audit Sampling
Marketing Research

44. Best Practices in Audit Sampling
Marketing Research
383

45. Best Practices in Audit Sampling
Marketing Research

46. Best Practices in Audit Sampling
The Projection
It is concluded, with a 95% confidence that 73.8%, +/-5%, of the billings which total $10.7 million include the cleaning of bathrooms.
This indicates that we can compete for approximately $7.9 million, +/- $535K in cleaning business that includes bathrooms.

47. Best Practices in Audit Sampling
The Projection
Average = $70.33
StdDev = $3.93

48. Best Practices in Audit Sampling
$70.33
$66.40
$62.50
Std = 3.93

49. Projecting my market share
Best Practices in Audit Sampling
Projecting my market share
If we set our cleaning service rate at $70, we would be competitive with the market, however,
If we set our rate at $66.50, approximately 82% of my competition would be higher than me.

50. Best Practices in Audit Sampling
Profitability
Re-compute the employee table by making the following changes:
Change the hours per cleaning from 2 to 3
Change sales per cleaning to $66.50
Add a $6.00 cost for cleaning supplies for cleaning
The results:
Total Sales = $11.0 million
Total Cost = $6.38 million
Gross Profit = $4.6 million

51. Making Government Work Better
The Future State
6,369,222
4,572,145
11,029,025
9/19/2018
Making Government Work Better

52. Making Government Work Better
The Challenge
Questions
Tel: (857)
Making Government Work Better
9/19/2018