1. Day 2 DQO Training Course Module 9 The EPA 7-Step DQO Process
Step 7 - Optimize Sample Design
Presenters:
Mitzi Miller and Al Robinson
3:30 PM - 4:45 PM (75 minutes)

2. Terminal Course Objective
To be able to use the output from the previous DQO Process steps to select sampling and analysis designs and understand design alternatives presented to you for a specific project

3. Step 7: Optimize Sample Design
Step 1: State the Problem
Step Objective:
Identify the most resource-effective data collection and analysis design that satisfies the DQOs specified in the preceding 6 steps
Step 2: Identify Decisions
Step 3: Identify Inputs
Step 4: Specify Boundaries
Step 5: Define Decision Rules
Step 6: Specify Error Tolerances
Step 7: Optimize Sample Design

4. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

5. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
The outputs should provide
information on the context
of, requirements for, and
constraints on data collection
design.
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

6. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Based on the DQO outputs
from Steps 1-6, for each
decision rule develop one or
more sample designs to be
considered and evaluated in
Step 7.
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

7. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
For each option, pay close attention
to the Step 4 outputs defining the
population to be represented
with the data:
Sample collection method
Sample mass size
Sample particle size
Etc.
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

8. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Remember:
Sampling Uncertainty is decreased
when sampling density is increased.
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

9. Statistical Methods for Environmental Pollution Monitoring,
Types of Designs
Simple Random
Systematic Grid with random start
Geometric Probability or “Hot Spot” Sampling
Stratified Random
Stratified Simple Random
Stratified Systematic Grid with random start
Statistical Methods for Environmental Pollution Monitoring,
Richard O. Gilbert, 1987

10. Simple Random
Definition- choice of sampling location or time is random
Assumptions
Every portion of the population has equal chance of being sampled
Limitation-may not cover area

11. Simple Random To generate a simple random design:
Either grid the site - set up equal lateral triangles or equal side rectangles and number each grid, use a random number generator to pick the grids from which to collect samples
Randomly select x, y, z coordinates, go to the random coordinates and collect samples

12. Example - Simple Random Using Coordinates

13. Systematic Grid, Random Start
Definition-taking measurements at locations or times according to spatial or temporal pattern (e.g., equidistant intervals along a line or grid pattern)
Assumptions
Good for estimating means, totals and patterns of contamination
Improved coverage of area

14. Systematic Grid, Random Start (cont.)
Limitations
Biased results can occur if assumed pattern of contamination does not match the actual pattern of contamination
Inaccurate if have serial correlation

15. Systematic Grid, Random Start (cont.)
Remember:
Start at random location
Move in a pre-selected pattern across
the site, making measurements
at each point

16. Geometric Probability or Hot-Spot Sampling
Uses squares, triangles, or rectangle to determine whether hot spots exist
Finds hot spot, but may not estimate the mean with adequate confidence

17. Geometric Probability or Hot-Spot Sampling (cont.)
Number of samples is calculated based on probability of finding hot area or geometric probability
Assumptions
Target hot spot has circular or elliptical shape
Samples are taken on square, rectangular or triangular grid
Definition of what concentration/activity defines hot spot is unambiguous

18. Geometric Probability or Hot-Spot Sampling (cont.)
Limitations
Not appropriate for hot spots that are not elliptical
Not appropriate if cannot define what is hot or the likely size of hot spot

19. Example Grid for Hot-Spot Sampling

20. Hot-Spot Sampling
In order to use this approach the decision makers MUST
Define the size of the hot spot they wish to find
Define what constitutes HOT (e.g., what concentration is HOT)
Define the effect of that HOT spot on achieving the release criteria

21. Stratified Random
Definition-divide population into strata and collect samples in each strata randomly
Attributes
Provides excellent coverage of area
Need process knowledge to create strata
Yields more precise estimate of mean
Typically more efficient then simple random
Limitations
Need process knowledge

22. Stratified Systematic Sampling

23. Begin With the Decision in Mind
Data
field
onsite methods
traditional laboratory
Contaminant Concentrations in the Spatial Distribution of the Population
Population Frequency Distribution
Correct Equation for n (Statistical Method)
, , , 
Alternative Sample Designs
Optimal Sampling Design
How Many Samples do I Need?
The end

24. Logic to Assess Distribution and Calculate Number of Samples

25. Sampling Approaches Sampling Approach 1 Sampling Approach 2
Traditional fixed laboratory analyses
Sampling Approach 2
Field analytical measurements
Computer simulations
Dynamic work plan

26. Sampling Approach 2 1. Perform field analytical (using driver COPCs)
2. Define separate populations (pseudo-homogeneous strata)
3. Estimate the distribution(s) based on field/historical data
4. If reasonably normal, use Equation 6.6 (parametric test)
5. If not, use either non-parametric tests or go on to #6
6. Perform simulations on the estimated distribution(s) to determine the number of multi-increment samples (n) required for lab analyses for each strata,varying , , and 
7. Collect n samples, and evaluate m and k and perform lab analysis
8. Perform a red/yellow/green sequential test of data from the labs samples
9. Collect and/or analyze more increments (m) if in yellow region
10. Make the decision(s) when in the red/green region
11. Perform formal, overall DQA to confirm decision(s)
Red, yellow, and green are DQA. Yellow is case where results close to AL

27. Approach 2 Sampling & Lab Analyses
k = 3
k = 3
m = 2
Laboratory

28. Approach 2 Sampling & Lab Analyses (cont.)
n = m * k
Select k of specified Mass/diameter3
FE²  22.5 * d³ / M (to control sampling error)
Prepare m multi-increment samples for lab analysis
Perform lab analyses on m samples
Remember:
Sampling Uncertainty is decreased
when sampling density is increased

29. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
1. Statistical Method/Sample Size Formula
2. Cost Function
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

30. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
1. Statistical Method/Sample Size Formula
Define suggested method(s) for testing the statistical hypothesis
and define sample size formula(e) that corresponds to the method(s).
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

31. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Perform a preliminary DQA:
Generate frequency distribution histogram(s) for each population
Select one or more statistical methods that will address the PSQs
List the assumptions for choosing these statistical methods
List the appropriate formula for calculating the number of samples, n
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

32. CS
Histogram

33. CS
Histogram (cont.)

34. CS
Histogram (cont.)

35. CS
Histogram (cont.)

36. Step 7- Optimize Sample Design
Using the formulae appropriate to these methods, calculate the number of samples required, varying ,  for a given . Repeat the same process using new s.
Review all of calculated sample sizes and along with
their corresponding levels of , , and .
Select those sample sizes that have acceptable levels of
, , and  associated with them.
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

37. 3 Approaches for Calculating n
Normal approach
Skewed approach
FAM/DWP approach
Badly skewed or for all distributions use computer simulation approach
e.g., Monte Carlo

38. Logic to Assess Distribution and Calculate Number of Samples

39. Normal Approach Reject the ‘Normal’ Approach andCS
Normal Approach
Due to using only 12 RI/FS samples for initial distribution assessment, one cannot infer a ‘normal’ frequency distribution
Reject the ‘Normal’ Approach and
Examine ‘Non-Normal’ or ‘Skewed’Approach

40. Logic to Assess Distribution and Calculate Number of Samples

41. Design Approaches Approach 1
Use predominantly fixed traditional laboratory analyses and specify the method specific details at the beginning of DQO and do not change measurement objectives as more information is obtained

42. CS
Cs-137, Eu-152
Because there were multiple COPCs with varied standard deviations, action limits and LBGRs, separate tables for varying alpha, beta, and (LBGR) delta were calculated
For the Cs-137, Eu-152 (in the perimeter samples), the number of samples for a given alpha, beta and delta are presented in the following table

43. Non-Parametric Test CS
For the Perimeter data Cs-137 and Eu-152 have the largest variance
For the Trench footprint data, Pu-239/240 and Cs-137 are the only two COCs with action levels
The following table presents the variation of alpha, beta and deltas for
Cs-137 and Eu-152 in the Perimeter
Pu-239/240 in the Trench Footprint

44. Cs-137 in Perimeter Based on Non-Parametric Test

45. Eu-152 in Perimeter Based on Non-Parametric TestCS

46. Pu-239/240 Trench Footprint Non-Parametric TestCS

47. Approach 1 Based Sampling DesignCS
Design for Radionuclide COCs in Perimeter Soil
Alpha = 0.05 & Beta = 0.20; Delta = 20% of AL
The number of samples in the Perimeter soil was driven by the Eu-152 data as taken from preceding table
The decision makers agreed on collection of 217 surface samples from the Perimeter side-slope soils when excavation was complete
The number of samples in the Trench footprint was the same for either the Pu-239/240 or Cs-137

48. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
2. Cost Function
For each selected sample size, develop a cost
function that relates the number of samples
to the total cost of sampling and analysis.
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

49. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
In order to develop the cost function, the aggregate unit cost per
sample must be determined. This is the cost of collecting one
sample and conducting all the required analyses for a given
decision rule.
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

50. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
These costs include:
The unit sample collection cost
The unit field analysis cost
The unit laboratory analysis cost
For each analytical method selected in Step 3, there is a unit sample collection cost and a unit sample analytical cost.
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

51. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
1. Add the unit sample collection cost (USC$) and the unit sample analytical cost (USA$) for each method chosen.
2. Sum each of the above values for all of the analytical methods chosen to get the aggregate unit sample collection and analysis cost (AUSCA$).
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

52. Aggregate Unit Sampling and Analysis Cost
AUSCA$ = USC$ +  USA$
i=1
Where (here):
USC$ = Unit Sample Collection Cost
USA$ = Unit Sample Analysis Cost
AUSCA$ = Aggregate Unit Sample Collection and Analysis Cost
n = Number of analytical methods planned

53. Total Sampling and Analysis CostCS

54. Step 7- Optimize Sample Design
Merge the selected sample size outputs with the Aggregate Unit Sample Collection and Analysis cost output.
This results in a table that shows the product of each selected sample size and the AUSCA$.
This table is used to present the project managers and decision makers with a range of analytical costs and the resulting uncertainties.
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
From the table, select the optimal sample size that meets the project budget and uncertainty requirements.
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

55. Areas to be InvestigatedCS
Areas to be Investigated

56. Remediation Costs CS
Trench is a rectangle 106 ft (32.3 m) long and 37 ft (11.2 m) wide
Estimated working zone with trench centered within is 166 ft (50.3 m) by 97 ft (29.4 m)
Area of Trench is 3,922 ft2
Area of Perimeter Zone is 12,180 ft2 (excluding Trench area)

57. Remediation Costs (cont.)CS
Volume of Trench, -5 to -20 ft, is 1,654 yd3 Assume $200/yd3 for Soil Being Disposed
Cost of Excavation $330,800
Volume of Perimeter Zone, 1.5/1 slope from 20 ft depth, is 4,507 yd3 (excluding Trench area), Volume of 5 ft of Overburden is 551 yd3, $100/yd3 Onsite Use
Cost of Excavation $505,800

58. CS
Design Options and Costs for Radionuclides Approach 1, Based on 2 Strata

59. Approach 1 Based Sampling DesignCS
Approach 1 Based Sampling Design
Compare Approach 1 S&A costs versus remediation costs
Approach 1 S&A costs = $206,955
Remediation costs = $836,600
Cost to remediate surface soil around perimeter of trench: $330,800
Cost to remediate subsurface soil under footprint of trench: $505,800
Total Analytical + Remediation costs = $1,044,000

60. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
If no sample design meets the
error tolerances within the budget or
consider Approach 2, relax one or more
of the constraints or request more
funding, etc.
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

61. Design - Approaches Approach 2:
Dynamic Work Plan (DWP) & Field Analytical Methods (FAMs)
Manage uncertainty by increasing sample density by using field analytical measurements
Use DWP to allow more field decisions to meet the measurement objectives and allow the objectives to be refined in the field using dynamic work plans

62. Approach 2 Sampling DesignCS
Phase 1: Cs-137 FAM
Establish cost of FAM
Provide detailed SOP for performance of in-situ Cs-137 surveys
Choose a grid size/shape
Following completion of excavation, perform NaI survey for Cs-137
Will produce a representative distribution used to calculate the number of samples for laboratory verification analysis for the perimeter side-slope soils after excavation

63. Approach 2 Sampling Design (cont.)CS
Phase 1: Cs-137
Approach 2 will not be applied to the floor of the excavation
Current RI/FS data for the floor of the excavation in the trench shows site is far below the AL
RIFS data estimate 2 samples needed
Using LBGR of 80% of AL
Alpha=0.05, Beta = 0.2
Present this information along with the recommended designs to the decision makers for review and approval

64. Approach 2 Sampling Design (cont.)CS
Phase 2: Cs-137
Evaluate the FAM results
Evaluate distribution for Cs-137 data
Using the appropriate statistics,
Calculate the mean and standard deviation
Select appropriate alpha, beta, and delta values, and estimate the resulting n based on the Cs-137 data
Collect n samples to confirm the FAM data
Using traditional laboratory analysis per SW-846 or other appropriate methods listed in Step 3

65. Approach 2 Sampling Design (cont.)CS
Utilize in-situ MCA NaI survey of Cs-137
Established a 5 ft square grid over the perimeter side-slope soils after excavation
5 ft grid chosen based on professional judgement
Approximately 122 nodes (approximately half a day to perform)
Used a random start and performed 30 sec. counts at each node of the grid

66. Approach 2 Sampling Design (cont.)CS
Data from similar site
Data showed a non-normal distribution
Calculated mean of 0.28 pCi/g
Calculated standard deviation of 1.02 pCi/g
Choosing alpha=0.05, beta=0.2, delta = 20% of action level
7 samples needed

67. Approach 2 Sampling Design (cont.)CS
Approach 2 Sampling Design (cont.)

68. Approach 2 Sampling Design (cont.)CS

69. Approach 2 Sampling Design (cont.)CS
Approach 2 Sampling Design (cont.)

70. Approach 2 Sampling Design (cont.)CS
Approach 2 Sampling Design (cont.)
Evaluate costs of Approach 2 vs. Approach 1 and remediation costs
Approach 2 S&A costs = $7,164
Approach 1 S&A costs = $206,955
Original budget for S&A = $50,000
Remediation costs = $836,600
Cost to remediate surface soil around perimeter of trench: $330,800
Cost to remediate subsurface soil under footprint of trench: $505,800

71. Approach 2 Was Selected Most Cost-Effective and Best Uncertainty Management

72. Approach 2 Based Sampling Design (cont.)CS
Analysis for Radionuclide COCs
Methods to analyze all of the COCs in soil samples are available
All samples will be shipped and processed as one batch to decrease QC cost

73. Approach 2 Based Sampling Design (cont.)CS
Design for Radionuclide COCs
For each batch, QC will include, as appropriate 1 LCS, 1 method blank, 1 equipment blank (if field equipment is reused between collection of each sample).
Step 3 of the DQO lists the QC measurement criteria

74. Approach 2 Based Sampling Design (cont.)CS
Design for Radionuclide COC Sample analysis
Preservation will not be necessary
QA plan will be written and reviewed by decision makers before implementation

75. Iterative Process
Steps 1- 6
Step 7
Optimal Design

76. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Justification for a judgmental sampling design
Timeframe
Qualitative consequences of an inadequate sampling design (low, moderate, severe)
Re-sampling access after decision has been made (accessible or inaccessible)
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

77. WARNING!!
If a judgmental design is selected in lieu of a statistical design the following disclaimer must be stated in the DQO Summary Report:
“Results from a judgmental sampling design can only be used to make decisions about the locations from which the samples were taken and cannot be generalized or extrapolated to any other facility or population, and error analysis cannot be performed on the resulting data. Thus, using judgmental designs prohibits any assessment of uncertainty in the decisions.”

78. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
The output is the most
resource-effective design for
the study that is expected to
achieve the DQOs.
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions.
Yes No

79. Data Quality Assessment
Step 1: Review DQOs and Sampling Design
Step 2: Conduct Preliminary Data Review
Step 3: Select the Statistical Test
Step 4: Verify the Assumptions of the Test
Step 5: Draw Conclusions From the Data
Guidance for Data Quality Assessment,
EPA QA/G9, 2000

80. Summary
To succeed in a systematic planning process for environmental decision making, you need Statistical Support:
One or more qualified statisticians, experienced in environmental data collection designs and statistical data quality assessments of such designs.

81. Summary (cont.)
Going through the 7-Step DQO Process will ensure a defensible and cost effective sampling program
In order for the 7-Step DQO Process to be effective:
Senior management MUST provide support
Inputs must be based on comprehensive scoping and maximum participation/contributions by decision makers
Sample design must be based on the severity of the consequences of decision error
Uncertainty must be identified and quantified

82. Step 7- Optimize Sample Design
Information IN Actions Information OUT
From Previous Step To Next Step
Review DQO outputs from Steps 1-6 to be sure they are internally consistent
Decision Error Tolerances
Develop alternative sample designs
For each design option, select needed mathematical expressions
Gray Region
Select the optimal sample size that satisfies the DQOs for each data collection design option
Check if number of samples exceeds project resource constraints
Optimal Sample Design
Go back to Steps 1- 6 and revisit decisions. No
Yes

83. End of Module 9
Thank you