1. 1 Analytical Services Program Workshop September 2015 Data Usability Applications and Importance kblack@neptuneinc.org Kelly Black Statistician / President

2. 2 Analytical Services Program Workshop September 2015 What does data usability mean to you?

3. 3 Analytical Services Program Workshop September 2015 How Do I Properly Evaluate My Data? Follow proper steps for evaluating data: 1) Verification of data – completeness review 2) Validation of data – technical evaluation of data for compliance with method, procedure, contract or QAPP 3) Usability assessment – review results of steps 1 and 2, determine data adequacy for decisions being made

4. 4 Analytical Services Program Workshop September 2015 You all have stories that could fit here… I’ll share some of mine… What Could Go Wrong?

5. 5 Analytical Services Program Workshop September 2015 Verification Matters! Historical data being used for current site assessment. Data hand-entered from pdfs of analytical results. Site appeared to be “clean” despite project team recollections that there had been areas of elevated concentrations previously identified. OOPS! Analytical results were printed for data entry, then copied….but only the front side of two- sided documents were copied and entered.

6. 6 Analytical Services Program Workshop September 2015 Site mercury data at levels that triggered mandatory reporting to State… and subsequently fines, remediation, additional sampling, etc. Validation Matters! OOPS! Laboratory incorrectly calculated the assigned value for calibration standards for mercury. Low by a factor of 10x  samples inflated 10x Discovered by focused data validation 6 months later Lesson learned: Use qualified data validators and give them appropriate resources to do their job well.

7. 7 Analytical Services Program Workshop September 2015 Data Usability considers whether data meet project quality objectives as they relate to the decision to be made What is Data Usability (DU)? NOTE: DU assessments are specific to a particular data use / decision. Secondary use of data may require a separate DU assessment.

8. 8 Analytical Services Program Workshop September 2015 All Hands on Deck for DU! Include ALL members of the project team: Field team - impacts of sampling, preservation or shipping issues Chemist – impacts of analytical issues (e.g., poor performing compounds or systemic issues). …ologists – explain data oddities Statistician – of course you need a statistician! Decision maker / regulator – agree to any revisions of the PQOs

9. 9 Analytical Services Program Workshop September 2015 All Hands on Deck for Migration!

10. 10 Analytical Services Program Workshop September 2015 DU isn’t important….except when it is! Data Quality Indicators (DQIs) or Project Quality Objectives (PQOs) defined prior to data collection tell us about the type quantity and quality of data needed. Measurement Quality Objectives (MQOs) or Measurement Performance Criteria set the specific requirements to achieve those PQOs. Why is DU Important?

11. 11 Analytical Services Program Workshop September 2015 Measurement Performance Criteria components Precision Bias Representativeness Comparability Completeness Sensitivity Why is DU Important?

12. 12 Analytical Services Program Workshop September 2015 Precision Matters! Pu-238 measurements at a LANL SWMU. Measurements range up to 6. Values above 4 were considered detects and of potential concern. OOPS! The data were randomly distributed about zero, from -6 to 6 pCi/g. No evidence of any contamination or presence of Pu-238, but variability implied possible presence.

13. 13 Analytical Services Program Workshop September 2015 Bias Matters! Site arsenic levels between 2.0 and 3.4 mg/kg. Local background levels between 4.8 and 6.4 mg/kg. Cool! No arsenic issue. OOPS! Wait! What? How is site lower than local background? Lab method for subtracting out ambient levels caused this bias in measurements.

14. 14 Analytical Services Program Workshop September 2015 Representativeness Matters! Los Alamos National Lab “Mercury SWMU” Data collection, analysis, report showed no contamination of concern. OOPS! Laboratory SOW requested TAL metals….without specifying that mercury should be included in the list.

15. 15 Analytical Services Program Workshop September 2015 Comparability Matters! Uranium results for LANL SWMU: OOPS Again! Some samples ground, some not. OOPS! Analytical results from ICP-MS and KPA.

16. 16 Analytical Services Program Workshop September 2015 Completeness Matters! 120 site characterization samples 112 analyzed by lab, 1 cooler “lost” in transit 93% complete meets MPC of >90% OOPS! Here are the locations of the missing data: x x x x x x x x x x

17. 17 Analytical Services Program Workshop September 2015 Sensitivity Matters! OOPS! If we follow EPA guidance to use ½ LOD, we get a different outcome than if we use LOD…and ½ LOD is below average background concentration. NDs set at ½ LOD (5) Site background EPA soil/dermal RSL

18. 18 Analytical Services Program Workshop September 2015 Know and understand the usability and limitations of your data before reporting conclusions. For secondary use of data, re-think data usability for the new application. Use Data Wisely!

19. 19 Analytical Services Program Workshop September 2015 Thank You!