Data Quality A Science Community Perspective 17/13/11K. Lehnert, ESIP Panel on Data Quality Kerstin Lehnert Lamont-Doherty Earth Observatory Columbia University Thanks for helpful comments: Mark Ghiorso Ken Ferrier Al Hofmann Alexey Kaplan Roger Nielsen Mohan Ramamoorthy Tom Whittaker

2 DQ & Science 7/13/11K. Lehnert, ESIP Panel on Data Quality2 ScienceTechnology Norms Standards Tools

The Social Side of DQ 7/13/11K. Lehnert, ESIP Panel on Data Quality3 “The reliability of knowledge about climate change depends on the commensurability of data in space and time.” From Paul N. EdwardsPaul N. Edwards: "A Vast Machine": Standards as Social Technology Science, vol. 304, 2004 DOI: /science Matthew Maury's 1858 diagram of the global atmospheric circulation.

4 Earth Science Data 7/13/11K. Lehnert, ESIP Panel on Data Quality4

Error Budgets Diagram from White Paper on the SST Error Budget, produced by the U.S. SST Science Team 7/13/11K. Lehnert, ESIP Panel on Data Quality 5

6 DQ: Instrument Errors 7/13/11K. Lehnert, ESIP Panel on Data Quality6 “Most of the rapid decrease in globally integrated upper (0– 750 m) ocean heat content anomalies (OHCA) between 2003 and 2005 reported by Lyman et al. [2006] appears to be an artifact resulting from the combination of two different instrument biases recently discovered in the in situ profile data.”

“Mantle Myths, Reservoirs, and Databases” Presentation by A. Hofmann at the Goldschmidt Conference 2008 DQ: Precision 7/13/11K. Lehnert, ESIP Panel on Data Quality7

8 What Defines DQ?  “Knowing that I can trust the numbers.”  “Data having an uncertainty that actually corresponds to the uncertainty stated in the the source.”  “In one word, ‘completeness’.” (allows others to assess the validity of data, because then you can check for standards used, techniques, reproducibility, etc.  Reproducibility, precision, … 7/13/11K. Lehnert, ESIP Panel on Data Quality8

9 How Do You Evaluate DQ?  ‘Analytical completeness’, including uncertainties, and metadata.  Statistical tests, internal consistency.  Rely on reputation of the investigator, either directly or by association.  “Well, usually I don't, because that's a lot of work.” 7/13/11K. Lehnert, ESIP Panel on Data Quality9

10 DQ Needs Carrots & Sticks  Tools for DQ metadata management, e.g. capture during data acquisition  Software for using DQ metadata in data analysis, synthesis, modeling  Policies for and enforcement of data & metadata reporting  Peer-review of data 7/13/11K. Lehnert, ESIP Panel on Data Quality10

11 Data Publication  Publication of data in repositories  QC/QA at repository (completeness, consistency)  Open Access  Long-term archiving  Link to scientific articles via unique identifiers  Support for investigators to comply with agency policies 7/13/11K. Lehnert, ESIP Panel on Data Quality11

12 Conclusions (I): Science Community  Needs to define the disciplinary norms for DQ measures  Needs to drive the implementation of disciplinary standards  Policies for data reporting & publication  Recommendations for data acquisition 7/13/11K. Lehnert, ESIP Panel on Data Quality12

13 Conclusions (II): Technology  Needs to translate disciplinary standards to technical standards  Needs to provide software tools that facilitate DQ management (capture, communication, & assessment) 7/13/11K. Lehnert, ESIP Panel on Data Quality13

14 Conclusion (III)  Science and technology need to work closely to develop meaningful solutions for DQ management.  The process needs to take into account the diversity of Earth Science disciplines and data types. 7/13/11K. Lehnert, ESIP Panel on Data Quality14