Provenance in Earth Science Gregory Leptoukh NASA GSFC.

Slides:

Advertisements

Similar presentations

What’s new in MODIS Collection 6 Aerosol Deep Blue Products? N. Christina Hsu, Rick Hansell, MJ Jeong, Jingfeng Huang, and Jeremy Warner Photo taken from.

Advertisements

Achieving Global Ocean Color Climate Data Records ASLO Aquatic Sciences Meeting 17 February 2011 – San Juan, Puerto Rico Bryan A. Franz and the NASA Ocean.

Data Stewardship and Data Provenance Activities May 10-11, 2011 Steve Kempler and Greg Leptoukh.

Experiences Developing a User-centric Presentation of Provenance for a Web- based Science Data Analysis Tool Stephan Zednik 1, Gregory Leptoukh 2, Peter.

GEOS-5 Simulations of Aerosol Index and Aerosol Absorption Optical Depth with Comparison to OMI retrievals. V. Buchard, A. da Silva, P. Colarco, R. Spurr.

Gregory Leptoukh, David Lary, Suhung Shen, Christopher Lynnes What’s in a day?

The GSM merging model. Previous achievements and application to GlobCOLOUR Globcolour / Medspiration user consultation, Dec 4-6, 2006, Villefranche/mer.

Satellite Remote Sensing of Surface Air Quality

GOES-R AWG Product Validation Tool Development Aerosol Optical Depth/Suspended Matter and Aerosol Particle Size Mi Zhou (IMSG) Pubu Ciren (DELL) Hongqing.

Evaluating Remote Sensing Data Or How to Avoid Making Great Discoveries by Misinterpreting Data Richard Kleidman ARSET-AQ Applied Remote Sensing Education.

Dr. Natalia Chubarova Faculty of Geography, Moscow State University, , Moscow, Russia, 2012 Gregory G. Leptoukh Online Giovanni.

1 Satellite Remote Sensing of Particulate Matter Air Quality ARSET Applied Remote Sensing Education and Training A project of NASA Applied Sciences Pawan.

IDEA - Infusing satellite Data into Environmental (air quality) Applications Summer 2003: Prototype Analysis, Fusion, and Visualization of NASA ESE and.

Aircraft spiral on July 20, 2011 at 14 UTC Validation of GOES-R ABI Surface PM2.5 Concentrations using AIRNOW and Aircraft Data Shobha Kondragunta (NOAA),

Visualization, Exploration, and Model Comparison of NASA Air Quality Remote Sensing data via Giovanni Ana I. Prados, Gregory Leptoukh, Arun Gopalan, and.

Experiences Developing a Semantic Representation of Product Quality, Bias, and Uncertainty for a Satellite Data Product Patrick West 1, Gregory Leptoukh.

AeroStat: Online Platform for the Statistical Intercomparison of Aerosols Gregory Leptoukh, NASA/GSFC (P.I.) Christopher Lynnes, NASA/GSFC (Co-I.) Robert.

Evaluation aerosol CCI satellite retrievals MACC assimilations Reading 2012.

VALIDATION OF SUOMI NPP/VIIRS OPERATIONAL AEROSOL PRODUCTS THROUGH MULTI-SENSOR INTERCOMPARISONS Huang, J. I. Laszlo, S. Kondragunta,

Giovanni for AQ Gregory Leptoukh NASA Goddard Space Flight Center Goddard Earth Sciences Data and Information Services Center (GES DISC)

1 AOD to PM2.5 to AQC – An excel sheet exercise ARSET Applied Remote Sensing Education and Training A project of NASA Applied Sciences Pawan Gupta Salt.

Summer Institute in Earth Sciences 2009 Comparison of GEOS-5 Model to MPLNET Aerosol Data Bryon J. Baumstarck Departments of Physics, Computer Science,

1 Satellite Remote Sensing of Particulate Matter Air Quality ARSET Applied Remote Sensing Education and Training A project of NASA Applied Sciences Pawan.

IAAR Seminar 21 May 2013 AOD trends over megacities based on space monitoring using MODIS and MISR Pinhas Alpert 1,2, Olga Shvainshtein 1 and Pavel Kishcha.

The MEaSUREs PAR Project Robert Frouin Scripps Institution of Oceanography La Jolla, CA _______________________________________ OCRT Meeting, 4-6 may 2009,

Quality, Uncertainty and Bias Representations of Atmospheric Remote Sensing Information Products Peter Fox, and … others Xinformatics 4400/6400 Week 11,

The first decade OMI Near UV aerosol observations: An A-train algorithm Assessment of AOD and SSA The long-term OMAERUV record Omar Torres, Changwoo Ahn,

AN ENHANCED SST COMPOSITE FOR WEATHER FORECASTING AND REGIONAL CLIMATE STUDIES Gary Jedlovec 1, Jorge Vazquez 2, and Ed Armstrong 2 1NASA/MSFC Earth Science.

MAPSS and AeroStat: integrated analysis of aerosol measurements using Level 2 Data and Point Data in Giovanni Maksym Petrenko Charles Ichoku (with the.

September 4, 2003MODIS Ocean Data Products Workshop, Oregon State University1 Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC) MODIS.

ESIP Federation 2004 : L.B.Pham S. Berrick, L. Pham, G. Leptoukh, Z. Liu, H. Rui, S. Shen, W. Teng, T. Zhu NASA Goddard Earth Sciences (GES) Data & Information.

1 of 26 Characterization of Atmospheric Aerosols using Integrated Multi-Sensor Earth Observations Presented by Ratish Menon (Roll Number ) PhD.

Transparency, applications, and ab- stuff – effect on tools for e-science: it’s all about Informatics June 21, 2010, IATUL 2010 Peter Fox (RPI and WHOI)

Dec 15, 2004 AGUMolly E. Brown, PhD1 Inter-Sensor Validation of NDVI time series from AVHRR, SPOT-Vegetation, SeaWIFS, MODIS, and LandSAT ETM+ Molly E.

Evaluation aerosol CCI retrievals Reading the participants / the task AATSR F v142 AATSR O v202/v2q2 AATSR S v040/v031 MERIS A v21 MERIS B v11 MERIS.

DEVELOPING HIGH RESOLUTION AOD IMAGING COMPATIBLE WITH WEATHER FORECAST MODEL OUTPUTS FOR PM2.5 ESTIMATION Daniel Vidal, Lina Cordero, Dr. Barry Gross.

Aquarius Level-3 Binning and Mapping Fred Patt. Definitions Projection - any process which transforms a spatially organized data set from one coordinate.

NASA and Earth Science Applied Sciences Program

The Second TEMPO Science Team Meeting Physical Basis of the Near-UV Aerosol Algorithm Omar Torres NASA Goddard Space Flight Center Atmospheric Chemistry.

1 Satellite Remote Sensing of Particulate Matter Air Quality ARSET Applied Remote SEnsing Training A project of NASA Applied Sciences Pawan Gupta Satellite.

QA filtering of individual pixels to enable a more accurate validation of aerosol products Maksym Petrenko Presented at MODIS Collection 7 and beyond Retreat.

UV Aerosol Product Status and Outlook Omar Torres and Changwoo Ahn OMI Science Team Meeting Outline -Status -Product Assessment OMI-MODIS Comparison OMI-Aeronet.

Fog- and cloud-induced aerosol modification observed by the Aerosol Robotic Network (AERONET) Thomas F. Eck (Code 618 NASA GSFC) and Brent N. Holben (Code.

Aerosol_cci ECV. Aerosol_cci > Thomas Holzer-Popp > ESA Living Planet Symposium, Bergen, 1 July 2010 slide 2 Major improvements over precursor AOD datasets.

Visualization and workflows Gregory Leptoukh & Christopher Lynnes NASA GSFC.

Satellite Aerosol Validation Pawan Gupta NASA ARSET- AQ – GEPD & SESARM, Atlanta, GA September 1-3, 2015.

1 AOD to PM2.5 to AQC – An excel sheet exercise ARSET Applied Remote Sensing Education and Training A project of NASA Applied Sciences Pawan Gupta NASA.

Ambiguity of Quality in Remote Sensing Data Christopher Lynnes, NASA/GSFC Greg Leptoukh, NASA/GSFC Funded by : NASA’s Advancing Collaborative Connections.

1 N. Christina Hsu, Deputy NPP Project Scientist Recent Update on MODIS C6 Deep Blue Aerosol Products and Beyond N. Christina Hsu, Corey Bettenhausen,

Characterization of Aerosol Data Quality from MODIS for Coastal Regions Jacob Anderson Mentor: Gregory Leptoukh.

Improving MISR-retrieved Aerosol Properties Using GOCART Simulations Yang Liu, PhD June 3, 2015 St. Louis, MO.

Experiences Developing a Semantic Representation of Product Quality, Bias, and Uncertainty for a Satellite Data Product Patrick West 1, Gregory Leptoukh.

High Resolution MODIS Aerosols Observations over Cities: Long Term Trends and Air Quality.

0 0 Robert Wolfe NASA GSFC, Greenbelt, MD GSFC Hydrospheric and Biospheric Sciences Laboratory, Terrestrial Information System Branch (614.5) Carbon Cycle.

MODIS Cryosphere Science Data Product Metrics Prepared by the ESDIS SOO Metrics Team for the Cryosphere Science Data Review January 11-12, 2006.

GEWEX Aerosol Assessment Panel members Sundar Christopher, Rich Ferrare, Paul Ginoux, Stefan Kinne, Jeff Reid, Paul Stackhouse Program Lead : Hal Maring,

Direct aerosol radiative effects based on combined A-Train observations Jens Redemann, Y. Shinozuka, J. Livingston, M. Vaughan, P. Russell, M.Kacenelenbogen,

Goddard Earth Sciences Data and Information Services Center, NASA/GSFC, Code 902, Greenbelt, Maryland 20771, USA INTRODUCTION  NASA Goddard Earth Sciences.

MODIS Near-IR Water Vapor and Cirrus Reflectance Algorithms & Recent Updates for Collection 5 Bo-Cai Gao Remote Sensing Division, Code 7232, Naval Research.

Intercomparison of satellite retrieved aerosol optical depth over ocean G. Myhre, F. Stordal, M. Johnsrud, A. Ignatov, M.I. Mishchenko, I.V. Geogdzhayev,

Satellite Aerosol Comparative Analysis using the Multi-Sensor MAPSS and AeroStat, powered by Giovanni Presented at the Goddard Annual Aerosol Update, at.

Analysis of satellite imagery to map burned areas in Sub-Saharan Africa CARBOAFRICA conference “Africa and Carbon Cycle: the CarboAfrica project” Accra.

MODIS Atmosphere Products: The Importance of Record Quality and Length in Quantifying Trends and Correlations S. Platnick 1, N. Amarasinghe 1,2, P. Hubanks.

International Ocean Color Science Meeting, Darmstadt, Germany, May 6-8, 2013 III. MODIS-Aqua normalized water leaving radiance nLw III.1. R2010 vs. R2012.

Global Air Pollution Inferred from Satellite Remote Sensing Randall Martin, Dalhousie and Harvard-Smithsonian with contributions from Aaron van Donkelaar,

Air Quality Satellite Products from 3D-AQS and Giovanni Ana I. Prados (UMBC/JCET)

Evaluating Remote Sensing Data

OVERVIEW OF THE AEROSTAT PROJECT

Using dynamic aerosol optical properties from a chemical transport model (CTM) to retrieve aerosol optical depths from MODIS reflectances over land Fall.

Global Climatology of Aerosol Optical Depth

Presentation transcript:

Provenance in Earth Science Gregory Leptoukh NASA GSFC

Case 1: MODIS vs. MERIS Same parameterSame space & time Different results – why? MODIS MERIS Provenance aspect: A threshold used in MERIS processing effectively excludes high aerosol values.

Outline Case 1: MODIS vs. MERIS Why Provenance is needed: – knowledge for using the data Case 2: Temporal aggregation Collecting and delivering provenance Harmonizing Multi-sensor provenances: – Joint provenance ≠ prov1 + prov2 Case 3: Orbital characteristics and Dataday Conclusions 12/2/2015Leptoukh:

Merged AOD data from 5 retrieval algorithms (4 sensors: MODIS-Terra, MODIS- Aqua, MISR, and OMI) provide almost complete coverage. Merged multi-sensor aerosol data

Case 2: Temporal aggregation Time series of the global mean values of the AOD over the oceans from Mishchenko et al., 2007 Differences in Aerosol Optical Depth (AOD) between various sensors seemingly exceed reported accuracies of each sensor 12/2/20155Leptoukh:

How sensitive are MODIS aerosols to different time aggregations? Mishchenko et al., 2007 The AOD difference can be up to 40%. Levy, Leptoukh, et al., 2009 MODIS Terra only AOD: difference between diff. aggregations Provenance aspect: Must record and assess differences in aggregation

Collecting and Delivering Data Provenance Where to find the knowledge about data? It is scattered in scientific papers, the actual code, unwritten assumptions, folklore, etc. Assess sensitivity of the results to variations in processing algorithms/steps… Work closely with scientists to guarantee science quality How to deliver provenance? Deliver to users together with the data Present to users in a convenient, easy-to-read fashion Provide recommendations for different data usage (applications vs. climate studies)

Data from multiple sensors: harmonization It is not sufficient just to have the data and their provenance from different sensors in one place Before data can be compared and fused, many items need to be harmonized: Data: format, grid, spatial and temporal resolution Metadata: standard fields, units, scales, quality? Provenance: what to do with it? Product AProduct B Good3 Bad2 Ugly1 0 Are these quality flags compatible?

How to work with multi-sensor data? Capture and classify the details of measurement technique, data collection and processing Identify and spell out similarities and differences Assess importance of these differences Deliver all this information in such a way that a user can easily see and understand the details Present recommendations to guide the data usage and avoid apples-to-oranges comparison and fusion 12/2/2015Leptoukh:

12/2/ Case 3: Why don’t MODIS Terra and Aqua Aerosols agree? MODIS-Terra vs. MODIS-Aqua: Map of AOD temporal correlation, 2008 Leptoukh:

AOD MODIS Terra vs. Aqua in Pacific Over the datelineAway from the dateline R 2 = 0.45 RMS = 0.05 R 2 = 0.72 RMS = /2/201511Leptoukh: Regressing AOD in two adjacent regions lead to different results

12/2/2015Leptoukh: Artifact explained! Max ∆t between Terra and Aqua

Provenance aspect of difference All processing steps are the SAME for both MODIS Dataday definition is the same for both MODIS Processing provenance alone doesn’t provide any explanation for the difference Difference in the Equatorial Crossing time alone is not crucial (a diff. dataday def. handles it correctly) Only a combination of few factors lead to the artifact: MODIS dataday def. together with Equatorial Crossing time It is Knowledge Provenance together with the Processing Provenance 12/2/2015Leptoukh:

About your selected parameters: Parameter AParameter B Difference alert Parameter Name :Aerosol Optical Depth at 550 nm Dataset:MYD08_D3.005MOD08_D3.005  Diff Data-Day definitionUTC (00:00-24:00Z) The same but…. Temporal resolutionDaily Spatial resolution1x1 degree Sensor:MODIS Platform:AquaTerra  Diff EQCT13:3010:30  Diff Day Time NodeAscendingDescending  Diff Pre-Giovanni Processes :ATBD-MOD-30 Giovanni Processes:Spatial subset Time average Spatial subset Time average MDSA: Presenting data provenances 12/2/201514Leptoukh:

Conclusions Data from multiple sensors provides a more complete coverage of physical phenomena Data provenance is needed to insure science quality Developing processing provenance is laborious Joint provenance is even a bigger challenge Proper capture and delivery of joint provenance improves quality of multi-sensor data utilization Combination of knowledge provenance and steps in processing provenance is needed to explain artifacts 12/2/2015Leptoukh: