Data Merge Examples, Toolsets for Airborne Data (TAD): Customized Data Merging Function ASDC Introduction The Atmospheric Science Data Center (ASDC) at.

Slides:



Advertisements
Similar presentations
Future Directions and Initiatives in the Use of Remote Sensing for Water Quality.
Advertisements

Group on Earth bservations Discussion Paper on a Framework Dr. Ghassem Asrar August 1, 2003.
The North American Carbon Program Google Earth Collection Peter C. Griffith, NACP Coordinator; Lisa E. Wilcox; Amy L. Morrell, NACP Web Group Organization:
CPSC 695 Future of GIS Marina L. Gavrilova. The future of GIS.
ARM Atmospheric Radiation Measurement Program. 2 Improve the performance of general circulation models (GCMs) used for climate research and prediction.
NASA World Wind. What is NASA World Wind? A richly interactive 3D planetary visualization tool. Smart client architecture. Portal for NASA data. Integrates.
1 ORNL DAAC: Data and Services Robert Cook and Suresh SanthanaVannan Environmental Sciences Division Oak Ridge National Laboratory Oak Ridge, TN Presentation.
Distributed Data Analysis & Dissemination System (D-DADS) Prepared by Stefan Falke Rudolf Husar Bret Schichtel June 2000.
Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.
Obtaining MISR Data and Information Jeff Walter Atmospheric Science Data Center April 17, 2009.
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE) Increasing Accessibility and Interoperability of NASA Data Products with GIS Tools.
The Natural Resources Digital Library Needs, Partners, and Challenges Bonnie Avery, Janine Salwasser, & Janet Webster Oregon State University.
, Data for Disaster Planning, Response, Management and Awareness ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.
The Digital Library for Earth System Education: A Community Resource
Clinical Trials Program PhUSE Semantic Technology WG.
, Increasing Discoverability and Accessibility of NASA Atmospheric Science Data Center (ASDC) Data Products with GIS Technology ASDC Introduction The Atmospheric.
, Implementing GIS for Expanded Data Accessibility and Discoverability ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.
Publishing and Visualizing Large-Scale Semantically-enabled Earth Science Resources on the Web Benno Lee 1 Sumit Purohit 2
User-Friendly Website for Accessing NOAA/ESRL Chemical Sciences Division (CSD) Tropospheric Airborne Data Archive Ken Aikin - NOAA CSD and CIRES, University.
Al Cooper Jeff Stith Earth Observing Laboratory (EOL) HIAPER Debrief 9 November 2005 Ongoing Development and Support: EOL Plans for Deployment of HIAPER.
Introduction to NASA Goddard Space Flight Center(GSFC) Data and Information Services Center(GES DISC) National Aeronautics and Space Administration
1 Data Integration Community of Practice Meeting September 15, 2009 Science Data Integration.
LAKEVIEWS A Brief Introduction.... Federal/Provincial agreement guided by vision of a “healthy, prosperous, & sustainable Great Lakes Ecosystem” for present.
ATMOSPHERIC SCIENCE DATA CENTER ‘Best’ Practices for Aggregating Subset Results from Archived Datasets Walter E. Baskin 1, Jennifer Perez 2 (1) Science.
Enhancing Linkages Between Projects and Datasets: Examples from LBA-ECO for NACP Lisa Wilcox, Amy L. Morrell,
Why do I want to know about HDF and HDF- EOS? Hierarchical Data Format for the Earth Observing System (HDF-EOS) is NASA's primary format for standard data.
IPlant cyberifrastructure to support ecological modeling Presented at the Species Distribution Modeling Group at the American Museum of Natural History.
ESIP Federation Air Quality Cluster Partner Agencies.
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [
Cloud Computing for the NASA Atmospheric Sciences Data Center with Amazon Web Services Cloud Computing for the NASA Atmospheric Sciences Data Center with.
, Key Components of a Successful Earth Science Subsetter Architecture ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.
NASA Earth Science UAS Mission Requirements Don Sullivan NASA Ames Research Center
ESIP Federation 2004 : L.B.Pham S. Berrick, L. Pham, G. Leptoukh, Z. Liu, H. Rui, S. Shen, W. Teng, T. Zhu NASA Goddard Earth Sciences (GES) Data & Information.
View_hdf Kam-Pui Lee Science Applications International Corporation CERES Data Management Team Linda Hunt Computer Sciences Corporation Atmospheric Sciences.
RAINEX Data Management UCAR Joint Office for Science Support José Meitín Jim Moore Dick Dirks UCAR Joint Office for Science Support José Meitín Jim Moore.
NetCDF file generated from ASDC CERES SSF Subsetter ATMOSPHERIC SCIENCE DATA CENTER Conversion of Archived HDF Satellite Level 2 Swath Data Products to.
ESG Observational Data Integration Presented by Feiyi Wang Technology Integration Group National Center of Computational Sciences.
A NASA / NSF / NRL airborne field campaign focusing on atmospheric composition, chemistry, and climate over Southeast Asia. Programmatic Context, Issues.
Cyberinfrastructure to promote Model - Data Integration Robert Cook, Yaxing Wei, and Suresh S. Vannan Oak Ridge National Laboratory Presented at the Model-Data.
Evolving a Legacy System Evolution of the Earth Observing Data and Information System M. Esfandiari 1, H. Ramapriyan 1, J. Behnke 1, E. Sofinowski 2 1.
Data Management Plan for KORUS-AQ Airborne Field Study Dr. Jeong-Hoo Park National Institute of Environmental Research Dr. Gao Chen NASA Langley Research.
“A Library outranks any other one thing a community can do to benefit its people.” Andrew Carnegie Mary Marlino DLESE Program Center Tamara Sumner University.
Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.
SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.
1 Summary. 2 ESG-CET Purpose and Objectives Purpose  Provide climate researchers worldwide with access to data, information, models, analysis tools,
Vision of an Integrated Global Observing System Gregory W. Withee Assistant Administrator for Satellite and Information Services National Oceanic and Atmospheric.
More Information Working Group Composition End Users Data Modelers Data Analysts Airborne Measurement Scientists Airborne Instrument Scientists Data Management.
The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.
Global Change Master Directory (GCMD) Mission “To assist the scientific community in the discovery of Earth science data, related services, and ancillary.
The Earth Information Exchange. Portal Structure Portal Functions/Capabilities Portal Content ESIP Portal and Geospatial One-Stop ESIP Portal and NOAA.
Data Management Practices for Early Career Scientists: Closing Robert Cook Environmental Sciences Division Oak Ridge National Laboratory Oak Ridge, TN.
Support to scientific research on seasonal-to-decadal climate and air quality modelling Pierre-Antoine Bretonnière Francesco Benincasa IC3-BSC - Spain.
Application of NASA ESE Data and Tools to Particulate Air Quality Management A proposal to NASA Earth Science REASoN Solicitation CAN-02-OES-01 REASoN:
1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)
Land Surface Imaging VC: Terms of Reference Julio Dalge, P.G. Diwakar, John Faundeen INPE ISRO USGS CEOS SIT-28 Meeting Hampton, Virginia, USA March 12,
NASA Earth Exchange (NEX) A collaborative supercomputing environment for global change science Earth Science Division/NASA Advanced Supercomputing (NAS)
WMO Global Atmosphere Watch – Atmospheric Composition Matters: To Air Quality, Weather, Climate and More GAW Motivation: Research conducted on atmospheric.
The Virtual Observatory and Ecological Informatics System (VOEIS): Using RESTful architecture and an extensible data model to provide a unique data management.
CEOS Working Group on Information System and Services (WGISS) Data Access Infrastructure and Interoperability Standards Andrew Mitchell - NASA Goddard.
Introduction to BODC and GEOTRACES data office Edward Mawji British Oceanographic Data Centre
Making Satellite Datasets Accessible for Everyone A look into my NASA Internship – summer 2015 Aaron Scott University of North Dakota.
UEE Seminar Series Lidar Sensing of Tropospheric Aerosols and Clouds
Toolsets for Airborne Data
Global Precipitation Data Access, Value-added Services and Scientific Exploration Tools at NASA GES DISC Zhong Liu1,4, D. Ostrenga1,2, G. Leptoukh4, S.
iRODS at the ASDC – Performance Results and Lessons Learned
NPOESS Airborne Sounder Testbed (NAST)
CERES Data Management Team Science Data Processing Workshop 2002
PI: Will Ivancic/GRC Co-PI: Don Sullivan/ARC
CEOS WGISS Carbon Data Portal: Progress and Demo CEOS WGISS Carbon Portal Team Reported at WGISS’48 Vietnam Academy of Science and Technology, Hanoi,
Presentation transcript:

Data Merge Examples, Toolsets for Airborne Data (TAD): Customized Data Merging Function ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. The ASDC specializes in atmospheric data that is important to understanding the causes and processes of global climate change and the consequences of human activities on the climate. The ASDC currently supports more than 44 projects and has over 1,700 archived data sets, which increase daily. ASDC customers include scientists, researchers, federal, state, and local governments, academia, industry, and application users, the remote sensing community, and the general public. Weighted Merge Most aircraft data reported during a single flight is not on a consistent time base and is difficult to intercompare. The TAD merge module provides the user with the ability to merge original data measurements from multiple data providers into a specific time interval or common time base. In 2012, the ASDC facilitated the development of the first ever strategic plan intended for fiscal year 2013 and beyond. The 2013 Strategic Plan serves as a mission-focused plan with six defined goals, each with supporting objectives and tasks for implementation that emphasize the vision and support the mission and values of the ASDC. Amanda Benson 2, Marie Christina Peeters 3, Jennifer Perez 1, Lindsay Parker 2, Gao Chen 1 1. Atmospheric Scence Data Center, NASA Langley Research Center, Hampton VA 2. Science Systems and Applications, Inc., Hampton, VA 3. Langley Aerospace Research Student Scholars (LARSS), NASA Langley Research Center, Hampton, VA Goal #1 The ASDC will strive to expand beyond its existing customer base by increasing accessibility to a broader, worldwide market; through the use of innovative technologies, the ASDC will enhance data access capabilities and develop plans to share data with new user communities. The ICARTT format in which the field campaign data is stored lacks standardization. To compensate, an extensive metadata database was created to track all Principal Investigator-provided metadata and to link the measurement variables to a common naming system. This database is used by data merge module Pursuant to this goal, the ASDC is exploring and piloting new technologies to implement for enhanced data access capabilities. The aircraft datasets collected by ASDC pose unique challenges to science data users due to the sheer volume and variety of the data and the lack of intuitive features of the order tools available to the investigator. To meet the needs of emerging users in the aircraft community, the ASDC is addressing issues in data discovery and delivery. Strategy & Innovation Airborne Tropospheric Chemistry Studies NASA has conducted airborne tropospheric chemistry studies for about three decades. These field campaigns have generated a great wealth of observations, including a wide range of the trace gases and aerosol properties. The ASDC Toolset for Airborne Data (TAD) is being designed to meet the user community needs for manipulating aircraft data for scientific research on climate change and air quality relevant issues. Timebase Data Merge Data ID Timebase Merge Timebase t mi Δt mi Δt pj t pj, if PI data value Merged data value Metadata Database Variable Common Naming System The naming schema used by data providers for their data variables is not standardized. In order to link like variables, a variable common naming system was developed. #:#:#(#)_name Level One Level Two Level Three Frequency Flag Variable Common Name Aerosol: Microphysical Properties 1:1:0 Aerosol: Optical Properties 1:2:0 Aerosol: Composition 1:3:0 Trace Gases: Hydrogen and Oxygen Compounds 2:1:0 Trace Gases: Nitrogen Compounds 2:2:0 Trace Gases: Sulfur Compounds 2:3:0 Trace Gases: Halocarbons and Halogens 2:4:0 Trace Gases: Carbon and Hydrocarbon Compounds: Alkanes 2:5:1 Trace Gases: Carbon and Hydrocarbon Compounds: Alkenes & Alkynes 2:5:2 Trace Gases: Carbon and Hydrocarbon Compounds: Aromatics 2:5:3 Trace Gases: Carbon and Hydrocarbon Compounds: Oxygenates and Acids 2:5:4 Trace Gases: Other Trace Gases 2:6:0 Cloud Properties 3:0:0 J values 4:0:0 Meterological and Nav Parameters: Navigational Parameters 5:1:0 Meterological and Nav Parameters: Meterological Parameters 5:2:0 Radiation Variable 6:0:0 Frequency Flag 0 = Infrequently Used Variable 1 = Frequently Used Variable Way Forward The metadata database and common naming scheme will be used by the merge module within the TAD User Interface to allow for users to merge and download data. Future plans include: Standardized ICARTT format Spatially and temporally subsetted files Data visualization capabilities Spatial grid data/stats DISCOVER-AQ 2013 P3B Flight Path