Metadata Implementation Ted Habermann NOAA National Geophysical Data Center Documentation: It’s not just discovery... 50% change in global average Why?

Slides:



Advertisements
Similar presentations
Metadata workshop, June The Workshop Workshop Timetable introduction to the Go-Geo! project metadata overview Go-Geo! portal hands on session.
Advertisements

1 OGC Web Services Kai Lin San Diego Supercomputer Center
Geospatial One-Stop A Federal Gateway to Federal, State & Local Geographic Data
FGDC & ISO: What is the Current Status and Considerations when Moving Forward? Viv Hutchison USGS Core Science Systems November 10, 2010 Salem, OR.
Creating Geospatial Metadata for the Long-term Lynda Wayne Federal Geographic Data Committee Geospatial One-Stop GeoMaxim.
Merging Metadata Standards: FGDC CSDGM and ISO Sharon Shin Federal Geographic Data Committee Metadata Coordinator
NOAA Documentation Improvement Ted Habermann How do we measure and visualize improvements in NOAA Documentation? Record Count Completeness (Rubric Scores)
PREMIS Implementation Fair San Francisco, CA, October Stanford Digital Repository PREMIS & Geospatial Resources Nancy J. Hoebelheinrich Knowledge.
Connecting HDF And ISO Metadata Ted Habermann, NASA/ESDIS Hook Hua, Barry Weiss, NASA/Jet Propulsion Lab Mike Folk, Gerd Heber, Elena Pourmal, The HDF.
Oregon Spatial Data Library Partnership Metadata Training OU Knight Library Eugene, Oregon December 3, 2009 Kuuipo Walsh Institute for Natural Resources.
1 ISO – Metadata Next Generation International consensus being built on structured metadata within a broader Geomatics Standard under ISO Technical.
Geospatial standards Beyond FGDC Geog 458: Map Sources and Errors March 3, 2006.
NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.
Introduction to Geospatial Metadata – ISO 191** Metadata National Coastal Data Development Center A division of the National Oceanographic Data Center.
ISO Standards: Status, Tools, Implementations, and Training Standards/David Danko.
Introduction to Geospatial Metadata – FGDC CSDGM National Coastal Data Development Center A division of the National Oceanographic Data Center Please .
Documentation Developments Ted Habermann, NOAA/NESDIS/NGDC Adoption of ISO Metadata Standards - Crossing the Chasm Facilitating the Transition Documentation.
Metadata (for the data users downstream) RFC GIS Workshop July 2007 NOAA/NESDIS/NGDC Documentation.
ORGANIZING AND STRUCTURING DATA FOR DIGITAL PROJECTS Suzanne Huffman Digital Resources Librarian Simpson Library.
AIXM 5.1 Seminar 12 – 13 December 2011
Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.
Interoperable Documentation Ted Habermann, NOAA/NESDIS/NGDC NCAR Earth Observing Laboratory, June 2010 Links: GEO-IDE Wiki:
North American Profile: Partnership across borders. Sharon Shin, Metadata Coordinator, Federal Geographic Data Committee Raphael Sussman; Manager, Lands.
Creating Documentation and Metadata: Metadata for Discovery Lola Olsen 1, Tyler Stevens 2, 1 National Aeronautics and Space Administration (NASA) 2 Wyle.
2008 EPA and Partners Metadata Training Program: 2008 CAP Project Geospatial Metadata: Introduction Module 2: FGDC CSDGM Metadata Compliancy.
Sept 19,  Provides a common set of terminology and definitions  A framework for describing resources and processes  Enables computer based interoperability.
Metadata and Geographical Information Systems Adrian Moss KINDS project, Manchester Metropolitan University, UK
Page 1 LAITS Laboratory for Advanced Information Technology and Standards ISO & Status Liping Di Laboratory for Advanced Information Technology.
FGDC and GOS Metadata: Foundations to Build the NSDI Sharon Shin FGDC Secretariat / Geospatial One-Stop.
Data discovery and data processing for environmental research infrastructures Roberto Cossu ENVRI WP4 leader ESA.
National Coastal Data Development Center A division of the National Oceanographic Data Center Please a list of participants at each location to
Documentation from NcML to ISO Ted Habermann, NOAA NESDIS NGDC.
NODC ↔ Data Consumers Steve Rutz NOAA/NESDIS National Oceanographic Data Center NODC Observing Systems Team Leader June 21, 2011.
Sharing Metadata Recommendations Ted Habermann, John Kozimor Earth Science The HDF Group 1 John Farley Raytheon.
Workshop Resources.
CSDGM Overview CSDGM Tools and Resources. Resources Series Materials: ftp://ftp.ncddc.noaa.gov/pub/Metadata/Online_ISO_Tr aining/Intro_to_Geospatial_Metadata/
Adoption of RDA-DFT Terminology and Data Model to the Description and Structuring of Atmospheric Data Aaron Addison, Rudolf Husar, Cynthia Hudson-Vitale.
ESIP & Geospatial One-Stop (GOS) Registering ESIP Products and Services with Geospatial One-Stop.
Metadata for Data Understandability Ted Habermann NOAA National Geophysical Data Center AGU, Spring 2008.
Using the Global Change Master Directory (GCMD) to Promote and Discover ESIP Data, Services, and Climate Visualizations Presented by GCMD Staff January.
Transitioning from FGDC CSDGM Metadata to ISO 191** Metadata
Metadata – use data discovery e.g. a library catalog data assessment determine the fitness-for-purpose of a data set data retrieval e.g., format.
2008 EPA and Partners Metadata Training Program: 2008 CAP Project Geospatial Metadata: Introduction Module 1: Introduction & Overview of the FGDC CSDGM.
Documenting UAF Data Ted Habermann NOAA/NESDIS/National Geophysical Data Center.
Creating Good Documentation NOAA National Geophysical Data Center
Sneak Preview: Sneak Preview: The New US Geospatial Metadata Standard GeoMaxim Federal Geographic Data Committee (FGDC) Lynda Wayne, GISP Sharon Shin.
Introducing Geospatial Metadata ---1 Don’t Duck Metadata Introducing Geospatial Metadata A Metadata Workshop.
Metadata Training for SEFSC Science Staff Part Two.
ESRI Education User Conference – July 6-8, 2001 ESRI Education User Conference – July 6-8, 2001 Introducing ArcCatalog: Tools for Metadata and Data Management.
Writing Metadata Working Towards Best Practices for SEFSC.
Merging Metadata Standards: FGDC CSDGM and ISO and Sharon Shin Federal Geographic Data Committee Metadata Coordinator
Standard Metadata in Scientific Data Formats September 19, 2007 Flash at:
ISO 191** Overview A “Family” of Standards. Resources ISO Standards Web Page – Technical.
Getting Familiar with Metadata Laurie Porth Rocky Mountain Research Station Audience: Scientists/researchers who have heard of metadata and now need to.
The Proliferation of Metadata Standards and the Evolution of NASA’s Global Change Master Directory (GCMD) Standard for Uses in Earth Science Data Discovery.
A look to the past for the future- The North American Profile Sharon Shin Metadata Coordinator Federal Geographic Data Committee.
British Atmospheric Data Centre ( Searching: Whither NDG? Bryan Lawrence.
Convergence And Trust in Earth and Space Science Data Systems Ted Habermann, NOAA National Geophysical Data Center Documentation: It’s not just discovery...
Global Change Master Directory (GCMD) Mission “To assist the scientific community in the discovery of Earth science data, related services, and ancillary.
Advertising your data Alecia Aleman 1, Ruth Duerr 2 1 National Aeronautics and Space Administration (NASA) 2 National Snow and Ice Data Center, University.
Content Standards for Digital Geospatial Metadata Mandatory Legend Identification Information Data Quality Information Spatial Data Organization Information.
CLASS Metadata and Remote Sensing Extensions CLASS Data Provider’s Conference September 2005 Anna Milan, Ted.Habermann,
Data Quality for Long-Term Datasets
Demystifying Metadata Examining and Understanding the Elements of the Content Standards for Geospatial Metadata.
Geospatial metadata Prof. Wenwen Li School of Geographical Sciences and Urban Planning 5644 Coor Hall
Common Framework for Earth Observation Data
WGISS-41: IDN Report Michael Morahan CEOS WGISS-41 Meeting
Relation between WIS and WIGOS metadata
Creating Geospatial Metadata for the Long-term
Proposal of a Geographic Metadata Profile for WISE
Presentation transcript:

Metadata Implementation Ted Habermann NOAA National Geophysical Data Center Documentation: It’s not just discovery... 50% change in global average Why? i checked my archives, and here is what i found out: it appears that the current 3rd generation algorithm was implemented into operations around Oct-Nov 2002 time frame. cannot say more precisely, as all correspondence i am looking at, talks about this indirectly. (maybe it's what's refered to as the Phase II algorithm.) At the same time, we had implemented quite a few other changes fixing data bugs and formats: view angle problem, increased digitization in all channel's reflectances and AODs, etc. The jump is deemed due to introducing 3rd generation algorithm, which replaced the 2nd generation. The new numbers (~0.08) look more realistic than the previous ones (~0.05 or so). The changes seen in the data is close to the expected effect of this change. The 3rd gen alg takes into account the exact spectral response, whereas the 2nd gen is generic ("one size fits all"). hopefully this settles the issue..

New Documentation Needs For skeptics, the 1,000 or so s and documents hacked last year from the Climactic Research Unit of the University of East Anglia (UEA), in England, establish that global warming is a scientific conspiracy. The climate scientists at the centre of a media storm over leaked s were yesterday cleared of accusations that they fudged their results and silenced critics, but a review found they had failed to be open enough about their work.

Designated Communities - Users Data preservation is communicating with the future

What Goes Where?

Spiral 1: Initial Content Spiral 1: Initial Content Metadata Content Independent of standard Metadata Content Independent of standard Standard Guidance / Implementation Standard Guidance / Implementation Check Back With Users Check Back With Users Spiral 2-N: Scientific Questions New Requirements New Use Cases Check Back With Data Collectors/ Providers Check Back With Data Collectors/ Providers Spiral Development / Training

Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Text Searches Purpose Extent Description Lineage Statement Project Keywords Text Searches Purpose Extent Description Lineage Statement Project Keywords Quality/Lineage Sources Process Steps Quality Reports / Coverages Quality/Lineage Sources Process Steps Quality Reports / Coverages Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Potential Spirals Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Discovery Understanding Connection OnlineResource: Name Description Function Application Schema Connection OnlineResource: Name Description Function Application Schema

SpiralNone0-33%34-66%67-99%All Identification Connection Extent Distribution Description Content Lineage Acquisition Information Metadata Evaluation for Dataset X Date: 2010/10/24 Spirals and Rubrics Progress GOALSGOALS Each record has a score that can be calculated using a consistent, standard tool

Initial Rubric for FGDC > ISO Translation

Identifying Bounding Extents

Connections are Critical

Rubric Reflects the Improvements

Web Accessible Folders - Populations of Records Web Accessible Folder Web Accessible Folder Web Accessible Folder

Completeness Distributions Machine Generated Records Hand Hewn Records

Hierarchical Organization: Data File PlatformSensor + has 0..* + readme + parentDirectory 0..* + subdirectory 0..* Directory Sensor

Hierarchical Organization: Documentation MI_Metadata (from Metadata entity set information) DS_DataSetDS_Platform +has 1..* + seriesMetadata 1..* + composedOf 1..* Multiple Aggregation + superset 0..* + subset 0..* DS_SensorDS_Series

Documentation Repository ISO 19115, , and extensions Documentation Repository ISO 19115, , and extensions THREDDS netCDF DIF, FGDC, Data.Gov DIF, FGDC, Data.Gov SensorML WCS, WMS, WFS, SOS ISO KML Multiple Dialects and Connections