HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.

Slides:

Advertisements

Similar presentations

Introduction to the BinX Library eDIKT project team Ted Wen Robert Carroll

Advertisements

HDF-EOS 2/5 to netCDF Converter Bob Bane, Richard Ullman, Jingli Yang Data Usability Group NASA/Goddard Space Flight Center.

An Introduction to XML Based on the W3C XML Recommendations.

The HDF Group HDF Group Support for NPP/JPSS Mike Folk, Elena Pourmal, Larry Knox, Albert Cheng The HDF Group The 15 th HDF and HDF-EOS.

HTML5 and CSS3 Illustrated Unit B: Getting Started with HTML

Automating Bespoke Attack Ruei-Jiun Chapter 13. Outline Uses of bespoke automation ◦ Enumerating identifiers ◦ Harvesting data ◦ Web application fuzzing.

QUALITY KEN & EUROSDR ELF Data Quality Tools Bob Chell, Managing Consultant.

The HDF Group Improving long-term preservation of EOS data by independently mapping HDF4 data objects Mike Folk, Ruth Aydt, Joe Lee, Binh-Minh.

Introduction to Geospatial Metadata – FGDC CSDGM National Coastal Data Development Center A division of the National Oceanographic Data Center Please .

Guide to Using Message Maker Robert Snelick National Institute of Standards & Technology (NIST) December 2005

The HDF-EOS5 Tutorial Ray Milburn L3 Communciations, EER Systems Inc McCormick Drive, 170 Largo, MD USA

Javascript and the Web Whys and Hows of Javascript.

Copyright © 2003 Pearson Education, Inc. Slide 3-1 Created by Cheryl M. Hughes, Harvard University Extension School — Cambridge, MA The Web Wizard’s Guide.

HDF5 Tools Update Peter Cao - The HDF Group November 6, 2007 This report is based upon work supported in part by a Cooperative Agreement.

Why XML ? Problems with HTML HTML design - HTML is intended for presentation of information as Web pages. - HTML contains a fixed set of markup tags. This.

HDF-EOS Workshop VII, An XML Approach to HDF-EOS5 Files Jingli Yang 1, Bob Bane 1, Muhammad Rabi 1, Zhangshi Yin 1, Richard Ullman 1, Robert McGrath.

Zhangshi Yin, Jingli Yang, Bob Bane, Muhammad Rabi, Richard Ullman Data Usability Group, NASA/GSFC NASA/GSFC, Code 423 For HDF-EOS Workshop VII September.

CSCI 6962: Server-side Design and Programming Validation Tools in Java Server Faces.

Important ESDIS 2009 tasks review Kent Yang, Mike Folk The HDF Group April 1st, /1/20151Annual briefing to ESDIS.

Metadata Interoperability Framework (MIF) ELAG 2014 Naeem Muhammad Sam Alloing.

Processing of structured documents Spring 2002, Part 2 Helena Ahonen-Myka.

Systems Development Lifecycle Testing and Documentation.

An IDL-BasedValidation Toolkit: Extensions to use the HDF-EOS Swath Format Ken Stone, Center for Lower Atmospheric Studies - University of Colorado, Boulder.

Serving society Stimulating innovation Supporting legislation Joint Research Centre The Inspire Geoportal Validator.

HDF Converting between HDF4 and HDF5 MuQun Yang, Robert E. McGrath, Mike Folk National Center for Supercomputing Applications University of Illinois,

N P O E S S I N T E G R A T E D P R O G R A M O F F I C E NPP/ NPOESS Product Data Format Richard E. Ullman NOAA/NESDIS/IPO NASA/GSFC/NPP Algorithm Division.

1/14/200925th IIPS Conference 1 Challenges to Archive and Access NASA HDF-EOS Data in the long Term MuQun Yang (The HDF Group) Choonghwan Lee (The HDF.

Page 1 Status of HDF-EOS, Related Software, and Tools Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshp XIII Riverdale, MD November 4, 2009.

The HDF Group November 3-5, 2009 HDF-OPeNDAP Project Update HDF/HDF-EOS Workshop XIII1 Joe Lee and Kent Yang The HDF Group James Gallagher.

1 HDF-EOS Status, Related Tools and Issues. 2 Overview.

McIDAS achievements Compiled & Installed McIDAS-X v2008 Tested BOWTIE correction Enhanced MOD4 and MOD6 server code to reflect changes in v005 algorithms.

HDF OPeNDAP Project Update MuQun Yang and Hyo-Kyung Lee The HDF Group March 31, Annual briefing to ESDIS10/31/2015.

FIX Eye FIX Eye Getting started: The guide EPAM Systems B2BITS.

Chapter 8 Usability Specification Techniques Hix & Hartson.

The HDF Group HDF/HDF-EOS Workshop XV1 Tools to Improve the Usability of NASA HDF Data Kent Yang and Joe Lee The HDF Group April 17, 2012.

Page 1 TOOLKIT / HDF-EOS Status and Development Abe Taaheri, Raytheon IIS Aura DSWG meeting October 2007.

XML Extract Paul Kimsey - 3/20/2006 Modified - 4 August 2009.

Page 1 CSISS Center for Spatial Information Science and Systems Access HDF-EOS data with OGC Web Coverage Service - Earth Observation Application Profile.

HDF4 OPeNDAP Project Progress Report MuQun Yang and Hyo-Kyung Lee 1 HDF Developers' Meeting11/24/2015.

NPOESS Enhanced Description Tool - “ned” Richard E. Ullman NASA/GSFC/NPP NOAA/NESDIS/IPO Data / Information Architecture Algorithm / System Engineering.

Starlink VOTable software Author: Mark Taylor Open source Java software for table manipulation STIL:

Software Development Problem Analysis and Specification Design Implementation (Coding) Testing, Execution and Debugging Maintenance.

Data formats and requirements in CMIP6: the climate-prediction case Pierre-Antoine Bretonnière EC-Earth meeting, Reading, May 2015.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.) 1 HDF and HDF-EOS Workshop XII10/17/2008.

1 Status of HDF-EOS, Related Software and Tools. 2 TOOLKIT / HDF-EOS Support.

Robert Wolfe NASA Goddard Space Flight Center Code 614.5, Greenbelt, MD Robert Wolfe NASA Goddard Space Flight Center Code 614.5,

Python Documentation Projects Developers Day 8th International Python Conference 27 January 2000.

Aura HDF-EOS File Format Guidelines: Overview and Status Cheryl Craig.

An HDF-EOS Data Server Based on OPeNDAP and ECHO Bob Bane, Mohammad Rabi, Weijun Su, Richard Ullman, Jingli Yang, Zhangshi Yin Data Usability Group, NASA/GSFC.

GEM METADATA DEVELOPMENT Xiaoping Wang, Macrosearch Allen Macklin, PMEL and Bernard Megrey, AFSC.

The MEDIN stylesheet and ESRI Arc 10: metadata format conversion MEDIN Workshop BGS, Edinburgh, June 2015.

HDF-EOS Workshop IV September 19-21, 2000 Richard E. Ullman ESDIS Information Architect NASA/ GSFC, Code 423.

11/8/2007HDF and HDF-EOS Workshop XI, Landover, MD1 Software to access HDF5 Datasets via OPeNDAP MuQun Yang, Hyo-Kyung Lee The HDF Group.

HDF-EOS Aura File Format Guidelines Cheryl Craig HDF-EOS Workshop IV Sept 20, HDF-EOS Aura File Format Guidelines Cheryl Craig September 20, 2000.

NASA HDF-EOS File Format Overview Joseph M Glassy, Director, MODIS Software Development at NTSG School of Forestry, Numerical Terradynamics Simulation.

CSE 4939 Alex Riordan Brian Pruitt-Goddard. Design an interactive source control application that works between an android phone and a project located.

William H. Bowers – Specification Techniques Torres 17.

Impact Analysis to Refactoring on the Current document Sony Corporation Toshiaki Kojima Mizuki Kanada.

Interop SC 02/03/2016. Agenda Jacques feedbacks Contribution process improvements proposal 2.

HDF and HDF-EOS Workshop XII

Project Objectives Publish to a remote server

Moving from HDF4 to HDF5/netCDF-4

Requirements Validation – II

MAKE SDTM EASIER START WITH CDASH !

CdmCL, a Specific Textual Constraint Language for Common Data Model

HDF-EOS Aura File Format Guidelines

Python’s Standard library part I

Suppose I want to add all the even integers from 1 to 100 (inclusive)

HDF-EOS Workshop XXI / The 2018 ESIP Summer Meeting

What is a System? A system is a collection of interrelated components that work together to perform a specific task.

Presentation transcript:

HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC

HDF-EOS Workshop VII2 HDF-EOS5 Validator Validates HDF-EOS5 files against guidelines Guidelines are encoded in XML Output is a report with specific non- conformance messages

HDF-EOS Workshop VII3 Why HDF-EOS5 Validator? Insures that product files: Have all data and metadata items they are supposed to contain Those items are of correct name, type, and size Validator is a tool mainly for data producers to check format of files before delivering them to users.

HDF-EOS Workshop VII4 HDF-EOS5 Validator Validator HDF-EOS5 file Guidelines (in XML) Validator report

HDF-EOS Workshop VII5 XML Description of File Format Guidelines Like a checklist List of features that must/should be in file And their characteristics Not all-inclusive  Extra things in file will not be reported

HDF-EOS Workshop VII6 What can be validated? Swaths Every swath Swaths with a specific name Fields  Data Every/specific name  Geolocation Every/specific name Dimensions  Name, size

HDF-EOS Workshop VII7 What can be validated (cont)? Attributes Global/Swath/Field Type  String Value/Set/regular expression  Integer/Float Value/Range

HDF-EOS Workshop VII8 Examples: StringValue Any string “seconds” followed by spaces foo bar “foo” or “bar”

HDF-EOS Workshop VII9 Examples: IntValue Any integer value Integer between 17 and 42 inclusive min=“0”/> Non-negative integer

HDF-EOS Workshop VII10 Example: Dimension name is required size is fixed Size can be in range 5-10

HDF-EOS Workshop VII11 Examples: Attributes

HDF-EOS Workshop VII12 Overall File Structure … …

HDF-EOS Workshop VII13 Overall File Structure (cont.) Lat Lon … …

HDF-EOS Workshop VII14 Implementation Command line C HDF-EOS5 library Open source XML parsing libraries expat SCEW Portable to any HDF-EOS5 platform

HDF-EOS Workshop VII15 Sample Output Report $ validate_hdfeos5 -x HIRDLS-definition.xml -h test.h5 SWATH HIRDLS Missing mandatory dimension: nWavel SWATH HIRDLS GEOLOCATION_FIELD Time ATTRIBUTE Units string "TAI " doesn't match SWATH HIRDLS Missing mandatory geolocation field: Altitude SWATH HIRDLS Missing mandatory data field: TemperatureNormChiSq Errors: 4

HDF-EOS Workshop VII16 Status Supports needs of Aura (our test case) Swaths (no Grids/Points yet) Prototype descriptions  For HIRDLS, MLS, OMI, TES  Based on Aura Data Format Guidelines document  Tested against Aura data samples, and correctly reported variances from guidelines Goal is for product producers to write/maintain their own description files