Zhangshi Yin, Jingli Yang, Bob Bane, Muhammad Rabi, Richard Ullman Data Usability Group, NASA/GSFC NASA/GSFC, Code 423 For HDF-EOS Workshop VII September.

Slides:

Advertisements

Similar presentations

Benchmark Series Microsoft Access 2010 Level 1

Advertisements

HDF-EOS 2/5 to netCDF Converter Bob Bane, Richard Ullman, Jingli Yang Data Usability Group NASA/Goddard Space Flight Center.

Why python? Automate processes Batch programming Faster Open source Easy recognition of errors Good for data management What is python? Scripting programming.

3/5/2009Computer systems1 Analyzing System Using Data Dictionaries Computer System: 1. Data Dictionary 2. Data Dictionary Categories 3. Creating Data Dictionary.

SJSK 1 HDF-EOS Workshop II Metadata Requirements for EOSDIS Data Providers Siri Jodha Singh Khalsa

Maintenance Modifying the data –Add records –Delete records –Update records Modifying the design –Add fields into tables –Remove fields from a table –Change.

Discussions. Overview Introduction Composing Sending an Attachment Viewing Downloading an Attachment Replying.

Mail. Overview Introduction Composing Sending an Attachment Viewing Downloading an Attachment Replying Deleting.

EAS781 Practical Geophysics: The Tools and How to Use Them ArcGis Introduction ArcView ArcInfo ArcGis ?

Different Streaming Technologies. Three major streaming technologies include:

Guide To UNIX Using Linux Third Edition

Guide To UNIX Using Linux Third Edition

Chapter 7 Managing Data Sources. ASP.NET 2.0, Third Edition2.

Web Application Vulnerabilities Checklist. EC-Council Parameter Checklist  URL request  URL encoding  Query string  Header  Cookie  Form field 

Form Handling, Validation and Functions. Form Handling Forms are a graphical user interfaces (GUIs) that enables the interaction between users and servers.

Advanced File Processing

Advanced Shell Programming. 2 Objectives Use techniques to ensure a script is employing the correct shell Set the default shell Configure Bash login and.

1 Creating Web Forms in HTML Web forms collect information from customers Web forms include different control elements including: –Input boxes –Selection.

Lecture 6 – Form processing (Part 1) SFDV3011 – Advanced Web Development 1.

With Microsoft Office 2007 Intermediate© 2008 Pearson Prentice Hall1 PowerPoint Presentation to Accompany GO! with Microsoft ® Office 2007 Intermediate.

EARTH SCIENCE MARKUP LANGUAGE “Define Once Use Anywhere” INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.

HDF-EOS Workshop VII, An XML Approach to HDF-EOS5 Files Jingli Yang 1, Bob Bane 1, Muhammad Rabi 1, Zhangshi Yin 1, Richard Ullman 1, Robert McGrath.

Agenda User Profile File (.profile) –Keyword Shell Variables Linux (Unix) filters –Purpose –Commands: grep, sort, awk cut, tr, wc, spell.

Chapter Four UNIX File Processing. 2 Lesson A Extracting Information from Files.

Guide To UNIX Using Linux Fourth Edition

Introduction to Unix (CA263) File Processing. Guide to UNIX Using Linux, Third Edition 2 Objectives Explain UNIX and Linux file processing Use basic file.

Python File Handling. In all the programs you have made so far when program is closed all the data is lost, but what if you want to keep the data to use.

Eurotrace Hands-On The Eurotrace File System. 2 The Eurotrace file system Under MS ACCESS EUROTRACE generates several different files when you create.

Internet and Distributed Representation of Agent Based Model by- Manish Sharma.

Session 252 What’s New in EDExpress 9.1 Direct Loan for

An HDF-EOS DataBlade using Informix’s Object- Relational Database Renu ChaudhryECOlogicwww.ecologic.net.

LLRP GUI Client User Guide

Chapter 8 Cookies And Security JavaScript, Third Edition.

Why do I want to know about HDF and HDF- EOS? Hierarchical Data Format for the Earth Observing System (HDF-EOS) is NASA's primary format for standard data.

The WinMine Toolkit Max Chickering. Build Statistical Models From Data Dependency Networks Bayesian Networks Local Distributions –Trees Multinomial /

© MIT 2000 Building Web Applications With Webjects Michael D. Barker The MIT Educational Media Creation Center September 2001.

The HDF Group HDF5 Tools Updates Peter Cao, The HDF Group September 28-30, 20101HDF and HDF-EOS Workshop XIV.

Data Management Console Synonym Editor

HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.

HDF & HDF-EOS Workshop VIII 2004 October Aurora, CO Bruce Beaumont, Matt Smith, Helen Conover, Sara Graves Subsetting at UAH.

_______________________________________________________________CMAQ Libraries and Utilities ___________________________________________________Community.

Tutorial 6 Working with Web Forms. 2New Perspectives on HTML, XHTML, and XML, Comprehensive, 3rd Edition Objectives Explore how Web forms interact with.

The HDF Group HDF/HDF-EOS Workshop XV1 Tools to Improve the Usability of NASA HDF Data Kent Yang and Joe Lee The HDF Group April 17, 2012.

The Metadata Tool Custom Metadata Tool Who this tool is for: This tool designed to be used a data management system. This tool is geared more for the.

Lattice QCD Data Grid Middleware: Meta Data Catalog (MDC) -- CCS （ tsukuba) proposal -- M. Sato, for ILDG Middleware WG ILDG Workshop, May 2004.

Tutorial 6 Working with Web Forms. 2New Perspectives on HTML, XHTML, and XML, Comprehensive, 3rd Edition Objectives Explore how Web forms interact with.

An HDF-EOS Data Server Based on OPeNDAP and ECHO Bob Bane, Mohammad Rabi, Weijun Su, Richard Ullman, Jingli Yang, Zhangshi Yin Data Usability Group, NASA/GSFC.

Form Processing Week Four. Form Processing Concepts The principal tool used to process Web forms stored on UNIX servers is a CGI (Common Gateway Interface)

Using the ICAT API to ingest business and experiment metadata Tom Griffin, STFC ISIS Facility NOBUGS 2012 ICAT Workshop

September 23-25, 2003HDF-EOS Workshop VII ECS-HSA the HEW Subsetting Appliance HDF-EOS Workshop VII Silver Spring, MD – September 23-25, 2003 Dr. Sara.

Chapter 5 Introduction To Form Builder. Lesson C Objectives  Use sequences to automatically generate primary key values in a form  Create lists of values.

Files A collection of related data treated as a unit. Two types Text

CAA Database Overview Sinéad McCaffrey. Metadata ObservatoryExperiment Instrument Mission Dataset File.

Configuring MQ Connections and Handlers for MQ adapter 6.5 July 2008.

Day 22, Slide 1 CSE 103 Day 22 Non-students: Please logout by 10:12. Students:

Testing and delivery Web design principles. Web development is software development.

Problem Solving With C++ Doxygen Oct/Nov Introduction Doxygen is a documentation generator, a tool for writing software reference documentation.

Getting Started with CSS

Chapter 7 Text Input/Output Objectives

Chapter 7 Text Input/Output Objectives

19.10 Using Cookies A cookie is a piece of information that’s stored by a server in a text file on a client’s computer to maintain information about.

Chapter 7 Text Input/Output Objectives

Inside Module 10 Editing TurboIMAGE Datasets Page

Metadata Editor Introduction

CHAPTER 4 CLIENT SIDE SCRIPTING PART 2 OF 3

Czech Statistical Office

Remote Data Access Update

Chapter Four UNIX File Processing.

ILogic What’s New.

Getting Started With Solr

Presentation transcript:

Zhangshi Yin, Jingli Yang, Bob Bane, Muhammad Rabi, Richard Ullman Data Usability Group, NASA/GSFC NASA/GSFC, Code 423 For HDF-EOS Workshop VII September 23-25, 2003

HDF-EOS Data Extractor (HEEX) is a tool for data users to extract HDF-EOS data to binary or ASCII data format in HTML or XML.

Flat binary and ASCII data are required by some users. They can be imported to IDL, ARC/INFO and used in any programming. Extracted flat binary and ASCII data can be another intermediate data format for storing EOS data.

HDF-EOS Data ASCII Data Binary Data HTMLXML Binary Data

heex [-aehx] [-f] [FieldName] -o OutputDirectory HDF-EOS5FileName HDF-EOS5FileName: HDF-EOS5 file name. Required. -a Extract data to ASCII format. Default is binary. Optional. -e Extract every field. Optional. -f FieldName: extract a field data specified by objec/field. Optional. If no -e and -f options, only index file, no data. -h Help message. Optional. -o OutputDirectory: the directory to put data. Required. -x Extract IndexFile data to XML format. Default is html. Optional.

HEEX will extract all data into a directory indicated in -o optional by user. A file called Index.html or Index.xml in the directory will show all HDF-EOS data structure and where data are.

-HDFEOS-File Name=grid.he Type="HDFEOS" -HDFEOS -GRIDS -Grid +Parameters +Dimension -GridDataField +sst_mask_1 -HDFEOS_INFORMATION -COREMETADATA +CoreMetadata.0 +ARCHIVEMETADATA

-sst_mask_1 DataType=Integer Sign=false Size=1 +DimList +FieldStorage Data:./Grid/sst_mask_1.dat./Grid/sst_mask_1.dat

HDF-EOS Metadata Updater (HEMU) is a tool for data producers or users to update metadata inside an HDF-EOS file (HDF-EOS2 and HDF-EOS5).

Users need to read metadata from HDF-EOS data. Users need to write metadata files to HDF-EOS data replace all metadata in the HDF-EOS data. Users need to update metadata in HDF-EOS data by merging metadata inside HDF-EOS data and metadata from.met files.

1. Extract metadata from a dataset (-x option) to a text file which can then be modified with any text editor. This option does not modify the HDF-EOS file. 2. Replace metadata with text from an external file (-f option). The text is syntax-checked for validity as ODL before updating. 3. Update metadata with text from an external file (-m option). This option is designed to update metadata using a.met file.

HDF-EOS Data Read CoreMetadata InventoryMetadata ArchiveMetadata Metadata

HDF-EOS Data.Met Data Read Metadata Check Syntax Read Metadata Check Values: SHORTNAME PRODUCTIONDATETIME VERSIONID If equal, replace metadata in HDF-EOS using metadata in.met Check Syntax If not equal, exit

HDF-EOS Data.Met Data Read Metadata Check Syntax Read Metadata Check Values: SHORTNAME PRODUCTIONDATETIME VERSIONID If equal, replace metadata in HDF-EOS using CollectionMetadata InventoryMetadata in.met and ArchiveMetadata in HDF-EOS Check Syntax If not equal, exit

hemu [-h] HDF-EOSFileName -xmf [MetadataFileName] -h Print help message. -x Extract metadata from HDFEOS file. -m Update metadata with a.met file input. -f Forcibly replace metadata with metadata from input. HDF-EOSFileName: HDF-EOS file. Required. MetadataFileName: Metadata file. Optional (default is standard input/output).

HEMU will check metadata syntax by ODL program. For -m and –f options (update metadata with a.met file), three parameters in INVENTORYMETADATA group (SHORTNAME, PRODUCTIONDATETIME, and VERSIONID) must match before the tool will modify the metadata.