Mark Servilla & Duane Costa LTER Network Office LTER 2012 All Scientist Meeting LTER Network Office.

Slides:



Advertisements
Similar presentations
Dublin Core for Digital Video: Overview of the ViDe Application Profile.
Advertisements

© 2008 EBSCO Information Services SUSHI, COUNTER and ERM Systems An Update on Usage Standards Ressources électroniques dans les bibliothèques électroniques.
Usage Statistics in Context: related standards and tools Oliver Pesch Chief Strategist, E-Resources EBSCO Information Services Usage Statistics and Publishers:
GUIDs in EMu Ian Turnbull KE Software. GUID? UUID? A Globally Unique Identifier (GUID) is a persistent unique reference number used as an identifier.
GTS MetaData Generation data GTS data bases GTS Switch Volume C1 Central Support Office Information Classes white-list Metadata Synchronization.
WELCOME to the LTER Data Co-op with PASTA (Provenance Aware Synthesis Tracking Architecture) All Scientists Meeting 2012 Your source for LTER data.
CSCI3170 Introduction to Database Systems
The North American Carbon Program Google Earth Collection Peter C. Griffith, NACP Coordinator; Lisa E. Wilcox; Amy L. Morrell, NACP Web Group Organization:
Building the LTER Network Information System. NIS History, Then and Now YearMilestone 1993 – 1996NIS vision formed by Information Managers (IMs) and LTER.
Improving Data Discovery in Metadata Repositories through Semantic Search Chad Berkley 1, Shawn Bowers 2, Matt Jones 1, Mark Schildhauer 1, Josh Madin.
Electronically approve and create Suppliers in Oracle Financials using a combination of APEX and Oracle Workflow. NZOUG Conference 2010 Brad Sayer Team.
 Workshops: March & May 2011 and lots of VTCs! Details at:
PREMIS Tools and Services Rebecca Guenther Network Development & MARC Standards Office, Library of Congress NDIIPP Partners Meeting July 21,
Requirements Analysis Update 4 Broad Areas of Participatory Requirements Business The character of geospatial problem solving. System Capabilities to address.
XForms: A case study Rajiv Shivane & Pavitar Singh.
Creating the Foundation for Enterprise Information Management.
ClimDB/HydroDB (ClimHy) Integration ClimHy has been migrated from AND to LNO and will remain status quo in 2011 – Public page (
Database Programming in Java Corresponds with Chapter 32, 33.
AQS Web Quick Reference Guide Changing Raw Data Values Using Maintenance 1. From Main Menu, click Maintenance, Sample Values, Raw Data 2. Enter monitor.
Online Autonomous Citation Management for CiteSeer CSE598B Course Project By Huajing Li.
Building Search Portals With SP2013 Search. 2 SharePoint 2013 Search  Introduction  Changes in the Architecture  Result Sources  Query Rules/Result.
Long Term Ecological Research Network Information System LTER Grid Pilot Study LTER Information Manager’s Meeting Montreal, Canada 4-7 August 2005 Mark.
EML Congruency Checker A tool to assess and report on the quality of EML-based data packages.
Workshop on QC in Derived Data Products, Las Cruces, NM, 31 January 2007 ClimDB/HydroDB Objectives Don Henshaw Improve access to long-term collections.
Controlled Vocabulary Working Group PRESENTED BY JOHN PORTER.
FGDC and GOS Metadata: Foundations to Build the NSDI Sharon Shin FGDC Secretariat / Geospatial One-Stop.
1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.
Searching Business Data with MOSS 2007 Enterprise Search Presenter: Corey Roth Enterprise Consultant Stonebridge Blog:
PASTA Updates Work in progress… NIS Data Portal UI improvements – The website meets and or exceeds standards for accessibility by persons with.
EAK 362/2 MIS LECTURE 4 PART 2 Managing Databases.
USING XML AS A DATA SOURCE. Data binding is a process by which information in a data source is stored as an object in computer memory. In this presentation,
Strategies for Adding EML Support to the GCE Data Toolbox for Matlab Wade Sheldon Georgia Coastal Ecosystems LTER (WWW: gce-lter.marsci.uga.edu/lter)
Introduction to Morpho BEAM Workshop Samantha Romanello Long Term Ecological Research University of New Mexico.
Packaging for Voracity Solutions Control Panel David Turner.
1 Earth System Modeling Framework Documenting and comparing models using Earth System Curator Sylvia Murphy: Julien Chastang:
Building the LTER Network Information System. NIS History, Then and Now YearMilestone 1993 – 1996NIS vision formed by Information Managers (IMs) and LTER.
The Digital Library for Earth System Science: Contributing resources and collections GCCS Internship Orientation Holly Devaul 19 June 2003.
Why EML Metrics Primary quality checks are limited –schema compliance –EML parser (ids and references) Dataset quality not sufficient for automated use.
GBIF Data Access and Database Interoperability 2003 Work Programme Overview Donald Hobern, GBIF Programme Officer for Data Access and Database Interoperability.
A Practical Approach to Metadata Management Mark Jessop Prof. Jim Austin University of York.
1 Understanding Cataloging with DLESE Metadata Karon Kelly Katy Ginger Holly Devaul
Controlled Vocabulary Giri Palanisamy Eda C. Melendez-Colom Corinna Gries Duane Costa John Porter.
Introduction to Morpho RCN Workshop Samantha Romanello Long Term Ecological Research University of New Mexico.
Long Term Ecological Research Network Office Trends Project Spaghetti & Linguine (aka Trends Data Store) Mark Servilla 14 September.
8 Chapter Eight Server-side Scripts. 8 Chapter Objectives Create dynamic Web pages that retrieve and display database data using Active Server Pages Process.
John Porter Sheng Shan Lu M. Gastil Gastil-Buhl With special thanks to Chau-Chin Lin and Chi-Wen Hsaio.
LTER, PASTA, and persistent identifiers LTER IMC Water Cooler Series January 2011.
Find Research Data b2find.eudat.eu B2FIND User Training How to find data objects and collections using EUDAT’s B2FIND This work is licensed.
LTER IM Meeting 2008 – Benson, Boose, Bohm, Gries, Gu, Kaplan, Koskela, Laney, Porter, Remillard, Sheldon and others.
3 Copyright © 2010, Oracle. All rights reserved. Product Data Hub: PIM Functional Training Program Setup Workbench Fundamentals.
ESG-CET Meeting, Boulder, CO, April 2008 Gateway Implementation 4/30/2008.
Sharing Digital Scores: Will the Open Archives Initiative Protocol for Metadata Harvesting Provide the Key? Constance Mayer, Harvard University Peter Munstedt,
Steven Perry Dave Vieglais. W a s a b i Web Applications for the Semantic Architecture of Biodiversity Informatics Overview WASABI is a framework for.
Implementing (parts of) FRAD in a FRBR-based discovery system Jenn Riley Metadata Librarian Indiana University Digital Library Program.
GEONIS. From the IM Proposals Developing “PASTA” ready spatial data for the Network Information System (NIS) – 1. Attend a workshop to create best practices.
JavaScript Introduction and Background. 2 Web languages Three formal languages HTML JavaScript CSS Three different tasks Document description Client-side.
CGI – GeoSciML Testbed 3 Status for BRGM Jean-Jacques Serrano.
Dataset Usability IMC Annual Meeting 2011, EIMC. NIS Time Line IMC Annual Meeting 2011, EIMC.
Topical Analysis and Visualization of (Network) Data Using Sci2 Ted Polley Research & Editorial Assistant Cyberinfrastructure for Network Science Center.
Long Term Ecological Research Network Information System LTER EML Status LTER Information Manager’s Meeting 28 July 2004 Mark Servilla
INFORMATION TECHNOLOGY DATABASE MANAGEMENT. A database is a collection of information organized to provide efficient retrieval. The collected information.
Briefing and Planning meeting on INSPIRE validator implementation – Discussion 16/12/2015.
Developer Exam Preparation Thom Robbins Bryan Soltis
SILO File Upload & Feedback System By Marie Harms State Library of Iowa August 18 & 19, 2010.
Rule-Based Approach for Earth Science Metadata Quality Assurance (QA) Tyler Stevens and Ellen Neff NASA’s Global Change Master Directory (GCMD) WYLE Information.
The Earth System Curator Metadata Infrastructure for Climate Modeling Rocky Dunlap Georgia Tech.
Geospatial metadata Prof. Wenwen Li School of Geographical Sciences and Urban Planning 5644 Coor Hall
IPDA Registry Definitions Project Dan Crichton Pedro Osuna Alain Sarkissian.
Network Information System Advisory Committee (NISAC)
Data Management: Documentation & Metadata
Presentation transcript:

Mark Servilla & Duane Costa LTER Network Office LTER 2012 All Scientist Meeting LTER Network Office

Why LTER Data Co-op? A Diamond in the Rough Demonstrations How can I contribute data? How do I find data? How can I see who is using my data? How is Network synthesis enabled? How is provenance captured? Where do we go from here? Panel Discussion

LTER Network Office Its about community A cooperative … is an autonomous association of persons who voluntarily cooperate for their mutual social, economic, and cultural benefit. - Wikipedia Producers – LTER sites Middleware - PASTA Consumers – Science Community

LTER Network Office

Data producers can evaluate their data package prior to harvesting into PASTA Data packages are discovered via browsing and/or search tools Derived data may be generated when a data package insert or update event occurs Provenance metadata can be generated for derived data packages Data package use information is viewed by a contributor LTER Network Office

LTER Network Data Portal portal.lternet.edu

PASTA Web Service API

Subcomponent of the Data Package Manager component in PASTA Generates a quality report for each data package A quality report contains a set of quality checks Stored as XML but usually rendered in HTML for human readability 27 quality checks implemented in the NIS prototype (of 52 proposed by EML Metrics Working Group) Available to the greater ecoinformatics community via the Data Manager Library (ecoinformatics.org) LTER Network Office

An individual metric or a best practice May involve looking at: metadata (independent of data), or data (independent of metadata), or congruency between metadata and data Can result in one of four statuses valid info warn error LTER Network Office

Users can evaluate data packages before inserting them into PASTA An error status reported by any quality check blocks insertion of the data package into PASTA Every data package stored in PASTA has a quality report that can be accessed along with its metadata and data LTER Network Office

Data Package Quality Report

Evaluate Runs quality checks on the data package but doesnt insert it into PASTA May reveal more diagnostic information (as compared to harvest) because it doesnt necessarily halt after encountering the first error Harvest Runs quality checks on the data package; if no errors are discovered, inserts (or updates) the data package into PASTA May reveal less diagnostic information (as compared to evaluate) because it may halt as soon as an error is encountered Bottom line: Always evaluate before harvesting! LTER Network Office

EML is version or beyond Document is schema-valid EML Document is EML parser-valid All entity-level data URLs are live The packageId pattern matches scope.identifier.revision There are no duplicate entity names An entity-level URL which is not set to information returns data Data table does not have more fields than metadata attributes Data table does not have fewer fields than metadata attributes Database table can be created from EML metadata Field delimiter in metadata is a single character Document is schema-valid after dereferencing enumeratedDomain codes are unique (not yet implemented) LTER Network Office

Data can be loaded into the database Length of entityName is not excessive A methods element is present Record delimiter is present in metadata Data examined and possible record delimiters returned Number of records in metadata matches number of rows loaded At least one keyword element is present Dataset title length is at 5 least words Dataset abstract element is a minimum of 20 words...others not yet implemented LTER Network Office

Display downloaded data Display first insert row coverage element is present temporalCoverage element is present geographicCoverage element is present taxonomicCoverage element is present...others not yet implemented LTER Network Office

Data producers can evaluate their data package prior to harvesting into PASTA Data packages are discovered via browsing and/or search tools Derived data may be generated when a data package insert or update event occurs Provenance metadata can be generated for derived data packages Data package use information is viewed by a contributor LTER Network Office

North Inlet Meteorological – Air Temperature Yearly aggregation of data Down-sample Hourly to Daily and Monthly LTER Network Office … …

LTER Network Office PASTA NIN Workflow NIN Workflow Source Data

LTER Network Office PASTA NIN Workflow NIN Workflow Notify

LTER Network Office PASTA NIN Workflow NIN Workflow Request Data

LTER Network Office PASTA NIN Workflow NIN Workflow Source Data

LTER Network Office PASTA

LTER Network Office PASTA NIN Workflow NIN Workflow Derived Data

Subscribe to a Data Package event

LTER Network Office

Source Data Package Derived Data Package Workflow Description

Provenance Metadata

LTER Network Office

LTER Network Office

December 2012 Support DOI assignment to metadata and data objects Refine NIS Data Portal Complete metadata rendering Improve catalog browsing Hang out shingle Summer 2013 Standup DataONE member node