Introduction What purpose does a data archive center serve if users can’t find or access the holdings they might need to facilitate their research discoveries?

Slides:



Advertisements
Similar presentations
OpenDOAR The Directory of Open Access Repositories Bill Hubbard SHERPA Manager University of Nottingham.
Advertisements

The Seven Pillars of Open Language Archiving: A Vision Statement Gary Simons and Steven Bird Workshop on Web-based Language Documentation and Description.
1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.
CAP Support in Esris Open Source Geoportal Server WMO Information System (WIS) CAP Implementation Workshop Geneva, 6-7 April 2011 Clive Reece
Managing Dataset DOIs and Versions in a Changing Archive Steven Worley Bob Dattore Zaihua Ji National Center for Atmospheric Research Boulder, Colorado,
Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.
New Resources in the Research Data Archive Doug Schuster.
ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.
The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Unidata TDS Workshop THREDDS Data Server Overview October 2014.
Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.
October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.
January 4, 2012 Christine White, Esri. A long time ago in a galaxy far, far away....
1 Closing Report on WTF-CEOP Project Team Satoko Horiyama MIURA/JAXA.
Extending the Discovery Environment: Tool Integration and Customization.
Research Data at NCAR 1 August, 2002 Steven Worley Scientific Computing Division Data Support Section.
Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.
Data for Climate and Energy Studies Steven Worley Computational and Information Systems Laboratory NCAR.
Mobile Apps Expand Your Distribution Generate New Revenue.
The National Center for Atmospheric Research is operated by the University Corporation for Atmospheric Research under sponsorship of the National Science.
Weathertop Consulting, LLC Wednesday, January 14, 2009 IIPS 11A.2 1 A General Purpose System for Server-side Analysis of Earth Science Data Roland Schweitzer.
Indo-US Workshop, June23-25, 2003 Building Digital Libraries for Communities using Kepler Framework M. Zubair Old Dominion University.
UWG 2013 Meeting PO.DAAC Web Services Demo. What are PO.DAAC Web Services?
Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.
Describe workflows used to maintain and provide the RDA to users – Both are 24x7 operations Transition to the NWSC with zero downtime NWSC is new environment.
NODC ↔ Data Consumers Steve Rutz NOAA/NESDIS National Oceanographic Data Center NODC Observing Systems Team Leader June 21, 2011.
VO Sandpit, November 2009 CEDA Metadata Steve Donegan/Sam Pepler.
Improved Access to RDA from the MSS OSD Executive Meeting April 28, 2009.
Federated Discovery and Access in Astronomy Robert Hanisch (NIST), Ray Plante (NCSA)
Unidata TDS Workshop THREDDS Data Server Overview
1 NASA CEOP Status & Demo CEOS WGISS-24 Oberpfaffenhofen, Germany October 15, 2007 Yonsook Enloe.
Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.
NODC Metadata Management for Geoportal Server and Beyond John Relph NOAA National Oceanographic Data Center.
THREDDS Catalogs Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007.
RDA Data Support Section. Topics 1.What is it? 2.Who cares? 3.Why does the RDA need CISL? 4.What is on the horizon?
1 NASA CEOP Final Summary CEOS WGISS-26 Boulder, Colorado September 23, 2008 Yonsook Enloe
TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva February, 2010 Doug Schuster Steven Worley Dave.
May 26-28ICNEE 2003 ARCHON: BUILDING LEARNING ENVIRONMENTS THROUGH EXTENDED DIGITAL LIBRARY SERVICES Hesham Anan, Kurt Maly, Mohammad Zubair,et al. Digital.
The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Steven Worley National Center for.
29 March 2004 Steven Worley, NSF/NCAR/SCD 1 Research Data Stewardship and Access Steven Worley, CISL/SCD Cyberinfrastructure meeting with Priscilla Nelson.
TIGGE Archive Status at NCAR THORPEX Workshop and 6th GIFS-TIGGE Working Group Meetings WMO Headquarters Geneva September 2008 Steven Worley Doug.
Discovery Environment Tool Integration High Level Overview.
The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
TIGGE Archive Access at NCAR Steven Worley Doug Schuster Dave Stepaniak Hannah Wilcox.
Research Data Archive (RDA) Access and Services from Yellowstone Grace Peng and Doug Schuster 1.
Display Page (HTML/CSS)
Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.
RDA Data Support Section. Topics 1.What is it? 2.Who cares? 3.Why does the RDA need CISL? 4.What is on the horizon?
1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)
5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.
AOLI 2015 The NMME Experience: A Research Community Archive Lessons learned from Climate Model data archive and use AOLI Meeting 2015 Eric Nienhouse NCAR.
Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.
The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven.
1. Gridded Data Sub-setting Services through the RDA at NCAR Doug Schuster, Steve Worley, Bob Dattore, Dave Stepaniak.
A41I-0105 Supporting Decadal and Regional Climate Prediction through NCAR’s EaSM Data Portal Doug Schuster and Steve Worley National Center for Atmospheric.
International Planetary Data Alliance Registry Project Update September 16, 2011.
The Earth System Curator Metadata Infrastructure for Climate Modeling Rocky Dunlap Georgia Tech.
TIGGE Archives and Access
TIGGE Data Archive and Access System at NCAR
WGISS-41: IDN Report Michael Morahan CEOS WGISS-41 Meeting
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System Zaihua Ji Doug Schuster Steven Worley Computational.
WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe
Steven Worley, Douglas Schuster,
CISL’s Research Data Archive (RDA) : Description and Methods
Data Management Components for a Research Data Archive
Robert Dattore and Steven Worley
Successful Data Curation for Large Data Archives
Presentation transcript:

Introduction What purpose does a data archive center serve if users can’t find or access the holdings they might need to facilitate their research discoveries? Removing the barriers that prevent discovery and access of data assets, by a diverse set of users, is a ongoing challenge facing data archive centers today. To manage this challenge, The NCAR Research Data Archive (RDA) employs a combination of in-house developed and standards based community discovery and access tools which support more than 10,000 users annually. The tools and user communities they serve are highlighted in this presentation. Workers remove a road closed barrier as a new bridge opens. Image courtesy “ System Design The RDA provides a variety of search, discovery, and access tools to serve a broad user community (Figure 1). Distributed searches and metadata harvesting are supported by an Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) access point, and a Geoportal server with a Catalog Services for the Web (CSW) access point. The in-house developed web interface provides keyword and faceted search options, and access to detailed dataset metadata. Once users find the data they want, several options are available to support data access. Data subset requests can be submitted through web interfaces, command line app clients, command line apps located on internal HPC systems, and Unidata THREDDS catalog supported web protocols. Data files and resulting subset request output files can be downloaded by server generated Wget and cURL scripts, or through the command line app clients. Acknowledgments The members of the NCAR Computational Inforamtion Systems Laboratory (CISL), Data Support Section (DSS) support the infrastructure and software described in this presentation. They include Joey Comeaux, Thomas Cram, Cecilia Banner, Bob Dattore, Zaihua Ji, Kevin Manross, Dave Stepaniak, and Chi-Fan Shih. Figure 1. RDA Data Discovery and Access System Usage Metrics User metrics display a continued growth in use of customized request access options (Figure 2), and fairly steady use of direct archive file downloads through the web interface (Figure 3). Figure 2. Monthly customized RDA user access from the web interface. Access options include data subset and file format conversion requests. Figure 3. Monthly RDA direct archive file download from the web interface. Wget and cURL download scripts drive this usage. Partial file downloads of user selected parameters are supported by the cURL based scripts. Once sufficient data are available, usage metrics will also be compiled for the THREDDS and command line apps/app clients data access methods. Douglas Schuster and Steven Worley CISL/DSS, National Center for Atmospheric Research, Boulder, CO, United States.,, Relevant Links RDA Geoportal instance RDA Geoportal CSW endpoint s&Service=CSW&Version=2.0.2 RDA OAI-PMH endpoint RDA THREDDS data server RDA command line app clients RDA HTTP PHP API description The NCAR Research Data Archive’s Hybrid Approach for Data Discovery and Access Currently/Month 1 PB Data Processed 30 TB Data Delivered 3000 Requests 500 Users Currently/Month 75 TB Data Accessed 40 TB Data Delivered 1500 Users