Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.

Similar presentations


Presentation on theme: "Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process."— Presentation transcript:

1 Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process for researchers who would rather be spending that time finding answers to research questions. To assist more than 10K researchers annually through this process, the NCAR Research Data Archive (RDA) employs a combination of in-house developed and standards based access tools, including a recently deployed Unidata THREDDS Data Server (TDS). This presentation highlights the services provided through TDS and the expected benefits for end users. System Background The RDA provides a variety of access tools to serve a broad user community. Data subset requests can be submitted through in-house developed web interfaces and command line app clients for asynchronous processing. Resulting output files are are made available for download once processing is completed. To complement the asynchronous processing functionality and enhance the user experience, the RDA has started implementing the Unidata TDS for use on highly valued gridded dataset collections. The TDS uses a variety of standards-based services, including OPeNDAP and OGC protocols. These allow users to access virtual (aggregated) data files remotely through a variety of client tools, without needing to download the data first. Client tools can also directly create temporal, parameter, and spatial subsets, analyses, and other products. A growing tool set is accessing RDA data through the TDS (Figure 1). Acknowledgments The members of the NCAR Computational Information Systems Laboratory (CISL), Data Support Section (DSS) support the infrastructure and software described in this presentation. They include Joey Comeaux, Thomas Cram, Cecilia Banner, Bob Dattore, Dave Stepaniak, Chi-Fan Shih, and Steve Worley. Figure 1. RDA TDS Services and Access Pathways User Benefits from RDA TDS services Data extraction across multi-file datasets using single service point directives. No need to download large data files to perform a “first- look” analysis Visualizations and resulting subsets are created on-the-fly Following visual discovery the data can be easily exported to a local netCDF data file for further examination Services are standards based, use your favorite tool Multiple data access service protocols are available including OPeNDAP and OGC Metadata provided in NCML and ISO standards Archive data format transparent to the end user THREDDS metadata can be harvested and searched by distributed tools Asynchronous Request Metrics User metrics display a continued growth in use of customized request access options (Figure 2). This trend supports the need for addition access options including TDS. Figure 2.. Monthly customized RDA user access from the RDA web interface. Access options include data subset and file format conversion requests. Once sufficient data are available, usage metrics will also be compiled for the TDS data access methods. Kevin Manross, Zaihua Ji, and Doug Schuster CISL/DSS, National Center for Atmospheric Research, Boulder, CO, United States.,, http://rda.ucar.edu Relevant Links RDA THREDDS Data Server http://rda.ucar.edu/thredds Unidata THREDDS Data Server software home page http://www.unidata.ucar.edu/software/thredds/current/tds/ ` OPeNDAP homepage http://www.opendap.org/ OPeNDAP capable client data access/analysis tools http://www.opendap.org/whatClients OGC Protocol homepage http://www.opengeospatial.org/ OGC capable client data access/analysis tools http://rda.ucar.edu/#apps_api_desc OPeNDAP/THREDDS Access to the NCAR Research Data Archive Currently/Month 1 PB Data Processed 30 TB Data Delivered 3000 Requests 500 Users


Download ppt "Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process."

Similar presentations


Ads by Google