IRI Data Library Overview

Slides:



Advertisements
Similar presentations
Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.
Advertisements

1 The World Wide Web. 2  Web Fundamentals  Pages are defined by the Hypertext Markup Language (HTML) and contain text, graphics, audio, video and software.
1 Chapter 20 — Creating Web Projects Microsoft Visual Basic.NET, Introduction to Programming.
Distributed Data Analysis & Dissemination System (D-DADS) Prepared by Stefan Falke Rudolf Husar Bret Schichtel June 2000.
OPeNDAP and the Data Access Protocol (DAP) Original version by Dave Fulker.
14th IEEE-NPSS Real Time Conference 2005, 8 June Stockholm.
Tutorial 1 Getting Started with Adobe Dreamweaver CS3
Internet Concept and Terminology. The Internet The Internet is the largest computer system in the world. The Internet is often called the Net, the Information.
Unidata’s TDS Workshop TDS Overview – Part II October 2012.
The IRI Climate Data Library: translating between data cultures Benno Blumenthal International Research Institute for Climate Prediction Columbia University.
HTML. Principle of Programming  Interface with PC 2 English Japanese Chinese Machine Code Compiler / Interpreter C++ Perl Assembler Machine Code.
IRI Data Library: enhancing accessibility of climate knowledge M. Benno Blumenthal, Michael Bell, John del Corral, Rémi Cousin, and Igor Khomyakov.
Tracker data quality monitoring based on event display M.S. Mennea – G. Zito University & INFN Bari - Italy.
+ Information Systems and Databases 2.2 Organisation.
Application Layer Honolulu Community College Cisco Academy Training Center Semester 1 Version
Semantic Technologies and Application to Climate Data M. Benno Blumenthal IRI/Columbia University CDW /04-01.
Remote Data Access with OPeNDAP Dr. Dennis Heimbigner Unidata netCDF Workshop October 25, 2012.
Server-side Analysis and a Semantic Framework for Metadata M. Benno Blumenthal International Research Institute for Climate and Society Columbia University.
1 Web Servers (Chapter 21 – Pages( ) Outline 21.1 Introduction 21.2 HTTP Request Types 21.3 System Architecture.
Building Dashboards SharePoint and Business Intelligence.
Information Technology: GrADS INTEGRATED USER INTERFACE Maps, Charts, Animations Expressions, Functions of Original Variables General slices of { 4D Grids.
Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.
Lesson No:12 Introduction to Internet CHBT-01 Basic Micro process & Computer Operatio.
Data Interoperability at the IRI: translating between data cultures Benno Blumenthal International Research Institute for Climate Prediction Columbia University.
Weathertop Consulting, LLC Server-side OPeNDAP Analysis – Concrete steps toward a generalized framework via a reference implementation using F-TDS Roland.
Website Design, Development and Maintenance ONLY TAKE DOWN NOTES ON INDICATED SLIDES.
ERDDAP The Next Generation of Data Servers Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA Disclaimer: The opinions expressed.
IRI/LDEO Climate Data Library M.Benno Blumenthal, Michael Bell, John del Corral, Remi Cousin, and Haibo Liu International Research Institute for Climate.
Semantic Web underpinnings of the IRI Data Library Semantic Web as a Framework for Multiple Metadata IRI Data Library: presenting Data in multiple frameworks.
Internet  ’60 = an invention of the US army  Universities and libraries also start to use this communication tool  Protocol + physical network=> backbone.
M.Benno Blumenthal and John del Corral International Research Institute for Climate and Society Using a Resource.
OGC Web Services with complex data Stephen Pascoe How OGC Web Services relate to GML Application Schema.
Internet Searching the World Wide Web. The Internet and the World Wide Web The Internet is a worldwide collection of networks that allows people to communicate.
(class #2) CLICK TO CONTINUE done by T Batchelor.
IRI/LDEO Climate Data Library M.Benno Blumenthal, Michael Bell, and John del Corral International Research Institute for Climate and Society Columbia University.
The World Wide Web.
Data Browsing/Mining/Metadata
INTRODUCTION TO DATABASES (MICROSOFT ACCESS)
Application Layer Honolulu Community College
Other Services in Hyrax
Chapter 1: Introduction
The IRI was founded approximately 12 years ago with a mission To enhance society’s capability to understand, anticipate and manage the impacts of climate.
mysql and mysql workbench
Introduction Web Environments
Web Engineering.
BTEC NCF Dip in Comp - Unit 15 Website Development Lesson 12 – Publish and Test Mr C Johnston.
IRI Data Library: enhancing accessibility of climate knowledge
What’s New in Colectica 5.3 Part 1
IRI/LDEO Climate Data Library
IRI/LDEO Climate Data Library
IRI/LDEO Climate Data Library
Dick M.A. Schaap – Technical Coordinator SeaDataNet Training Workshop
IRI/LDEO Climate Data Library
IRI/LDEO Climate Data Library
IRI Data Library Overview
Remote Data Access Update
Information Technology Ms. Abeer Helwa
Data Standards at the IRI Data Library
INFO/CSE 100, Spring 2006 Fluency in Information Technology
Accessing Remote Datasets through the netCDF interface.
Teaching slides Chapter 6.
Database Connectivity and Web Development
IN32A-05 The IRI/LDEO Climate Data Library: Helping People use Climate Data M.Benno Blumenthal, Emily Grover-Kopec, Michael Bell, and John del Corral.
ExPLORE Complex Oceanographic Data
Chapter 1: Introduction
IRI/LDEO Climate Data Library
IRI Data Library efforts focus on making climate and other data products more widely accessible through tool development, data organization and transformation,
Chapter 1: Introduction
OPeNDAP/Hyrax Interfaces
The W3C More on images Entity codes Remote vs Local Computers
Presentation transcript:

IRI Data Library Overview Specialized Data Tools Maproom Generalized Data Tools Data Viewer Data Language Dataset Variable ivar multidimensional So here is the IRI/LDEO Data Library shot at this, connecting the space of data and data manipulations. At the bottom we have the compute engine/data organization, which is what maps the data/manipulation space into URLs, i.e. the WWW. Built on top of that are some general data tools, i.e. they can be applied to any dataset and adapt accordingly. There is a data language, making it possible to specify sophisticated analy ses. And there is a data viewer, making it possible to quickly graph data in a number of standard ways. And there are also more specialized tools, designed for particular audiences to view specific things. We have a Maproom (soon to be or already map rooms) which contains continuously updated views of aspects of the climate system, as well as specialized tools aimed at particular audiences that let a use extract views/data with a few clicks. There is a tradeoff here: the general tools are great, but require a user to navigate a vast set of datasets and a vast set of possible manipulations, which not everybody is up to. The specialized tools are a way to make sophisticated c alculations easy to access. IRI Data Collection URL/URI for data, calculations, figs, etc

descriptive and navigational pages IRI Data Collection GRIB netCDF images binary GIS spreadsheets “geolocated by lat/lon” shapefiles Database Tables queries “geolocated by entity” Servers OPeNDAP THREDDS images w/proj IRI Data Collection Dataset Variable ivar “geolocated by vector object” Calculations “virtual variables” images graphics descriptive and navigational pages Having got all that data into the data library, we can process it in a uniform way. The data structure leads directly to calculations and “virtual variables”, i.e. many of the Data Library entries are actually calculations done on other entities, e.g. PressureLevel data zonal velocities computed from hybrid level divergence and vorticity, or sea surface temperature anomaly computed from sea surface temperature and sea surface temperature climatology User Interface – the data collection structure and metadata is used to generate a web interface to the data – provides navigation through the datasets, a viewer that slices and dices, many manipulations and calculations. We also generate output Data Files in many different formats, as well as tables of various kinds, many of which are useful to one kind of user or another, i.e. different data cultures have different preferred formats. Atm/Ocean like netcdf and straight binary, GIS prefers sets of images, Public Health prefers tables. We also act as a data server using OpenDAP and THREDDS, again mostly useful for Ocean/Atm We have perhaps implemented OpenGIS Web Map Server v1.3 – a bit of a mistake, since v1.3 is the next version rather than the currently widely used one. Time will correct this, with any luck. It is important to note that everything following from the structure and attributes in the IRI Data Collection, no additional configuration is done to control the conversions to different fromats or to serve the data with different protocols. Not all data can be served in all formats. OpenDAP/THREDDS is particularly important because it can express any dataset and/or any analysis, so that I can transfer calculations between servers. At least, it will once I code transmission of SimpleFeatures with OpenDAP. Clients OPeNDAP THREDDS Data Files netcdf binary images Tables OpenGIS WMS/WCS