TPAC Tasmanian Partnership for Advanced Computing Partner in APAC (Australian Partnership for Advanced Computing) Expertise centre for Earth Systems Science.

Slides:

Advertisements

Similar presentations

Australian Partnership for Advanced Computing Partners: Australian Centre for Advanced Computing and Communications (ac3) in New South Wales CSIRO Queensland.

Advertisements

National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.

ISPRS WG II/3 Chongjun YANG Sept th 2003, Thailand Co-Chair of ISPRS WG II/3 IRSA/Chinese Academy of Sciences.

The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.

1 OGC Web Services Kai Lin San Diego Supercomputer Center

Geospatial One-Stop A Federal Gateway to Federal, State & Local Geographic Data

APAC Grid Geosciences Project: Linking Academia, Industry and Government to HPC Robert Woodcock, CSIRO and pmd*CRC Ryan Fraser, IVEC and Curtin.

Australian Partnership for Advanced Computing “providing advanced computing and grid infrastructure for eResearch” Rhys Francis Manager, APAC grid program.

OPeNDAP at the National Oceanographic Data Center Status Lessons Learned Recommendations.

Exploring large marine datasets using an interactive website and Google Earth Jon Blower, Dan Bretherton, Keith Haines, Chunlei Liu, Adit Santokhee Reading.

Integrated Ocean Observing System Data Management and Communications March 2004 The US Integrated Ocean Observing System (IOOS) Plan for Data Management.

TPAC Tasmanian Partnership for Advanced Computing Partner in APAC (Australian Partnership for Advanced Computing) Expertise centre for Earth Systems Science.

Mike Smorul Saurabh Channan Digital Preservation and Archiving at the Institute for Advanced Computer Studies University of Maryland, College Park.

M.Lautenschlager (WDCC / MPI-M) / / 1 GO-ESSP at LLNL Livermore, June 19th – 21st, 2006 World Data Center Climate: Status and Portal Integration.

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

Unidata TDS Workshop THREDDS Data Server Overview October 2014.

1 Deployment of an LCG Infrastructure in Australia How-To Setup the LCG Grid Middleware – A beginner's perspective Marco La Rosa

© University of Reading 2008www.reading.ac.uk Reading e-Science Centre September 10, 2015 Integrating a Web Map Service into the THREDDS Data Server Jon.

GADS: A Web Service for accessing large environmental data sets Jon Blower, Keith Haines, Adit Santokhee Reading e-Science Centre University of Reading.

Presented by The Earth System Grid: Turning Climate Datasets into Community Resources David E. Bernholdt, ORNL on behalf of the Earth System Grid team.

Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.

CWIC/IDN synchronization Lingjun Kang Michael Morahan

Bryan Lawrence on behalf of BADC, BODC, CCLRC, PML and SOC An Introduction to NDG concepts [ ]=

Australian Oceans Distributed Active Archive Center Peter Turner, CSIRO/CAWCR IMOS is an initiative of the Australian Government being conducted as part.

IMOS Satellite Remote Sensing (including Ocean Colour) CMAR, CLW, UTas, BoM, CurtinU, GA, AIMS Edward King & SRS Team.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

ALCME: OAI at OCLC Jeffrey A. Young OCLC Online Computer Library Center, Inc.

Weathertop Consulting, LLC Wednesday, January 14, 2009 IIPS 11A.2 1 A General Purpose System for Server-side Analysis of Earth Science Data Roland Schweitzer.

Guten Tag Michael Morahan CEOS-WGISS 29 May 17-21, 2010 Bonn, Germany.

Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.

Indo-US Workshop, June23-25, 2003 Building Digital Libraries for Communities using Kepler Framework M. Zubair Old Dominion University.

Web services at TRFIC TRFIC has developed the Access Technologies to achieve its goals of interoperability and provide access to data and information on.

1 AJAX and Dapper: The Good, the Bad, and the Ugly Joe Sirott PMEL/NOAA.

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

Integrated Model Data Management S.Hankin ESMF July ‘04 Integrated data management in the ESMF (ESME) Steve Hankin (NOAA/PMEL & IOOS/DMAC) ESMF Team meeting.

ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.

1 Dapper and Argo Joe Sirott PMEL/NOAA. 2 What is Dapper? Web server that provides distributed access to in-situ data via OPeNDAP protocol Clients include.

VO Sandpit, November 2009 CEDA Metadata Steve Donegan/Sam Pepler.

WDC-MARE – World Data Center for Marine Environmental Sciences Data portal based on Open Archives Initiative Protocols and Apache Lucene Uwe Schindler,

Unidata TDS Workshop THREDDS Data Server Overview

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.

1 DAPPER: An OPENDAP Server for In-Situ Data Joe Sirott Donald W. Denbo, Willa H Zhu University of Washington PMEL/NOAA.

The CERA2 Data Base Data input – Data output Hans Luthardt Model & Data/MPI-M, Hamburg Services and Facilities of DKRZ and Model & Data Hamburg,

GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma

OAI Overview DLESE OAI Workshop April 29-30, 2002 John Weatherley

A Data Access Framework for ESMF Model Outputs Roland Schweitzer Steve Hankin Jonathan Callahan Kevin O’Brien Ansley Manke.

Information Technology: GrADS INTEGRATED USER INTERFACE Maps, Charts, Animations Expressions, Functions of Original Variables General slices of { 4D Grids.

Interoperability = Leverage + Collaboration  Chris Lynnes  GES DISC.

EMODnet Chemistry 2 Service Contract MARE/2012/10 S How to make EMODnet Chemistry fit for purpose at system level By Dick M.A. Schaap – Technical.

1 Accomplishments. 2 Overview of Accomplishments  Sustaining the Production Earth System Grid Serving the current needs of the climate modeling community.

Find Research Data b2find.eudat.eu B2FIND User Training How to find data objects and collections using EUDAT’s B2FIND This work is licensed.

April 2008ESG All-Hands meeting ESG Product Services Overview of components Issues in need of discussion Steve Hankin, NOAA/PMEL Roland Schweitzer, Weathertop.

LAS and THREDDS: Partners for Education Roland Schweitzer Steve Hankin Jonathan Callahan Joe Mclean Kevin O’Brien Ansley Manke Yonghua Wei.

The Proliferation of Metadata Standards and the Evolution of NASA’s Global Change Master Directory (GCMD) Standard for Uses in Earth Science Data Discovery.

IOOS Sea Surface Temperatures: Realizing Truly Distributed Data with Open Standards Presenter Kyle Draganov.

Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.

A Climate Data Portal Focused on realtime and retrospective in situ data Nancy Soreide, Don Denbo, Willa Zhu, PMEL Charles Sun, NODC Bernie Kilonsky, U.

Ideas on Opening Up GEOSS Architecture and Extending AIP-5 Wim Hugo SAEON.

THREDDS Data Server (TDS) and Data Discovery John Caron Unidata/UCAR May 15, 2006.

JCU Australian Marine Science Data Network.

Advanced Computing and Grid Infrastructure in Australia ISGC April 2005 Taiwan John O’Callaghan Executive Director Australian Partnership for.

Data Browsing/Mining/Metadata

CAP-378 and “Conhecer para não ignorar”

Distributed Marine Data System:

Data Discovery Boulder, CO May 15, 2006 Scott Ritz

Research Data Archives at NCAR

WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe

Comeaux and Worley, NSF/NCAR/SCD

Presentation transcript:

TPAC Tasmanian Partnership for Advanced Computing Partner in APAC (Australian Partnership for Advanced Computing) Expertise centre for Earth Systems Science (ESS) Provides expertise & educational programs, as well as HPC facilities Located at University of Tasmania, Hobart Partnership between: University of TasmaniaCSIRO Atmospheric Research CSIRO Marine ResearchAustralian Maritime College Australian Antarctic DivisionAntarctic Climate & Ecosystems CRC Bureau of Met. Research Centre Who is TPAC?

TPAC Overview  Brief overview of TPAC Grid Program  TPAC – NDG Federation  Development of LAS Web Map Service  Development of server-side analysis service (Steve Phipps)

TPAC ESS Grid Project  ESS Data Grid - making ESS datasets transparently available to users of the APAC National Grid - developing a range of grid-based services that allow users to better utilise these datasets  ESS Compute Grid porting ESS models/applications to the APAC National Grid

TPAC Analysis / Models  Discovery   Visualisation  Digital Repository  OPeNDAP  Crawler  APAC NF VPAC AC3 SAPAC IVEC Job/Data Management The Vision APAC National Grid Infrastructure APAC Grid

TPAC TPAC OPeNDAP Network APAC NF (Canberra) International IPCC model results (10-50Tb) TPAC 1/8 degree ocean simulations (7Tb) Met Bureau Research Centre (Melbourne) Near real-time LAPS analyses products (<1Gb) Sea- and sub-surface temperature products University of Tasmania (Hobart) NCEP2 (150Gb), WOCE3 Global (90Gb) Antarctic AWS (150Gb), Climate modelling (4Gb) ASPeCT sea-ice data (<1Gb) CSIRO Marine Research (Hobart) Ocean colour products & climatologies (1Tb) Satellite altimetry data (<1Gb) Argo float data (<1Gb), MSLA data (5Gb) CSIRO HPSC (Melbourne) IPCC CSIRO Mk3 model results (6Tb) AC3 Facility (Sydney) Land surface datasets

TPAC Portal: Main Page

TPAC Visualisation Portlet Data Retrieval Portlet Analysis/Model Portlet Web Map Service Web Processing Service Web Coverage Service OAI Library Search API (Java) Live Access Server (LAS) OPeNDAP Server GrADS Server/Models Meta data Crawler Digital Repository Gridsphere Portal Web Services Application Layer Data Layer Hardware Layer Compute Engine Config Meta data Discovery Portlet Grid Project: Current Status Grid Users GIS Users Traditional Users

TPAC TPAC – NDG Federation A data Grid linking climate data resources in Australia and the UK  Metadata harvesting protocol – Open Archives Initiative (OAI)  Metadata schema – Directory Interchange Format (DIF)  Deployed for TPAC data holdings  TPAC imposes no metadata standards re data holdings

TPAC Mapping TPAC Metadata to DIF DIF elementTPAC metadata Data_Center/Data_Center_Name/Short_Name[5][5]AU/UTAS/TPAC Data_Center/Data_Center_Name/Long_NameTasmanian Partnership for Advanced Computing Data_Center/Data_Center_URLhttp:// Data_Center/Data_Set_ID(Same as Entry_ID) TPAC_categoryId _datasetId Data_Center/Personnel/RoleData Center Contact Data_Center/Personnel/First_Name (first name) Data_Center/Personnel/Last_Name (last name) Data_Center/Personnel/ Data_Center/Personnel/Contact_Address/AddressGPO Box 80 Data_Center/Personnel/Contact_Address/CityHobart Data_Center/Personnel/Contact_Address/StateTAS Data_Center/Personnel/Contact_Address/Postal_Code7001 Data_Center/Personnel/Contact_Address/CountryAustralia Distribution/Distribution_Media[6][6]Online Internet (HTTP) Summary Related_URL/URL_Content_Type[7][7]DODS_DIR Related_URL/URL ***DIG LIB URL???*** Metadata_NameCEOS IDN DIF Metadata_Version9.4 DIF_Creation_DateDate of creation of DIF record. Last_DIF_Revision_DateDate of last revision. DIF_Revision_HistoryRevision history.

TPAC Mapping TPAC Metadata to DIF DIF elementTPAC metadata Entry_IDTPAC_categoryId _datasetId Entry_Title >> Data_Set_Citation/Dataset_CreatorTasmanian Partnership for Advanced Computing Data_Set_Citation/Dataset_Title Data_Set_Citation/Dataset_Series_Name Data_Set_Citation/Dataset_Release_Date Data_Set_Citation/Dataset_PublisherTasmanian Partnership for Advanced Computing Data_Set_Citation/Online_Resource Personnel/RoleTECHNICAL CONTACT Personnel/First_NameGlenn Personnel/Last_NameHyland Discipline/Discipline_Name[1][1]EARTH SCIENCE Parameters/{Category,Topic,Term,Variable}[2][2]GCMD valids best matching DODS variable ‘long_name’ attributes. Manual ISO_Topic_Category[3][3]Probably climatologyMeteorologyAtmosphere or oceans. Manual TemporalCoverage/Start_DateMay be possible to derive from DODS variables or attribute values. Manual TemporalCoverage/Stop_DateMay be possible to derive from DODS variables or attribute values. Manual Data_Set_ProgressComplete Spatial_CoverageFor some datasets, may be possible to derive from DODS variables or attribute values. Elements are optional if not relevant. Manual Location/Location_Name[4][4]Global Data_ResolutionMay be possible to derive from DODS variables or attribute values. Manual cont …

TPAC Discovery Results

TPAC Visualisation Portlet Data Retrieval Portlet Analysis/Model Portlet Web Map Service Web Processing Service Web Coverage Service OAI Library Search API (Java) Live Access Server (LAS) OPeNDAP Server GrADS Server/Models Meta data Crawler Digital Repository Gridsphere Portal Web Services Application Layer Data Layer Hardware Layer Compute Engine Config Meta data Discovery Portlet Grid Project: Current Status Grid Users GIS Users Traditional Users

TPAC LAS Architecture LAS data server back end application LAS UI server configuration metadata (XML) server data commands UI (java/script) browser LAS request (XML) products  The Live Access Server (LAS) is a highly configurable open-source Web server designed to provide flexible access to geo-referenced scientific data (mainly oceanographic and atmospheric datasets).

TPAC Server-side Visualisation Services

TPAC Server-side Visualisation Services

TPAC Server-side Visualisation Services

TPAC Server-side Visualisation Services

TPAC wrapper LAS – WMS Wrapper LAS data server back end application LAS UI+ server configuration metadata (XML) server data UI WMS Client WMS GetCap. LAS XML Capabilities XML LAS–WMS converter servlet 1.Maps from LAS metadata to the WMS ‘capabilities XML’ (i.e. metadata returned in response to a WMS GetCapabilities request).

TPAC wrapper LAS – WMS Wrapper LAS data server back end application LAS UI+ server configuration metadata (XML) server data commands+ UI WMS Client LAS request (XML) products WMS GetMap LAS–WMS converter servlet 2.Maps from a WMS GetMap request to a LAS data request.

TPAC Mapping WMS to LAS Requests  WMS request A WMS request for air temperature data for the COADS dataset would look like: 180,-90,180,190&WIDTH=800&HEIGHT=600&LAYERS=airt&STYLES=&FORMAT=image/gif  LAS request The corresponding LAS request would look like:

TPAC