NERC DataGrid: Googling for Secure Data

Slides:

Advertisements

Similar presentations

CLADDIER project fundamentals Citation, Location and Deposition in Discipline and Institutional Repositories Sam Pepler Project Manager BADC CLADDIER workshop,

Advertisements

UKOLN is supported by: The JISC Information Environment Metadata Schema Registry (IEMSR): Update DC-2006, Manzanillo, Mexico October 3-6, 2006 Rachel Heery.

1 UK e-Science All-Hands Meeting Nottingham, 2004 Enterprise specification of the NERC DataGrid Andrew Woolf, Ray Cramer.

BADC, BODC, CCLRC, PML and SOC NDG Security: Distributed Governance, Distributed Access Control, Distributed Data [ ]= Bryan Lawrence (on behalf.

SeaDataNet Web Services Roy Lowry British Oceanographic Data Centre SeaDataNet Training Course.

Information Modelling MOLES Metadata Objects for Linking Environmental Sciences S. Ventouras Rutherford Appleton Laboratory.

Parameter Vocabularies in the NERC DataGrid Project Presented by Roy Lowry Roy Lowry British Oceanographic Data Centre on behalf of the on behalf of the.

MEDIN Standards M. Charlesworth and the MEDIN Standards Working Group.

Connect. Communicate. Collaborate Click to edit Master title style MODULE 1: perfSONAR TECHNICAL OVERVIEW.

Distributed components

CAS2K3, September 2003 BADC: badc.nerc.ac.uk, NERC DataGrid: The NERC DataGrid – Building Bridges for the Environmental Sciences.

Bryan Lawrence on behalf of BADC, BODC, CCLRC, PML and SOC The British Atmospheric Data Centre and the NERC DataGrid (for) [ ]=

The MashMyData project Combining and comparing environmental science data on the web Alastair Gemmell 1, Jon Blower 1, Keith Haines 1, Stephen Pascoe 2,

CHIME: A Metadata-Based Distributed Software Development Environment Stephen E. Dossick Dept. of Computer Science Columbia University

Integrating SeaDOG into RODIN T. Le Bas, F. Venuti, H. Snaith, and B. Soall.

NERC Data Grid Helen Snaith and the NDG consortium …

Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.

BADC, BODC, CCLRC, PML and SOC The NERC Metadata Gateway: a product of the NERC DataGrid [ ]= Bryan Lawrence (on behalf of a big team)

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

VO Sandpit, November 2009 Metadata for Data Discovery: The NERC Data Catalogue Service Steve Donegan.

The NERC DataGrid Vocabulary Server Roy Lowry British Oceanographic Data Centre Ontology Registry Meeting.

Bryan Lawrence on behalf of BADC, BODC, CCLRC, PML and SOC An Introduction to NDG concepts [ ]=

CF Conventions Support at BADC Alison Pamment Roy Lowry (BODC)

GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.

1 The NERC DataGrid DataGrid The NERC DataGrid DataGrid AHM 2003 – 2 Sept, 2003 e-Science Centre Metadata of the NERC DataGrid Kevin O’Neill CCLRC e-Science.

1 NESC workshop Grid and Geospatial Standards 7-Sep-2005 Data integration with the Climate Science Modelling Language Andrew Woolf 1, Bryan Lawrence 2,

NOCS, PML, STFC, BODC, BADC The NERC DataGrid = Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC.

The Grid System Design Liu Xiangrui Beijing Institute of Technology.

NERC DataGrid NERC DataGrid Vocabulary Server Use Cases Vocabulary Workshop, RAL, February 25, 2009.

A Flexible Component based Access Control Architecture for OPeNDAP Services Philip Kershaw STFC Rutherford Appleton Laboratory.

Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Code Applications Tamas Kiss Centre for Parallel.

BADC, BODC, CCLRC, PML and SOC Interacting with NDG [ ]= Bryan Lawrence (on behalf of a big team)

TPAC Tasmanian Partnership for Advanced Computing Partner in APAC (Australian Partnership for Advanced Computing) Expertise centre for Earth Systems Science.

NDG Discovery Gateway ISO19139 Issues Bryan Lawrence Director of Environmental Data Archival and Associated Research, CCLRC Head of the British Atmospheric.

The NERC DataGrid Prototype Bryan Lawrence 2, Ray Cramer 3, Marta Gutierrez 2, Kerstin Kleese van Dam 1, Siva Kondapalli 3, Susan Latham 2, Roy Lowry 3,

Practical Access Control Using NDG Security e-Science All Hands Meeting 11 September 2007 Philip Kershaw BADC Bryan Lawrence BADC Jon Blower ESSC.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

1 GRID Based Federated Digital Library K. Maly, M. Zubair, V. Chilukamarri, and P. Kothari Department of Computer Science Old Dominion University February,

COWS CEDA OGC Web Services Framework Stephen Pascoe.

AUKEGGSWorkshop ANU, Canberra, 29 November 2006 Implementing CSML Feature Types in applications within the NERC DataGrid Dominic Lowe, British Atmospheric.

Cole David Ronnie Julio. Introduction Globus is A community of users and developers who collaborate on the use and development of open source software,

Mercury – A Service Oriented Web-based system for finding and retrieving Biogeochemical, Ecological and other land- based data National Aeronautics and.

Alison Pamment 1, Steve Donegan 1, Calum Byrom 2, Oliver Clements 3, Bryan Lawrence 1, Roy Lowry 3 1 NCAS/BADC, Science and Technology Facilities Council,

Earth System Curator and Model Metadata Discovery and Display for CMIP5 Sylvia Murphy and Cecelia Deluca (NOAA/CIRES) Hannah Wilcox (NCAR/CISL) Metafor.

GO-ESSP LLNL, June 2006 CSML – Stocktake and Forward Look Andrew Woolf Dominic CCLRC Rutherford Appleton Laboratory.

1 AHM, 2–4 Sept 2003 e-Science Centre GRID Authorization Framework for CCLRC Data Portal Ananta Manandhar.

Bryan Lawrence on behalf of the NDG, BADC and BODC. Ray Cramer, Marta Gutierrez, Kerstin Kleese, Siva Kondapalli, Sue Latham, Roy Lowry, Kevin O’Neill,

British Atmospheric Data Centre ( Searching: Whither NDG? Bryan Lawrence.

NESC Worshop – 07 September 2005 Development of a Marine Metadata Standard Greg Reed Executive Officer Australian Ocean Data Centre Joint Facility.

1 SIMDAT Simdat Project –GTD. Meteo Activity – SIMDAT Meteo Activity OGF June 2008 Barcelona Marta Gutierrez, Baudouin Raoult, Cristina.

Holding slide prior to starting show. Lessons Learned from the GECEM Portal David Walker Cardiff University

1 Alison Pamment, 2 Calum Byrom, 1 Bryan Lawrence, 3 Roy Lowry 1 NCAS/BADC,Science and Technology Facilities Council, 2 Tessella plc, 3 British Oceanogrphic.

Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.

A Collaborative e-Science Architecture towards a Virtual Research Environment Tran Vu Pham 1, Dr. Lydia MS Lau 1, Prof. Peter M Dew 2 & Prof. Michael J.

The AstroGrid-D Information Service Stellaris A central grid component to store, manage and transform metadata - and connect to the VO!

ECMWF 24 th November 2008 Deploying secure OGC services in front of a heterogeneous data archive. Bryan Lawrence, Phil Kershaw, Dominic Lowe, and Stephen.

Grid Services for Digital Archive Tao-Sheng Chen Academia Sinica Computing Centre

Beyond simple features: Do complex feature types need complex service descriptions?' B.N. Lawrence (1,2), D. Lowe (1,2), S. Pascoe (1,2) and A. Woolf (1).

Practical Access Control Using NDG Security

DELIVERING ENVIRONMENTAL WEB SERVICES (DEWS)

An Approach to Software Preservation

TRSS Terminology Registry Scoping Study

The NERC Metadata Gateway: a product of the NERC DataGrid

Attie Bioinformatics Server Redesign

UK e-Science OGSA-DAI November 2002 Malcolm Atkinson

Grid Portal Services IeSE (the Integrated e-Science Environment)

BoF: VREs- Keith G Jeffery & Helen Glaves

BADC, BODC, CCLRC, PML and SOC

The JISC Core Middleware Call

Robert Dattore and Steven Worley

Presentation transcript:

NERC DataGrid: Googling for Secure Data Introduce myself and Wendy and ask them to contact her. Bryan Lawrence on behalf of the NDG, BADC and BODC. Ray Cramer, Marta Gutierrez, Kerstin Kleese, Siva Kondapalli, Sue Latham, Roy Lowry, Kevin O’Neill, Ag Stephens, Andrew Woolf British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Outline NDG Aims and Metadata Taxonomy Demonstration of NDG in action NDG Authorisation – the security bit! Status British Atmospheric Data Centre http://badc.nerc.ac.uk

Timelines & Bottom Line 2002: E-science arrives at NERC: Legacy Systems with millions of files and terabytes of data and existing access and authorisation systems that cannot easily be replaced. Complex existing DISCOVERY metadata systems. Discovery (where it exists) based on Z39.50 Utilisation based on file retrieval. 2004: NERC DataGrid ready to move forward New metadata systems describe data as well as datasets. OAI based harvesting supports scalable FAST data discovery. Requirements capture for new authorisation systems complete, and coding underway for implementation. New communities involved, and international discovery very close to operational reality. 2005: Utilisation based on metadata, on demand server side behaviours, grid-based back end parallelisation etc British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Complexity + Volume + Remote Access = Grid Challenge British Atmospheric Data Centre Simulations Assimilation British Oceanographic Data Centre http://ndg.nerc.ac.uk British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre NDG Metadata Taxonomy British Atmospheric Data Centre http://badc.nerc.ac.uk

NDG Metadata Architecture Service based model: clear separation between discovery and use discovery service standards compliant and interoperable British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre (D) - Discovery Open Archives Initiative – Digital Library Protocol for harvesting metadata. NDG Supports Multiple Discovery Services – “build your own” OAI Multiple Protocol Support will be built into the “NDG Vanilla Discovery Service” OAI British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Discovery British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Can order responses by title or data centre (or default random) Choose to go to A service or B service. Flexible Information Return Look at DIFs in either HTML or XML British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Current Interface British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Background activity being parallelised with GODIVA/CCLRC e-science collaboration (spectral -> gridpoint + CDMS + visualisation tools) Download either plot or the data that went into the plot. British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre http://badc.nerc.ac.uk

International Dimension British Atmospheric Data Centre http://badc.nerc.ac.uk

Southampton Oceanography Centre British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Authorisation Role-based access: <dataset> <host> badc.nerc.ac.uk </host> <name>ukmo-obs </name> <access-requires> researcher <access-requires> <access-requires> ukmo-obs </access-requires> <processing-requires> nerc </processing-requires> </dataset> Key concept: Only hosts that trust each other share data, even within a larger virtual organisation: e.g. at BADC: <trusted> <bodc> <host>ndg.bodc.nerc.ac.uk</host> <attribute remotename=”nerc”> nerc </attribute> <attribute remotename=”ashoe”> ashoe </attribute> <attribute remotename=”staff”> nerc </attribute> <other> bodc </other> </bodc> </trusted> Signed “conditions of use” form exists for this dataset British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre NDG Security Certificate based, pass encrypted credentials between user and gatekeeper. British Atmospheric Data Centre http://badc.nerc.ac.uk

British Atmospheric Data Centre Where are we? Migration to web services underway for some components, new A services in design phase, implementation details not yet obvious (e.g. GT4 etc). Major effort on defining feature types for observation types so we can build an OGC/ISO compatible data extractor for observations and numerical data. Security Infrastructure Development Collaboration with CCLRC e-science, ECOGrid Ongoing work on metadata definition and population: Oceanographic data Atmospheric Chemistry data Major issues with (un)controlled vocabularies Numerical Modelling data DIF numerical definition (moving to ISO), BADC and UK Community Katherine Bouton’s work at NCAS/CGAM (“B” MODEL METADATA) Remote Sensing Data Collaboration with NEODC and PML Ongoing work on databases and interfaces, DIF to ISO and “B” British Atmospheric Data Centre http://badc.nerc.ac.uk