Data Breakout. OGSA Architecture – databases Eldas, OGSA-DAI and GridMiner implement a slightly old version of OGSA / DAIS –Architecture doc describes.

Slides:



Advertisements
Similar presentations
Tom Sugden EPCC OGSA-DAI Future Directions OGSA-DAI User's Forum GridWorld 2006, Washington DC 14 September 2006.
Advertisements

© 2007 Open Grid Forum Data Management Challenge - The View from OGF OGF22 – February 28, 2008 Cambridge, MA, USA Erwin Laure David E. Martin Data Area.
E-Science Data Information and Knowledge Transformation Eldas Building Service Grids with Enterprise Level Data Access Services Alan Gray
Eldas 1.0 Enterprise Level Data Access Services Design Issues, Implementation and Future Development Davy Virdee.
Data Management Expert Panel - WP2. WP2 Overview.
An Overview of OGSA-DAI Kostas Tourlas
Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.
NextGRID & OGSA Data Architectures: Example Scenarios Stephen Davey, NeSC, UK ISSGC06 Summer School, Ischia, Italy 12 th July 2006.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Confidential ODBC May 7, Features What is ODBC? Why Create an ODBC Driver for Rochade? How do we Expose Rochade as Relational Transformation.
Understanding Active Directory
Database Environment 1.  Purpose of three-level database architecture.  Contents of external, conceptual, and internal levels.  Purpose of external/conceptual.
OGSA-DAI: Future Work and Wrap-up The OGSA-DAI Team
NAREGI WP4 (Data Grid Environment) Hideo Matsuda Osaka University.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
A Metadata Catalog Service for Data Intensive Applications Presented by Chin-Yi Tsai.
WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.
INFSO-RI Enabling Grids for E-sciencE gLite Data Management Services - Overview Mike Mineter National e-Science Centre, Edinburgh.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Information breakout. Things what we did What is the difference between a registry and a catalog? What do we mean by naming? What does OGSA define? Using.
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
© Geodise Project, University of Southampton, Data Management in Geodise Zhuoan Jiao, Jasmin Wason and Marc Molinari
San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center National Partnership for Advanced.
December 1, 2005HDF & HDF-EOS Workshop IX P eter Cao, NCSA December 1, 2005 Sponsored by NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration.
XML Registries Source: Java TM API for XML Registries Specification.
The LCG File Catalog (LFC) Jean-Philippe Baud – Sophie Lemaitre IT-GD, CERN May 2005.
EGEE Catalogs Peter Kunszt EGEE Data Management Middleware Service Grids NeSC, July 2004 EGEE is a project funded by the.
ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
OGSA-DAI in OMII-Europe Neil Chue Hong EPCC, University of Edinburgh.
OGSA-DAI-RDF & Its Ontology Interfaces Isao Kojima and Masahiro Kimoto Data Grid Team, Grid Technology Research Center
The Anatomy of the Grid Introduction The Nature of Grid Architecture Grid Architecture Description Grid Architecture in Practice Relationships with Other.
1 1 EPCC 2 Curtin Business School & Edinburgh University Management School Michael J. Jackson 1 Ashley D. Lloyd 2 Terence M. Sloan 1 Enabling Access to.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Globus Replica Management Bill Allcock, ANL PPDG Meeting at SLAC 20 Sep 2000.
OGSA-DAI.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
INFSO-RI Enabling Grids for E-sciencE OGSA DAI Data Access and Integration Marek Ciglan Institute of Informatics, Slovac Academy.
OGSA V1 Document (p19). Terminology What is a job? Job made of multiple tasks –Task: Single unit of work –Job: Multiple tasks GT2: DUROC: Link tasks with.
Mike Jackson EPCC OGSA-DAI Architecture + Extensibility OGSA-DAI Tutorial GGF17, Tokyo.
OGSA-DAI Neil Chue Hong 29 th January 2007 OGF19, Chapel Hill.
INFSO-RI Enabling Grids for E-sciencE gLite Data Management and Interoperability Peter Kunszt (JRA1 DM Cluster) 2 nd EGEE Conference,
Data Services (sample). OGSA cross WG discussion template2 Outline Requirements Key concepts/functionality Architecture/Model (if any) Services/portTypes.
Replica Management Kelly Clynes. Agenda Grid Computing Globus Toolkit What is Replica Management Replica Management in Globus Replica Management Catalog.
Introduction to The Storage Resource.
Shell Interface Shell Interface Functions Data. Graphical Interface Graphical Interface Command-line Interface Command-line Interface Experiments Private.
Managing Data DIRAC Project. Outline  Data management components  Storage Elements  File Catalogs  DIRAC conventions for user data  Data operation.
Data Manipulation with Globus Toolkit Ivan Ivanovski TU München,
1 Database Environment. 2 Objectives of Three-Level Architecture u All users should be able to access same data. u A user’s view is immune to changes.
AFS/OSD Project R.Belloni, L.Giammarino, A.Maslennikov, G.Palumbo, H.Reuter, R.Toebbicke.
1 OGSA-DAI: Service Grids Neil P Chue Hong. 2 Motivation  Access to data is a necessity on the Grid  The ability to integrate different data resources.
OGSA-DAI 简介及其它在 China-VO DAS 系统中的应用 杨阳 中国虚拟天文台研发团队 Chinese Virtual Observatory.
Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.
OGSA-DAI.
Amy Krause EPCC OGSA-DAI An Overview OGSA-DAI on OMII 2.0 OMII The Open Middleware Infrastructure Institute NeSC,
Maintaining and Searching Metadata Mario Antonioletti, Shannon Hastings, Peter Kunszt, Stephen Langella, Simon Laws, Susan Malaika, Gavin McCance, Alex.
Databases and DBMSs Todd S. Bacastow January 2005.
Data services on the NGS
Data Bridge Solving diverse data access in scientific applications
AMGA Web Interface Salvatore Scifo INFN sez. Catania
Data, Databases, and DBMSs
Distributed Systems Bina Ramamurthy 11/30/2018 B.Ramamurthy.
Distributed Systems Bina Ramamurthy 12/2/2018 B.Ramamurthy.
AMGA Web Interface Vincenzo Milazzo
Service Oriented Architecture (SOA)
Google Sky.
ACTIVE DIRECTORY An Overview.. By Karan Oberoi.
Distributed Systems Bina Ramamurthy 4/22/2019 B.Ramamurthy.
NIEM Tool Strategy Next Steps for Movement
Data Base.
Presentation transcript:

Data Breakout

OGSA Architecture – databases Eldas, OGSA-DAI and GridMiner implement a slightly old version of OGSA / DAIS –Architecture doc describes capabilities; DAIS explores details Current OGSA / DAIS spec does not include: –Extensibility (e.g. as used in e-Diamond) –Packing of multiple tasks and multiple data sets –Translation Definitely need (* as noted in OGSA doc): –Data connection / third-party delivery* –Unique names for identifying data content and data instance* –Robust and integrated architecture* –Model of metadata for databases Eldas and OGSA-DAI have OGSI, WSRF and WS-I interfaces planned –OGSA explicitly assumes WSRF –Some resistance to this; a federated approach suggested

OGSA Architecture – Files File Storage – GSM (SRM-like) Directory structure – GFS Replica Catalogs – OREP Metadata Catalogs –File stats (locking, mutability, ACLs) –User attributes API for access – ADF (POSIX-like) Definitely need: –Naming –Data Movement

Web Service Interfaces SRM is not WS-I compliant (yet) Eldas supports message-level encryption – https for Wed Services – WS-Security for Grid services OGSA-DAI requires XSLT Other standards that we have to adhere to –POSIX, Windows, NFS, FTP, Gzip, …

APIs: User commands Files: Explorer equivalent –Including ls, tail, find, … ? Need to prioritise –Quota management –ACLs Files: Directories –Familiar interface Databases: Query tool –OGSA-DAI and Eldas provide GUI tools –Quick query, schema inspection, etc –No existing standard Databases: DBA Tools –No existing standard

APIs: programming interfaces Files: SRM –POSIX-like Need to prioritise –plus management interface to handle scalable systems Files: Directories –Familiar interface Files: Catalogs –One-shot interface (because content unknown) –Bulk operations Databases: –“OGSA-DAI client API” E.g. simple query Add to SAGA or form DAIS API group? –Extensibility interface (client and server) –Bulk operation and bulk data interface (client and server)