Jan. 2006 Storage Resource Broker Managing Distributed Data in a Grid A discussion of a paper published by a group of researchers at the San Diego Supercomputer.

Slides:



Advertisements
Similar presentations
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.
Advertisements

OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center.
The Storage Resource Broker and.
The Storage Resource Broker and.
Peter Berrisford RAL – Data Management Group SRB Services.
Digital Preservation Lifecycle Management Building a demonstration prototype for the preservation of large-scale multi-media collections Arcot Rajasekar.
ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.
NATIONAL PARTNERSHIP FOR ADVANCED COMPUTATIONAL INFRASTRUCTURE SAN DIEGO SUPERCOMPUTER CENTER Particle Physics Data Grid PPDG Data Handling System Reagan.
San Diego Supercomputer CenterNational Partnership for Advanced Computational Infrastructure1 Grid Based Solutions for Distributed Data Management Reagan.
A Very Brief Introduction to iRODS
Chronopolis: Preserving Our Digital Heritage David Minor UC San Diego San Diego Supercomputer Center.
Applying Data Grids to Support Distributed Data Management Storage Resource Broker Reagan W. Moore Ian Fisk Bing Zhu University of California, San Diego.
Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.
UMIACS PAWN, LPE, and GRASP data grids Mike Smorul.
On Developing Data Grid Workflows using Storage Resource Broker (SRB) and Kepler Tim H. Wong - UC Davis Efrat Frank - SDSC Dr. Bertram Ludäscher - UC Davis.
Cloud based linked data platform for Structural Engineering Experiment Xiaohui Zhang
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Section 13.1 Add a hit counter to a Web page Identify the limitations of hit counters Describe the information gathered by tracking systems Create a guest.
Management Information Systems, 4 th Edition 1 Chapter 8 Data and Knowledge Management.
Chapter 16 – DNS. DNS Domain Name Service This service allows client machines to resolve computer names (domain names) to IP addresses DNS works at the.
12 1 Chapter 12 Distributed Database Management Systems Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
Database Architecture Introduction to Databases. The Nature of Data Un-structured Semi-structured Structured.
MCAT: A Metadata Catalog San Diego Supercomputing Center Part of the Storage Resource Broker (SRB)
1999 Asian Women's Network Training Workshop What the Internet Offers Communications  Across the country or across the world Information resources and.
Rule-Based Data Management Systems Reagan W. Moore Wayne Schroeder Mike Wan Arcot Rajasekar {moore, schroede, mwan, {moore, schroede, mwan,
San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center National Partnership for Advanced.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Data Grid Services/SRB/SRM & Practical Hai-Ning Wu Academia Sinica Grid Computing.
Unit – I CLIENT / SERVER ARCHITECTURE. Unit Structure  Evolution of Client/Server Architecture  Client/Server Model  Characteristics of Client/Server.
RELATIONAL FAULT TOLERANT INTERFACE TO HETEROGENEOUS DISTRIBUTED DATABASES Prof. Osama Abulnaja Afraa Khalifah
Session-8 Data Management for Decision Support
Production Data Grids SRB - iRODS Storage Resource Broker Reagan W. Moore
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Mainframe (Host) - Communications - User Interface - Business Logic - DBMS - Operating System - Storage (DB Files) Terminal (Display/Keyboard) Terminal.
1 4/23/2007 Introduction to Grid computing Sunil Avutu Graduate Student Dept.of Computer Science.
Part Four: The LSC DataGrid Part Four: LSC DataGrid A: Data Replication B: What is the LSC DataGrid? C: The LSCDataFind tool.
Rule-Based Preservation Systems Reagan W. Moore Wayne Schroeder Mike Wan Arcot Rajasekar Richard Marciano {moore, schroede, mwan, sekar,
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Archive for the NSDL Reagan W. Moore Charlie Cowart.
Oracle's Distributed Database Bora Yasa. Definition A Distributed Database is a set of databases stored on multiple computers at different locations and.
Chapter 8 Data and Knowledge Management. 2 Learning Objectives When you finish this chapter, you will  Know the difference between traditional file organization.
INTRODUCTION TO DBS Database: a collection of data describing the activities of one or more related organizations DBMS: software designed to assist in.
Distributed database system
Management Information Systems, 4 th Edition 1 Chapter 8 Data and Knowledge Management.
The Global Land Cover Facility is sponsored by NASA and the University of Maryland.The GLCF is a founding member of the Federation of Earth Science Information.
Application Development
1 e-Science AHM st Aug – 3 rd Sept 2004 Nottingham Distributed Storage management using SRB on UK National Grid Service Manandhar A, Haines K,
Introduction to The Storage Resource.
E-Curator: A Web-based Curatorial Tool Ian Brown, Mona Hess Sally MacDonald, Francesca Millar Yean-Hoon Ong, Stuart Robson Graeme Were UCL Museums & Collections.
SDSC Storage Resource Broker & Meta-data Catalog SRB Archives HPSS, ADSM, UniTree, DMF Databases DB2, Oracle, Sybase File Systems Unix, NT, Mac OSX Application.
Partnerships in Innovation: Serving a Networked Nation Grid Technologies: Foundations for Preservation Environments Portals for managing user interactions.
National Archives and Records Administration1 Integrated Rules Ordered Data System (“IRODS”) Technology Research: Digital Preservation Technology in a.
Rights Management for Shared Collections Storage Resource Broker Reagan W. Moore
The Storage Resource Broker and.
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
Collection-Based Persistent Archives Arcot Rajasekar, Richard Marciano, Reagan Moore San Diego Supercomputer Center Presented by: Preetham A Gowda.
Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.
Enabling Grids for E-sciencE EGEE-II INFSO-RI Status of SRB/SRM interface development Fu-Ming Tsai Academia Sinica Grid Computing.
1 eScience Grid Environments th May 2004 NESC - Edinburgh Deployment of Storage Resource Broker at CCLRC for E-science Projects Ananta Manandhar.
Data Grids, Digital Libraries and Persistent Archives: An Integrated Approach to Publishing, Sharing and Archiving Data. Written By: R. Moore, A. Rajasekar,
The Data Grid: Towards an architecture for Distributed Management
Cloud based linked data platform for Structural Engineering Experiment
Policy-Based Data Management integrated Rule Oriented Data System
DUCKS – Distributed User-mode Chirp-Knowledgeable Server
Arcot Rajasekar Michael Wan Reagan Moore (sekar, mwan,
Distributed Systems Bina Ramamurthy 11/30/2018 B.Ramamurthy.
Distributed Systems Bina Ramamurthy 12/2/2018 B.Ramamurthy.
Tiers vs. Layers.
Distributed Systems Bina Ramamurthy 4/22/2019 B.Ramamurthy.
Presentation transcript:

Jan Storage Resource Broker Managing Distributed Data in a Grid A discussion of a paper published by a group of researchers at the San Diego Supercomputer Center, UCSD By: Nayef TRAD INSA-Lyon

2Jan PLAN What is SRB? SRB Features & Capabilities SRB Features & Capabilities SRB communication in conjonction with MCAT Federated SRB Zones Why Federation? Why Federation? Cons & Pros

3Jan What is SRB? SRB in a Nutshell A data grid solution A digital library A distributed but unified file system An advanced archival system A semantic Web A five nines availability resource broker

4Jan What is SRB? Features In depth The SRB is a client-server middleware that uses datasets to build a logical name space for referencing distributed data. The SRB provides a way to organize information stored on multiple heterogeneous systems into logical entities for ease of use. The SRB provides access to data stored on archival resources, file systems and databases. The SRB also provides capabilities to store replicas of data, controlling and auditing access to documents and collections, which is also known by logical-to-physical mapping hence Metadata Cataloging The SRB also provides a facility for co-locating data together using containers with more flexibility in accessing and updating files and stores user-defined metadata at different levels for better search capabilities.

5Jan What is SRB? SRB specific capabilities Support for the execution of remote operations. Access controls and audit trails to control and track data usage ( MCAT ) Interacting with multiple types of storage systems, for its abstraction User-friendly access through many ways like shells, scripts, libraries and web browsers

6Jan SRB & MCAT

7Jan SRB communication User SRB Server A SRB Server B Requesting a file 1 2 MES Contacting MES DB Server 3 Interpreting queries into SQL statements 4 Returning DB query 5 Supplying the location of the file 6 Contacting hosting server 7 Sending requested file

8Jan Federated SRB Zones DB SRB Zone A Zone B SRB DB Trust Relation SRB Zones recognize the presence of each others and is able to interact, despite the independency of each zone which is forming a federation. The overall federation is always viewed as a single logical namespace.

9Jan Federated SRB Zones Why Federation? Federation enables fragmentation of MCAT DataBase When a system failure occurs that would bring down an MCAT fragment, but shouldn’t affect the entire SRB, as the other sections will isolate the temporarily inaccessible server. When a system failure occurs that would bring down an MCAT fragment, but shouldn’t affect the entire SRB, as the other sections will isolate the temporarily inaccessible server. Federation permits MetaData to reside closer to the resource To reduce network latency, an MCAT server is hosted closer to the resource; in turn the SRB performance improves on a network span basis. To reduce network latency, an MCAT server is hosted closer to the resource; in turn the SRB performance improves on a network span basis. Federation enables the Virtual Organizations notion and makes it easier to merge newbies to the network, not to forget the ability of transferring and replicating documents across VOs

10Jan Cons & Pros Cons That paper can be considered as a fine support for understanding the Data Grids Architecture. Experiencing such enormous projects describes the powerful and huge capabilities of an SRB. Pros Absence of citation of some important points, like SRB federation and MCAT. Redundancy of similar ideas and information along the examples discussed.

11Jan Some Useful References

12Jan Here come troubles… Show me some MERCY while SHOOTING your questions Thank you !