RLS and DRS Roadmap Items Ann Chervenak Robert Schuler USC Information Sciences Institute.

Slides:



Advertisements
Similar presentations
Giggle: A Framework for Constructing Scalable Replica Location Services Ann Chervenak, Ewa Deelman, Ian Foster, Leanne Guy, Wolfgang Hoschekk, Adriana.
Advertisements

The Replica Location Service In wide area computing systems, it is often desirable to create copies (replicas) of data objects. Replication can be used.
Indications in green = Live content Indications in white = Edit in master Indications in blue = Locked elements Indications in black = Optional elements.
The State of the Art in Distributed Query Processing by Donald Kossmann Presented by Chris Gianfrancesco.
System integrity The term system integrity has the following meanings: That condition of a system where in its specified operational and technical parameters.
Transaction.
Introduction to Database Management  Department of Computer Science Northern Illinois University January 2001.
On Replication July 2006 Yin Chen. What is? Why need? Types? Investigation of existing technologies –IBM SQL replication –Sybase replication –Oracle replication.
Distributed DBMSs A distributed database is a single logical database that is physically distributed to computers on a network. Homogeneous DDBMS has the.
6/24/2015B.RamamurthyPage 1 File System B. Ramamurthy.
©Silberschatz, Korth and Sudarshan18.1Database System Concepts Centralized Systems Run on a single computer system and do not interact with other computer.
7/15/2015B.RamamurthyPage 1 File System B. Ramamurthy.
Distributed Databases
DATABASE MANAGEMENT SYSTEM ARCHITECTURE
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
The Data Replication Service Ann Chervenak Robert Schuler USC Information Sciences Institute.
Chapter 6: Integrity and Security Thomas Nikl 19 October, 2004 CS157B.
 Definition  Components  Advantages  Limitations Contents  DBMS DBMS  Functions Functions  Architecture Architecture.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)
Don Quijote Data Management for the ATLAS Automatic Production System Miguel Branco – CERN ATC
Globus Data Replication Services Ann Chervenak, Robert Schuler USC Information Sciences Institute.
A Metadata Catalog Service for Data Intensive Applications Presented by Chin-Yi Tsai.
04/18/2005Yan Huang - CSCI5330 Database Implementation – Distributed Database Systems Distributed Database Systems.
Information Systems: Databases Define the role of general information systems Describe the elements of a database management system (DBMS) Describe the.
The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Dataset Caitlin Minteer & Kelly Clynes.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
Database Systems: Design, Implementation, and Management Tenth Edition Chapter 12 Distributed Database Management Systems.
Database Systems: Design, Implementation, and Management Ninth Edition Chapter 12 Distributed Database Management Systems.
BaBar Data Distribution using the Storage Resource Broker Adil Hasan, Wilko Kroeger (SLAC Computing Services), Dominique Boutigny (LAPP), Cristina Bulfon.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Introduction to DFS. Distributed File Systems A file system whose clients, servers and storage devices are dispersed among the machines of a distributed.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE middleware: gLite Data Management EGEE Tutorial 23rd APAN Meeting, Manila Jan.
Ch 10 Shared memory via message passing Problems –Explicit user action needed –Address spaces are distinct –Small Granularity of Transfer Distributed Shared.
The Replica Location Service The Globus Project™ And The DataGrid Project Copyright (c) 2002 University of Chicago and The University of Southern California.
Distributed Database. Introduction A major motivation behind the development of database systems is the desire to integrate the operational data of an.
Replica Management Services in the European DataGrid Project Work Package 2 European DataGrid.
Transaction-based Grid Data Replication Using OGSA-DAI Presented by Yin Chen February 2007.
SkimData and Replica Catalogue Alessandra Forti BaBar Collaboration Meeting November 13 th 2002 skimData based replica catalogue RLS (Replica Location.
Wide Area Data Replication for Scientific Collaborations Ann Chervenak, Robert Schuler, Carl Kesselman USC Information Sciences Institute Scott Koranda.
DATABASE MANAGEMENT SYSTEM ARCHITECTURE
CEDPS Data Services Ann Chervenak USC Information Sciences Institute.
Website: Answering Continuous Queries Using Views Over Data Streams Alasdair J G Gray Werner.
A Data Stream Publish/Subscribe Architecture with Self-adapting Queries Alasdair J G Gray and Werner Nutt School of Mathematical and Computer Sciences,
Topic Distributed DBMS Database Management Systems Fall 2012 Presented by: Osama Ben Omran.
Replica Management Kelly Clynes. Agenda Grid Computing Globus Toolkit What is Replica Management Replica Management in Globus Replica Management Catalog.
Objective What is RFT ? How does it work Architecture of RFT RFT and OGSA Issues Demo Questions.
Rights Management in Globus Data Services Ann Chervenak, ISI/USC Bill Allcock, ANL/UC.
Object storage and object interoperability
ALCF Argonne Leadership Computing Facility GridFTP Roadmap Bill Allcock (on behalf of the GridFTP team) Argonne National Laboratory.
1 AHM, 2–4 Sept 2003 e-Science Centre GRID Authorization Framework for CCLRC Data Portal Ananta Manandhar.
Super Computing 2000 DOE SCIENCE ON THE GRID Storage Resource Management For the Earth Science Grid Scientific Data Management Research Group NERSC, LBNL.
1 Chapter 2 Database Environment Pearson Education © 2009.
© Oxford University Press 2011 DISTRIBUTED COMPUTING Sunita Mahajan Sunita Mahajan, Principal, Institute of Computer Science, MET League of Colleges, Mumbai.
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
1 Information Retrieval and Use De-normalisation and Distributed database systems Geoff Leese September 2008, revised October 2009.
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
Databases and DBMSs Todd S. Bacastow January 2005.
Database Management.
Data services on the NGS
Data services on the NGS
A Replica Location Service
Chapter 12 Information Systems.
Chapter 19: Distributed Databases
Data, Databases, and DBMSs
File System B. Ramamurthy B.Ramamurthy 11/27/2018.
Database Systems Instructor Name: Lecture-3.
Outline Announcements Lab2 Distributed File Systems 1/17/2019 COP5611.
Outline Review of Quiz #1 Distributed File Systems 4/20/2019 COP5611.
The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets A.Chervenak, I.Foster, C.Kesselman, C.Salisbury,
Presentation transcript:

RLS and DRS Roadmap Items Ann Chervenak Robert Schuler USC Information Sciences Institute

Replication Services l Replica Location Service (RLS) u Maintains mappings between logical file names and physical locations u Associates attributes with mappings l Data Replication Service (DRS) u Combines RLS cataloguing with data replication operations performed by Reliable File Transfer Service (RFT)

RLS Roadmap: Pluggable RLS Backend l Current RLS backend is some type of relational database u Good performance and scalability l Want to be able to support a backend mechanism for storing name mappings that is optimized to provide higher performance l Requestor: LIGO l Pluggable backend to allow use of a specialized hashing technique

RLS Roadmap (Requested): Improved Support for Bulk Renames l SCEC project adding and updating hundreds of thousands of entries l Would like better support for bulk modification command u E.g., regular expression-based replacement of filename prefixes

DRS Roadmap: Requirements analysis and improvements to DRS in support of LIGO l Design of DRS is based on LIGO's Lightweight Data Replicator service l Explore requirements for integrating DRS into the next generation of LIGO data architecture l Identify and implement enhancements to DRS to support the requirements of high-performance data Grids

DRS Roadmap: Replica Validation Capability l Design and implement replica validation and consistency checking l Requested by LIGO and others l Current DRS supports on-demand replication of data files between sites u Once data has been replicated, there is no guarantee that replica consistency will be maintained l Users need the ability to validate replicated data files and identify files that do not meet a specified equivalence criterion