Presentation is loading. Please wait.

Presentation is loading. Please wait.

Towards Storage On-Demand on Petabyte Grids A case for: Quotas and Reserve-ahead, Timed Leases and Contracts Frank Wuerthwein UC San Diego Abhishek Singh.

Published byHoratio Cain Modified over 8 years ago

Similar presentations

Presentation on theme: "Towards Storage On-Demand on Petabyte Grids A case for: Quotas and Reserve-ahead, Timed Leases and Contracts Frank Wuerthwein UC San Diego Abhishek Singh."— Presentation transcript:

1 Towards Storage On-Demand on Petabyte Grids A case for: Quotas and Reserve-ahead, Timed Leases and Contracts Frank Wuerthwein UC San Diego Abhishek Singh Rana UC San Diego

2 Towards Storage On-Demand on Petabyte Grids A case for: Quotas and Reserve-ahead, Timed Leases and Contracts Frank Wuerthwein UC San Diego Abhishek Singh Rana UC San Diego

3 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids3 Case List Priority 0 Reserve-ahead: File transfer level (~Implicit Reservation) Priority 1 Bulk Quotas: VO level (~Explicit Partitioning) Fine Quotas: Individual User level (~generally Oversubscribed) Priority 2 Reserve-ahead (from a Quota): User level (~Explicit Reservation) Reserve-ahead (from a Quota): VO level (~Explicit Reservation) Priority 3 Hierarchical and Cumulative Exhaustion of Quotas Priority 4 Storage space accrual and Time Timed Leases & Contracts: VO/User level Timed Leases & Contracts: Grid/Enterprise level

4 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids4 Quota Based on agreements between a Site and VOs Assigned by a Site Ceiling on storage space to a VO or User Oversubscription possible Timeless configuration Based on needs of Users/Jobs Requested by a User/Job Can only be a subset of Quota Timed lease Reserve-ahead

5 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids5 Owner/Primary VO Storage at a Site Reserve-ahead: File transfer level Peer VO1Peer VO2 Needed primarily for robust transfers. Independent of quota as file size << quota.

6 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids6 Storage at a Site Bulk Quotas: VO level Owner/Primary VO

7 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids7 Storage at a Site Bulk Quotas: VO level Peer VO1Peer VO2 Owner/Primary VO

8 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids8 Storage at a Site Bulk Quotas: VO level Peer VO1Peer VO2 Owner/Primary VO Need bulk quotas for VOs to co-exist on SE

9 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids9 Storage at a Site Fine Quotas: User level Peer VO1Peer VO2 Owner/Primary VO

10 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids10 Owner/Primary VO Storage at a Site Fine Quotas: User level Peer VO1Peer VO2 ?

11 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids11 Owner/Primary VO Storage at a Site Fine Quotas: User level Peer VO1Peer VO2 Need user level quota For users to co-exist.

12 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids12 Owner/Primary VO Storage at a Site Reserve-ahead: User level Peer VO1Peer VO2

13 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids13 Owner/Primary VO Storage at a Site Reserve-ahead: VO level Peer VO1Peer VO2

14 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids14 Owner/Primary VO Storage at a Site Hierarchical and Cumulative Exhaustion of Quotas Peer VO1Peer VO2 R1 Used = U1 Reserved = R1 Exhausted = Used + Reserved Available = Total – Exhausted U1

15 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids15 Owner/Primary VO Storage at a Site Hierarchical and Cumulative Exhaustion of Quotas Peer VO1Peer VO2 R1 Used = U1 Reserved = R1 Exhausted = Used + Reserved Available = Total – Exhausted U1

16 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids16 Owner/Primary VO Storage at a Site Hierarchical and Cumulative Exhaustion of Quotas Peer VO1Peer VO2 R11 Used = U11+U12 Reserved = R11 + R12 Exhausted = Used + Reserved Available = Total – Exhausted U11 R12 U12

17 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids17 Storage space accrual and Time Leases & Contracts: VO/User level Owner/Primary VO Storage at a Site Peer VO1Peer VO2 R11 Used = U11+ U12 Reserved = R11 + R12 Exhausted = Used + Reserved Available = Total – Exhausted U11 R12 U12 R-peer1 U-peer1 f(t)

18 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids18 Storage space accrual and Time Leases & Contracts: Grid/Enterprise level f(t)

19 Frank Wuerthwein & Abhishek Singh Rana, UC San Diego Towards Storage On-Demand on Petabyte Grids19 Storage space accrual and Time Leases & Contracts: Grid/Enterprise level f(t)

Download ppt "Towards Storage On-Demand on Petabyte Grids A case for: Quotas and Reserve-ahead, Timed Leases and Contracts Frank Wuerthwein UC San Diego Abhishek Singh."

Similar presentations

Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana.

Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana.
Applications Area Issues RWL Jones GridPP13 – 5 th June 2005.

Applications Area Issues RWL Jones GridPP13 – 5 th June 2005.
Global Grid Forum GridWorld GGF15 Boston USA October 03 2005 Abhishek Singh Rana and Frank Wuerthwein UC San Diegowww.opensciencegrid.org The Open Science.

Global Grid Forum GridWorld GGF15 Boston USA October Abhishek Singh Rana and Frank Wuerthwein UC San Diegowww.opensciencegrid.org The Open Science.
Ted Hesselroth USCMS T3 Meeting 2008-03-03 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Ted Hesselroth STAF/STAX OSG Storage 2009-11-18 Demo of.

Ted Hesselroth USCMS T3 Meeting Abhishek Singh Rana and Frank Wuerthwein UC San Diego Ted Hesselroth STAF/STAX OSG Storage Demo of.
Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.

Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.
1 User Analysis Workgroup Update  All four experiments gave input by mid December  ALICE by document and links  Very independent.

1 User Analysis Workgroup Update  All four experiments gave input by mid December  ALICE by document and links  Very independent.
Implementing Finer Grained Authorization in the Open Science Grid Gabriele Carcassi, Ian Fisk, Gabriele, Garzoglio, Markus Lorch, Timur Perelmutov, Abhishek.

Implementing Finer Grained Authorization in the Open Science Grid Gabriele Carcassi, Ian Fisk, Gabriele, Garzoglio, Markus Lorch, Timur Perelmutov, Abhishek.
EGEE-II INFSO-RI-031688 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks Infrastructure overview Arnold Meijster &

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Infrastructure overview Arnold Meijster &
Ted Hesselroth Nordugrid 2007 September 24-28, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Open Science Grid Ted Hesselroth Fermilab.

Ted Hesselroth Nordugrid 2007 September 24-28, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Open Science Grid Ted Hesselroth Fermilab.
Distributed Application Management Using PLuSH Jeannie Albrecht, Christopher Tuttle, Alex C. Snoeren, and Amin Vahdat UC San Diego CSE {jalbrecht, ctuttle,

Distributed Application Management Using PLuSH Jeannie Albrecht, Christopher Tuttle, Alex C. Snoeren, and Amin Vahdat UC San Diego CSE {jalbrecht, ctuttle,
On Developing Data Grid Workflows using Storage Resource Broker (SRB) and Kepler Tim H. Wong - UC Davis Efrat Frank - SDSC Bertram Ludäscher - UC Davis.

On Developing Data Grid Workflows using Storage Resource Broker (SRB) and Kepler Tim H. Wong - UC Davis Efrat Frank - SDSC Bertram Ludäscher - UC Davis.
VGrids Virtual Grids in MiG. Idea Virtual organizations are a central point in the Grid model VO’s share –Resources –Files –Credits A VO is a set of users.

VGrids Virtual Grids in MiG. Idea Virtual organizations are a central point in the Grid model VO’s share –Resources –Files –Credits A VO is a set of users.
Www.consorzio-cometa.it Consorzio COMETA - PI2S2 Project UNIONE EUROPEA SAGE – Storage Accounting for Grid Environments in gLite Fabio Scibilia Consorzio.

Consorzio COMETA - PI2S2 Project UNIONE EUROPEA SAGE – Storage Accounting for Grid Environments in gLite Fabio Scibilia Consorzio.
National Computational Science National Center for Supercomputing Applications National Computational Science Alliance Setup Package Requirements Jim Basney.

National Computational Science National Center for Supercomputing Applications National Computational Science Alliance Setup Package Requirements Jim Basney.
Grid Information Systems. Two grid information problems Two problems  Monitoring  Discovery We can use similar techniques for both.

Grid Information Systems. Two grid information problems Two problems  Monitoring  Discovery We can use similar techniques for both.
OSG Public Storage and iRODS

OSG Public Storage and iRODS
ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)

ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)
A. Sim, CRD, L B N L GIN-Data : SRM Island Inter-Op Testing With SRM-TESTER Alex Sim, Vijaya Natarajan Computational Research Division Lawrence Berkeley.

A. Sim, CRD, L B N L GIN-Data : SRM Island Inter-Op Testing With SRM-TESTER Alex Sim, Vijaya Natarajan Computational Research Division Lawrence Berkeley.
Polish Infrastructure for Supporting Computational Science in the European Research Space QoS provisioning for data-oriented applications in PL-Grid D.

Polish Infrastructure for Supporting Computational Science in the European Research Space QoS provisioning for data-oriented applications in PL-Grid D.
Ted Hesselroth USATLAS Tier 2 and Tier 3 Workshop November 29, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Installing and Using SRM-dCache.

Ted Hesselroth USATLAS Tier 2 and Tier 3 Workshop November 29, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Installing and Using SRM-dCache.

Similar presentations

Ads by Google