Presentation is loading. Please wait.

Presentation is loading. Please wait.

11-June-2008Fabrizio Furano - Data access and Storage: new directions1.

Published byAllison Doyle Modified over 9 years ago

Similar presentations

Presentation on theme: "11-June-2008Fabrizio Furano - Data access and Storage: new directions1."— Presentation transcript:

1

2 11-June-2008Fabrizio Furano - Data access and Storage: new directions1

3 11-June-2008Fabrizio Furano - Data access and Storage: new directions2

4 11-June-2008Fabrizio Furano - Data access and Storage: new directions3

5 11-June-2008Fabrizio Furano - Data access and Storage: new directions4

6 11-June-2008Fabrizio Furano - Data access and Storage: new directions5

7 11-June-2008Fabrizio Furano - Data access and Storage: new directions6 Courtesy of Gerardo Ganis (CERN PH-SFT)

8 11-June-2008Fabrizio Furano - Data access and Storage: new directions7 Courtesy of Gerardo Ganis (CERN PH-SFT)

9 11-June-2008Fabrizio Furano - Data access and Storage: new directions8

10 11-June-2008Fabrizio Furano - Data access and Storage: new directions9

11 11-June-2008Fabrizio Furano - Data access and Storage: new directions10

12 11-June-2008Fabrizio Furano - Data access and Storage: new directions11 Pre-xfer data “locally” Pre-xfer data “locally” Legacy remote access Legacy remote access Remote access+ Remote access+ Data Processing Data access Overhead Need for potentially useless replicas And a huge Bookkeeping! Overhead Need for potentially useless replicas And a huge Bookkeeping! Latency Wasted CPU cycles But easy to understand Latency Wasted CPU cycles But easy to understand Interesting! Efficient practical Interesting! Efficient practical

13 Application 11-June-2008Fabrizio Furano - Data access and Storage: new directions12 Client1 Server Client2 Client3 TCP (control)‏ Clients still see One Physical connection per server Clients still see One Physical connection per server TCP(data)‏ Async data gets automaticall y splitted Async data gets automaticall y splitted

14 11-June-2008Fabrizio Furano - Data access and Storage: new directions13

15 11-June-2008Fabrizio Furano - Data access and Storage: new directions14 *Federico Carminati, The ALICE Computing Status and Readiness, LHCC, November 2007

16 11-June-2008Fabrizio Furano - Data access and Storage: new directions15 *Federico Carminati, The ALICE Computing Status and Readiness, LHCC, November 2007

17 11-June-2008Fabrizio Furano - Data access and Storage: new directions16

18 11-June-2008Fabrizio Furano - Data access and Storage: new directions17

19 11-June-2008Fabrizio Furano - Data access and Storage: new directions18

20 11-June-2008Fabrizio Furano - Data access and Storage: new directions19 cmsd xrootd Prague NIHAM … any other cmsd xrootd CERN cmsd xrootd ALICE global redirector (alirdr) all.role meta manager all.manager meta alirdr.cern.ch:1312root://alirdr.cern.ch/Includes CERN, GSI, and others xroot clusters Meta Managers can be geographically replicated Can have several in different places for region- aware load balancing Meta Managers can be geographically replicated Can have several in different places for region- aware load balancing cmsd xrootd GSI all.manager meta alirdr.cern.ch:1312 all.role manager

21 cmsd xrootd GSI 11-June-2008Fabrizio Furano - Data access and Storage: new directions20 cmsd xrootd Prague NIHAM … any other cmsd xrootd CERN cmsd xrootd ALICE global redirector all.role meta manager all.manager meta alirdr.cern.ch:1312 all.role manager Note: the security hats could require you use xrootd native proxy support But missing a file? Ask to the global metamgr Get it from any other collaborating cluster all.manager meta alirdr.cern.ch:1312 Local clients work normally

22 11-June-2008Fabrizio Furano - Data access and Storage: new directions21

23 11-June-2008Fabrizio Furano - Data access and Storage: new directions22

24 11-June-2008Fabrizio Furano - Data access and Storage: new directions23

25 11-June-2008Fabrizio Furano - Data access and Storage: new directions24

26 11-June-2008Fabrizio Furano - Data access and Storage: new directions25

27 11-June-2008Fabrizio Furano - Data access and Storage: new directions26

28 11-June-2008Fabrizio Furano - Data access and Storage: new directions27

29 11-June-2008Fabrizio Furano - Data access and Storage: new directions28

30 11-June-2008Fabrizio Furano - Data access and Storage: new directions29

31 11-June-2008Fabrizio Furano - Data access and Storage: new directions30 Redirector xrootd:1094 Name Space xrootd:2094RedirectorHost ClientHost opendir create mkdir mv rm rmdir xrootd POSIX Client Kernel User Space Appl POSIX File System Interface FUSE FUSE/Xroot Interface Should run cnsd on servers to capture non-FUSE events And keep the FS namespace!

32 11-June-2008Fabrizio Furano - Data access and Storage: new directions31

33 11-June-2008Fabrizio Furano - Data access and Storage: new directions32

34 11-June-2008Fabrizio Furano - Data access and Storage: new directions33

Download ppt "11-June-2008Fabrizio Furano - Data access and Storage: new directions1."

Similar presentations

Potential Data Access Architectures using xrootd OSG All Hands Meeting Harvard University March 7-11, 2011 Andrew Hanushevsky, SLAC

Potential Data Access Architectures using xrootd OSG All Hands Meeting Harvard University March 7-11, 2011 Andrew Hanushevsky, SLAC
ATLAS/LHCb GANGA DEVELOPMENT Introduction Requirements Architecture and design Interfacing to the Grid Ganga prototyping A. Soroko (Oxford), K. Harrison.

ATLAS/LHCb GANGA DEVELOPMENT Introduction Requirements Architecture and design Interfacing to the Grid Ganga prototyping A. Soroko (Oxford), K. Harrison.
Xrootd and clouds Doug Benjamin Duke University. Introduction Cloud computing is here to stay – likely more than just Hype (Gartner Research Hype Cycle.

Xrootd and clouds Doug Benjamin Duke University. Introduction Cloud computing is here to stay – likely more than just Hype (Gartner Research Hype Cycle.
Outline Network related issues and thinking for FAX Cost among sites, who has problems Analytics of FAX meta data, what are the problems  The main object.

Outline Network related issues and thinking for FAX Cost among sites, who has problems Analytics of FAX meta data, what are the problems  The main object.
1 Status of the ALICE CERN Analysis Facility Marco MEONI – CERN/ALICE Jan Fiete GROSSE-OETRINGHAUS - CERN /ALICE CHEP 2009 - Prague.

1 Status of the ALICE CERN Analysis Facility Marco MEONI – CERN/ALICE Jan Fiete GROSSE-OETRINGHAUS - CERN /ALICE CHEP Prague.
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google∗

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google∗
PROOF: the Parallel ROOT Facility Scheduling and Load-balancing ACAT 2007 Jan Iwaszkiewicz ¹ ² Gerardo Ganis ¹ Fons Rademakers ¹ ¹ CERN PH/SFT ² University.

PROOF: the Parallel ROOT Facility Scheduling and Load-balancing ACAT 2007 Jan Iwaszkiewicz ¹ ² Gerardo Ganis ¹ Fons Rademakers ¹ ¹ CERN PH/SFT ² University.
The LCG File Catalog (LFC) Jean-Philippe Baud – Sophie Lemaitre IT-GD, CERN May 2005.

The LCG File Catalog (LFC) Jean-Philippe Baud – Sophie Lemaitre IT-GD, CERN May 2005.
The Next Generation Root File Server Andrew Hanushevsky Stanford Linear Accelerator Center 27-September-2004

The Next Generation Root File Server Andrew Hanushevsky Stanford Linear Accelerator Center 27-September-2004
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong

1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
Scalla/xrootd Andrew Hanushevsky, SLAC SLAC National Accelerator Laboratory Stanford University 19-May-09 ANL Tier3(g,w) Meeting.

Scalla/xrootd Andrew Hanushevsky, SLAC SLAC National Accelerator Laboratory Stanford University 19-May-09 ANL Tier3(g,w) Meeting.
Scalla/xrootd Andrew Hanushevsky SLAC National Accelerator Laboratory Stanford University 29-October-09 ATLAS Tier 3 Meeting at ANL

Scalla/xrootd Andrew Hanushevsky SLAC National Accelerator Laboratory Stanford University 29-October-09 ATLAS Tier 3 Meeting at ANL
Xrootd, XrootdFS and BeStMan Wei Yang 2009-10-30 US ATALS Tier 3 meeting, ANL 1.

Xrootd, XrootdFS and BeStMan Wei Yang US ATALS Tier 3 meeting, ANL 1.
Scalla/xrootd Introduction Andrew Hanushevsky, SLAC SLAC National Accelerator Laboratory Stanford University 6-April-09 ATLAS Western Tier 2 User’s Forum.

Scalla/xrootd Introduction Andrew Hanushevsky, SLAC SLAC National Accelerator Laboratory Stanford University 6-April-09 ATLAS Western Tier 2 User’s Forum.
July-2008Fabrizio Furano - The Scalla suite and the Xrootd1 cmsd xrootd cmsd xrootd cmsd xrootd cmsd xrootd Client Client A small 2-level cluster. Can.

July-2008Fabrizio Furano - The Scalla suite and the Xrootd1 cmsd xrootd cmsd xrootd cmsd xrootd cmsd xrootd Client Client A small 2-level cluster. Can.
11-July-2008Fabrizio Furano - Data access and Storage: new directions1.

11-July-2008Fabrizio Furano - Data access and Storage: new directions1.
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.

D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
FAX Redirection Topology Wei Yang 7/16/121. Redirector hardware at CERN Redundant redirectors for EU, UK, DE, FR – Redundant (the “+” sign below) VMs.

FAX Redirection Topology Wei Yang 7/16/121. Redirector hardware at CERN Redundant redirectors for EU, UK, DE, FR – Redundant (the “+” sign below) VMs.
Xrootd Monitoring Atlas Software Week CERN November 27 – December 3, 2010 Andrew Hanushevsky, SLAC.

Xrootd Monitoring Atlas Software Week CERN November 27 – December 3, 2010 Andrew Hanushevsky, SLAC.
Introduction to HDFS Prasanth Kothuri, CERN 2 What’s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand.

Introduction to HDFS Prasanth Kothuri, CERN 2 What’s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand.

Similar presentations

Ads by Google