San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure University of Florida SRB-n-SRM for HEP Data Management (Possible architectures for brainstorming) Arun swaran Jagatheesan San Diego Supercomputer Center

Grid Physics Network (GriPhyN) University of Florida 2 A picture is worth a 1000 KB First, options/possibilities in the Over all Architecture

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 3 Distributed SRMs /…/text1.txt /…//text2.txt SRM /txt3.txt SRM University in UK (B) Organizations in Asia (C & D) National Lab in US (A)

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 4 Global Logical Namespace Needed National Lab in US (A) /…/text1.txt /…//text2.txt /txt3.txt SRM University in UK (B) Organizations in Asia (C & D) /home/arun.sdsc/cms /home/arun.sdsc/cms/text1.txt /home/arun.sdsc/cms/text2.txt /home/arun.sdsc/cms/text3.txt Logical Namespace: Organization of data, meta-data and storage (This logical view – need not be same as physical view of data sources )

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 5 Global Logical Namespace Needed National Lab in US (A) /…/text1.txt /…//text2.txt /txt3.txt SRM University in UK (B) Organizations in Asia (C & D) If we use SRB for this Global Logical Namespace.... (next slides)

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 6 Using Zone SRBs /…/text1.txt /…//text2.txt /txt3.txt SRM SRB Zone for National Lab in US One or more SRB Zones in Asia One SRB Zone for all UK universities Global Logical Name space

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 7 Possible Options for Zones Each Organization is a Zone with multiple SRMs One of more organizations make a Zone based on geography and administrative factors Each country is a Zone

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 8 A picture is worth a 1000 KB Second, Access

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 9 Do we want XML/SOAP for control? Services (SOAP-based) to manage and operate on the logical namespace of data and storage resources. Of course, this is only for control channel and not data movement channel

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 10 Access Option1: SRM still the interface /…/text1.txt /…//text2.txt /txt3.txt SRM SRB Zone for National Lab in US One or more SRB Zones in Asia One SRB Zone for all UK universities Global Logical Name space SRM

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 11 Access Option1: SRM still the interface + Existing applications in SRM can be preserved + HEP community already aware of SRM + SRM will emerge as the Grid Storage Management standard in GGF (so useful for us to follow the standard thingy) – SRM interface does not deal with a global logical namespace – Additional functionalities to manage or take advantage of this global logical namespace would be lost – Bulk operations are essential for data grid management – Zone to Zone (Peer to peer) – Meta data – SRM is NOT the Grid File System standard at GGF

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 12 Access Option2: SRB as the interface /…/text1.txt /…//text2.txt /txt3.txt SRM SRB Zone for National Lab in US One or more SRB Zones in Asia One SRB Zone for all UK universities Global Logical Name space SRB

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 13 Access Option2: SRB as the interface – Existing applications might have to be modified – HEP community has to be educated about the use and advantages of SRB (community resistance also) – Does it directly support storage space management NOW? + SRB interface deals with a global logical namespace of logical resources and logical data + Additional functionalities to manage or take advantage of this global logical namespace is available + Bulk Operations are essential for data grid management + Zone to Zone (Peer to peer) + Metadata + SRB could influence the Grid File System standards at GGF

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 14 Access Option3: GGF Grid File System Interface /…/text1.txt /…//text2.txt /txt3.txt SRM SRB Zone for National Lab in US One or more SRB Zones in Asia One SRB Zone for all UK universities Global Logical Name space GFS

San Diego Supercomputer Center Grid Physics Network (GriPhyN) University of Florida 15 Access Option3: GGF GFS as the interface – DOES NOT EXIST. We can NOT wait. – Our limited cycles might be used in just working for the GGF standard – rather than making this interface for us to work – What if we can not influence the standard creation (?) or its irrelevant for us + It is going to be the standard. So would possibly end up having multiple implementations (commercial and academic) + This community of different organizations and countries could influence the standard creation based on real needs + Experts from other communities would also suggest / help in the design of this interface