US ATLAS Grid Projects Rob Gardner Indiana University Mid Year Review of US ATLAS Computing NSF Headquarters, Arlington VA June 20,

June 20, 2002 Rob Gardner US Grid Projects 2 Projects Overview TParticle Physics Data Grid (T. Wenaus)  Year 2 of 3-year project to deliver vertically integrated grid services to experiments  Initial ATLAS activity: distributed data storage on the grid (Magda)  grid analysis (new)  EDG kit evaluation and US ATLAS testbed certification (ANL); Monitoring, information servers (BNL) TGriPhyN (R. Gardner)  Year 2 of 5 year project: main deliverables for ATLAS: VDT, Pacman, Grappa; Chimera toolkit: virtual data catalog and data language (VDC, VDL, VDLI)  Virtual data prototyping effort by S. Vanaichine (ANL) P. Nevski (BNL)  Architecture for virtual data portal developed (Argonne-Chicago-IU project) TiVDGL (R. Gardner), Networking (S. McKee)  Platform on which to design, implement, integrate VDT  Boston and Indiana University Prototype Tier2 Centers  Forum for grid interoperability – collaborate with EU DataGrid, DataTag, etc.  Networking: U of Michigan leads  see Internet2 working group: TUS ATLAS grid applications testbed (K.De)  Lots of contributions from various Lab and University groups! BNL,ANL,LBL,BU,OU,UM,UTA,IU

June 20, 2002 Rob Gardner US Grid Projects 3 TMAnager for Grid-based DAta TDesigned for ‘managed production’ and ‘chaotic end-user’ usage TDesigned for rapid development of components to support users quickly, with components later replaced by Grid Toolkit elements  Deploy as an evolving production tool and as a testing ground for Grid Toolkit components TAdopted by ATLAS for 2002 ATLAS Data Challenges TDevelopers - T. Wenaus and W. Deng (fulltime, postdoc) PPDG Magda Info: The system:

June 20, 2002 Rob Gardner US Grid Projects 4 Status TFull time developer/support Wensheng Deng  Development, support: ATLAS DC, US ATLAS testbed, LAr bench test DAQ, GDMP integration TATLAS DC production usage for cataloging (passive and active) and replication  Improved usability, doc for replication; dynamic replication; more replication modes  DC1 data replication from BNL to CERN to begin  To be used for data replication among the ~18 sites of DC1 phase 0 T260k files, ~11TB cataloged at present TGDMP integration beginning  Issues given to GDMP in Jan re: Magda integration fixed in V3  David Rebatto, a CS working for Laura Perini, is helping  GDMP3 being set up in Milan; will install Magda and do integration work there Torre Wenaus

June 20, 2002 Rob Gardner US Grid Projects 5 PPDG Year 2 & Magda TGDMP integration  Use GDMP to support a ‘publish/subscribe’ replication mode within Magda, targeted at ‘conventional production’ usage (what GDMP is designed for)  Magda still ‘adds value’ with a more flexible approach to data management and replication TApplication in demos TBuild Magda outwards into distributed job management/analysis  Distributed job management is to be the ATLAS PPDG yr 2 focus  Together with distributed analysis? TApply as replica manager in hybrid store common project TBuild it outwards (or adapt it) to support more metadata? E.g. data history information; HES metadata (HEMP a teeny step in this direction) TTo be eclipsed by Alien? Examine Alien and evaluate Torre Wenaus

June 20, 2002 Rob Gardner US Grid Projects 6 Pacman TPackage manager for the grid  Saul Youssef (Boston)  Used by VDT TSingle tool to easily manage installation and environment  fetch, install, configure, add to login environment, update  Sits over top of many software packaging approaches (rpm, tar.gz, etc.)  Uses dependency hierarchy, so one command can drive the installation of a complete environment of many packages  Packages organized into caches hosted at various sites  Distribute responsibility for support

June 20, 2002 Rob Gardner US Grid Projects 7 Grappa Work TGrid user interface for Athena TFlexible, use existing portal technology TSubmit Athena jobs to grid computing elements TManage JobOptions, record sessions TFile staging and output collection supported TRegisters output to MAGDA file catalog TPackaged for general ATLAS use with Pacman TDemonstrated on US ATLAS Grid Testbed (ATLFAST) TFuture: virtual data portal + GANGA (UK Grid) collaboration

June 20, 2002 Rob Gardner US Grid Projects 8 BU Tier 2 IU Tier 2 IU Physics cluster University of Oklahoma BNL Athena Grid Job Submission

June 20, 2002 Rob Gardner US Grid Projects 9

June 20, 2002 Rob Gardner US Grid Projects 10

June 20, 2002 Rob Gardner US Grid Projects 11

June 20, 2002 Rob Gardner US Grid Projects 12 iVDGL in ATLAS TMain goals for ATLAS  Integrate the GriPhyN VDT with ATLAS Core Software  Develop two prototype Tier 2 Centers for the US ATLAS  Integrate Tier 2 centers with US ATLAS Tier 1 Facility  Support development of US ATLAS Grid Testbed  Develop, integrate the US ATLAS piece of the iVDGL Laboratory TYear 1  Distribute, support VDT 1.1.x to US ATLAS sites  Tier2: 80K per Center for cluster upgrades and disk for ATLAS Data Challenges

June 20, 2002 Rob Gardner US Grid Projects 13 Indiana Tier 2

June 20, 2002 Rob Gardner US Grid Projects 14 Boston Tier 2

June 20, 2002 Rob Gardner US Grid Projects 15 Tier 2 Monitoring

June 20, 2002 Rob Gardner US Grid Projects 16 Virtualize ATLAS Production TTrack event data histories TVirtual data:  Transformation  Executable program  Athena service  Derivation  Execution of a transformation  Data Object  Named entity consumed by derivation  LFN, strings  Future: relational tables, persistent objects Generator HEPMC Hits Pileup Gen Merged Hits Simulation Combiner p-Hits Digitizer ROD Input ROD emulator Raw data Objects Reconstruction Event Data Objects pythia.exe atlsim.exe Athena services Input cards

June 20, 2002 Rob Gardner US Grid Projects 17 iSite MAGDA-GDMP Virtual Data Catalog Virtual Data Portal : Browser : Executor : Follower Abstract Planner Collection CatalogReplica Catalog Concrete Planner Replica Mgmt. Services Executor :DAGMan VDI VDL SE Persistency Service Athena Grid Execution Wrappers Algorithm Mgr. Services Histogram, Monitoring, … Services Event Selector Services Globus GRAM, GridFTP, … Core Athena Scripting ServicesCondor Pool Monitoring CAS, SAS Policy Athena Virtual Data System

June 20, 2002 Rob Gardner US Grid Projects 18 Virtual Data Tools TVirtual Data API  A Java class hierarchy to represent transformations and derivations TVirtual Data Language  Textual for illustrative examples  XML for machine-to-machine interfaces TVirtual Data Database  Makes the objects of a virtual data definition persistent TVirtual Data Service  Provides an OGSA interface to persistent objects

June 20, 2002 Rob Gardner US Grid Projects 19 VDT Status TVDT equivalent installed on most sites TVDT to be released after July 4  Globus 2.0  Condor  Condor-G  GDMP 3.0  ClassAds TSupport has been set up 