Atlas Grid Status - part 1 Jennifer Schopf ANL U.S. ATLAS Physics and Computing Advisory Panel Review Argonne National Laboratory Oct 30, 2001

Oct 2001 Jennifer Schopf, ANL 2 Globus/Atlas Interactions GRAPPA/ Gardner Grid Data Access/Malon Magda/ Wenaus GridView and other Monitoring De, Schopf, Yu Condor (G) GRAM GSI MDS/ GIIS/GRIS GridFTP Replica Cat Replica Mgr PacMan/ Youssef Packaging

Oct 2001 Jennifer Schopf, ANL 3 Globus Toolkit TM  Core protocols and services  Grid Security Infrastructure  Grid Resource Access & Management  MDS information service & monitoring  GridFTP data access & transfer  Data Grid technologies  Replica catalog, replica management service  Reliable file transfer  Defacto-standard for Grid projects:  GriPhyN, PPDG, NEES, EU DataGrid, ESG, Fusion Collaboratory, DISCOM, NASA IPG, NSF TeraGrid, DOE Science Grid, EU DataGrid, UK Grid Center, U.S. GRIDS Center, Access Grid, GridPort, MPICH-G2, Condor-G, GrADS, and others

Oct 2001 Jennifer Schopf, ANL 4 Globus Status  New release  In alpha4 now, beta before SC ‘01 (Nov 15), 4Q01 release  New packaging: enables modular binary and source distributions  GRAM 1.5 (job submission): enhanced robustness  MDS-2.1 (information service): security, better performance, etc.  GridFTP- secure large file transfer  Replica Mgt- Data management, catalogs for replicas  Continuing work  Community Authorization Services  Reliable file transfer  Java, other “Commodity Grid” toolkits

Oct 2001 Jennifer Schopf, ANL 5 ATLAS GriPhyN/iVDGL  GriPhyN funded Fall 2000, $11.9M/5 years  iVDGL funded Fall 2001, 5 years, $13.65M/5 year  Both involve ATLAS, CMS, Ligo and SDSS  ATLAS support: 2.5 FTE IU, 1.5 FTE BU, $331K/5yr IU HW  R. Gardner is the ATLAS lead, J. Schopf is the CS liaison  GriPhyN proposal emphasizes virtual data requirements, collaboration between experiments  iVDGL proposal emphasizes testbed and infrastructure issues across experiment

Oct 2001 Jennifer Schopf, ANL 6 GriPhyN  Principal ATLAS GriPhyN/iVDGL deliverables:  2001: Testbed with GriPhyN VDT 1.0, packaged with PacMan  2002: Serving DC1 data  2003: Dataset re-creation/Data signature  Additional efforts:  Monitoring: Dantong Yu, BNL, and J. Schopf,ANL, joint with PPDG  GRAPPA: Rob Gardner and Randy Bramley, IU

Oct 2001 Jennifer Schopf, ANL 7 GriPhyN Testbed Issues T GriPhyN is defining VDT 1.o q SW install for GriPhyN/PPDG, compatible with EDG as well q Globus 2.0 beta when it’s released (10/30) z GSI, GridFTP, MDS, repl, cat stuff, etc (Gram 1.5) q GDMP 2.0 (supports flat files) q Condor (also Condor-G, Dagman) T Extra tools for ATLAS q objectivity 6.1 q Magda T Still need to resolve CA issue between EDG and US test sites

Oct 2001 Jennifer Schopf, ANL 8 GriPhyN ATLAS Goal 1 Serving DC1 Data (July-Dec 2002) T Limited reconstruction analysis job using grid job submission interface T Serving the data results form DC1 q As part of DC1, data must be tagged with meta data for ease of access  Minimal keywords would be sufficient  Magda already implements portions of this T Job submission with minimal smarts q Extend GRAPPA work q Move compute resources to data sites

Oct 2001 Jennifer Schopf, ANL 9 GriPhyN ATLAS Goal 2 Dataset Re-creation (Jan-Sept 2003) T Goal: be able to re-create a data file T Need to evaluate what parameters need to be kept track of T Need to evaluate data needed for full data signature T Need to develop a metric for evaluating success - what is good enough?

Oct 2001 Jennifer Schopf, ANL 10 GRAPPA: Grid Access Portal for Physics Applications  Provide a point of access to ATLAS grid resources  IU (Physics, CS), Northwestern (ECE), ANL (CS), BU (Physics),  Provide a simple interface for physicists to submit and monitor jobs on the Grid  Web-based as well as script-based  Ability to “replay”  Compatible with both ATLSim and Athena architecture  Adaptable and/or “extensible” to new developments in Grid software, Athena, etc

Oct 2001 Jennifer Schopf, ANL 11 GRAPPA Components  User Interface  Job submission  Monitoring  Bookkeeping  Resource selection

Oct 2001 Jennifer Schopf, ANL 12 Current Status  Simple prototype that allows users to submit an Athena job from a web interface to the Condor pool on the Atlas IU cluster (via Globus)  Next steps  Adding more Athena functionality to the interface (e.g., user's defined libraries)  Experiment with other job launch mechanisms  Condor-G and DAGMAN description language  Web Services Flow Language as a more general workflow description  Explore interfaces for multiple metadata and replica catalog systems. 

Oct 2001 Jennifer Schopf, ANL 13 Grid-Enabled Data Access in Athena T David Malon, ANL T Integrate Grid Data Access techniques (Globus replica catalog and/or GDMP) into the Athena event selection module T When a file is needed, a check is done to see if the file is local q if not, use Grid Data Access tools to make it local

Oct 2001 Jennifer Schopf, ANL 14 Grid-Enabled Data Access in Athena  Athena and Globus  Search Globus replica catalog and select  Transfer file using the protocol associated with the location object in the catalog (gsiftp, https/globus-url-copy from a remote gass_server,...)  Athena and GDMP  Work to date uses GDMP 1.2.2, which had not yet incorporated the Globus replica catalog  Uses the GDMP import/export catalogs  Supports certain subscription-based approaches  Automatically updates Objectivity/DB internal catalog when Objectivity database files are transferred

Oct 2001 Jennifer Schopf, ANL 15 Status  Paper presented at CHEP  Prototype up and running on ANL systems  GDMP-based prototype work done between CERN and Milan  Next Steps:  GDMP will use Globus data replica work  Use GDMP to copy files  Requirements and design work for interfaces between Athena data producers and metadata catalogs and replica management services (e.g., Magda)  Metadata work – extending the Athena Event Selector properties to allow for data signature/virtual data

Oct 2001 Jennifer Schopf, ANL 16 Monitoring  Joint working group set up between PPDG and GriPhyN to investigate monitoring issues  Led by J. Schopf and D. Yu  Monitoring currently defined very broadly:  Is this router configured correctly?  Has the application finished using that file yet?  What information do I need to determine where to run my application?

Oct 2001 Jennifer Schopf, ANL 17 GridView (Kaushik De) T Tool to show the status of the 8 test bed machines on the web T Uses Globus GRAM to query sites every 30 mins T Hostname, Uptime, Idletime, # users, and Load average T Next steps include q Integration with Globus information service (MDS) q Visualization as part of cross PPDG/GriPhyN monitoring work

Oct 2001 Jennifer Schopf, ANL 18

Oct 2001 Jennifer Schopf, ANL 19 Application Level Monitoring (Taylor)  Initial testbed to incorporate GRID monitoring capabilities into Athena  Collaboration with Valerie Taylor, David Quarrie, and others  Very long ramp-up due to difficulty of “outsiders” running an Athena application  Working on developing an Auditor for Athena  ml

Oct 2001 Jennifer Schopf, ANL 20 Current Status  Mailing list has been set up   Defining usage cases  Sensors  Predictors  Archiving  Will gather requirements and look at extending Globus MDS as a common framework to meet these requirements

Oct 2001 Jennifer Schopf, ANL 21 Summary  Globus  GriPhyN/iVDGL  Grappa  Grid-Enabled Data Access  Monitoring and Visualization