11 March 2004Getting Ready for the Grid SAM: Tevatron Experiments Using the Grid CDF and D0 Need the Grid –Requirements, the CAF and SAM –Grid from the User Perspective Grid to Meet the Need –How SAM works –SAM usage by D0 and CDF Near Future: SAMGrid Rick St. Denis, University of Glasgow

11 March 2004Getting Ready for the Grid Reviews: Director’s (technically), International Finance Committee (fiscally) FNAL PAC (for its physics merit) Maximize physics low Lumi –L3 output rate: 80 -> 360Hz by 06 Spokespersons’ Requirements for CDF 50% computing outside FNAL CDF needs the Grid

11 March 2004Getting Ready for the Grid Scale of CDF Requirements THz%offsiteCPU Speed #duals FY %3GHz150 FY %5GHz+360 FY %8GHz sites, 100Duals each, by

11 March 2004Getting Ready for the Grid CDF Computing Model Develop Analysis on desktop –Access to all CDF data from anywhere Large scale processing on batch clusters –Submission from anywhere –interactive tools: ls,top,head/tail/cat –Output to scratch space or desktop Implemented Now with CAF (not Grid standard) Exists Now

11 March 2004Getting Ready for the Grid Central Analysis Facility CAF is a pile of PC’s with a pile of disks. (1200 processors and 100TB) This can be implemented anywhere as dCAF: Decentralized CAF. Output of jobs can go to desktop or a scratch area Need a password for this: authentication (kerberos).

11 March 2004Getting Ready for the Grid Sequential Access through Metadata Metadata: SAM allows groups of files to be identified into datasets using attributes (metadata) such as production pass version or top quark mass to associate them. File Retrieval: SAM moves files to users as they request them. File Storage: SAM allows output files to be stored with new metadata.

11 March 2004Getting Ready for the Grid Metadata File Type: SAMMC Data File File Name: Bs_conc_4o5_3.root File ID: File Size: [B] File Start Time: 01/29/ :00:00 File End Time: 01/29/ :00:00 Application Family: generator Application Version: 1.00 Description: BsDspi_phipi MONTE CARLO Dataset 4o5 part 3 Run Number: ~]$ sam get metadata --file=Bs_conc_4o5_3.root totalevents = 7290 Work Group: cdf Node Name: cdfsam.cnaf.infn.it dataset = BsMC-lucchesi_test html =

11 March 2004Getting Ready for the Grid Use Cases User Level MC Production –All Users have access –No data on site -> write to tape at FNAL User Level Data Access –All users have access –Selected samples automaticaly copied on site SAM provides this

11 March 2004Getting Ready for the Grid Functionality User selects a place to run, saying what dataset they will use System checks they can do this (privileges) User access to data at any place User output is stored on any disk or back to tape at FNAL and results are made available for transfer to any site for others to analyse.

11 March 2004Getting Ready for the Grid CAF Gui/CLI User Perspective Analysis program Grid TorontoKoreaItalyTaiwanFermiCAFUK CAF Gui/CLI User Perspective Only Fermilab Uses SAM Outside LabGrid Uses SAM

11 March 2004Getting Ready for the Grid Meeting the Needs SAM: How it works Progress in SAM CDFGridWorkshop: “Nerd’s Paradise” D0 and CDF Usage

11 March 2004Getting Ready for the Grid Fcdfdata016Disk/Cache Station central-analysis Daemon (smaster) Stager Daemon (stagerng) FSS (Deamon) (fss) Stager Daemon (stagerng) Disk/Cache Stager Daemon (stagerng) Stager Daemon (stagerng) Stager Daemon (stagerng)

11 March 2004Getting Ready for the Grid Node1 Cache Node2 Cache Node3 Cache Node4 Cache Node5 Cache Station smaster Stager stagerng Stager stagerng Stager stagerng Stager stagerng Stager stagerng A Farm: Station with Stagers and Caches

11 March 2004Getting Ready for the Grid What can 20 duals and 6 TB do? StreamEventsDaysInput Size Top,W/Z20.5 M TB Hadronic B and charm 156M TB Need to transfer 0.6 GB/min or 1 TB/Day

11 March 2004Getting Ready for the Grid fcdfdata016 Disks/Cache

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache Stager stagerng

11 March 2004Getting Ready for the Grid setenv SAM_STATION chris sam dump station --disks *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 53 minutes 25 seconds, admins: jozwiak terekhov Known batch systems: lsf Default batch system: lsf No replica selection criteria There are 0 authorized transfer groups Minimum delivery is 1KB; external deliveries are unconstrained STATION DISKS: disk 7844 nglas08.fnal.gov:/sam/test9/jozwiak/dev/chris, 29947KB/20GB free disk 8064 nglas08.fnal.gov:/sam/test10/jozwiak/dev/chris, 93110KB/20GB free *** END OF STATION DUMP *** sam dump station --disks

11 March 2004Getting Ready for the Grid sam dump station --groups *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 57 minutes 3 seconds, admins: jozwiak terekhov Known batch systems: lsf Default batch system: lsf No replica selection criteria There are 0 authorized transfer groups Minimum delivery is 1KB; external deliveries are unconstrained AUTHORIZED GROUPS: group test: admins: jozwiak, swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/150, disk: KB/40GB, locks:0B/0KB group test1: admins: jozwiak, swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/40, disk: KB/30GB, locks:0B/0KB group test2: admins: jozwiak, swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/50, disk: KB/40GB, locks:0B/0KB *** END OF STATION DUMP *** sam dump station --groups

11 March 2004Getting Ready for the Grid sam dump station --projects *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 4 minutes 49 seconds, admins: jozwiak terekhov Known batch systems: lsf Default batch system: lsf No replica selection criteria There are 0 authorized transfer groups Minimum delivery is 1KB; external deliveries are unconstrained PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited NO PROJECTS *** END OF STATION DUMP *** sam dump station --projects

11 March 2004Getting Ready for the Grid fcdfdata016 sam submit --script=userscript --group=groupname --cpu-per-event= --defname= Station central-analysis smaster Disks/Cache Stager stagerng

11 March 2004Getting Ready for the Grid fcdfdata016 >>>>>> Starting project with the Station Master Station Master contacted, result: Started project (49008_sam_) for group test Waiting for the project to initialize... Station central-analysis smaster Disks/Cache Stager stagerng

11 March 2004Getting Ready for the Grid fcdfdata016 Callback from server: 'OK|Project is ready' Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster

11 March 2004Getting Ready for the Grid fcdfdata016 >>>>>> Submitting the job to the batch system. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP

11 March 2004Getting Ready for the Grid sam dump station --projects *** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 12 minutes 44 seconds, admins: jozwiak terekhov Known batch systems: lsf Default batch system: lsf No replica selection criteria There are 0 authorized transfer groups Minimum delivery is 1KB; external deliveries are unconstrained PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited STATION PROJECTS: project 49205_sam_(49205) user jozwiak.test started 01 Nov 14:08:45 UNIX pid still wants/currently uses 5/0 files *** END OF STATION DUMP *** Sam dump station --projects

11 March 2004Getting Ready for the Grid sam dump project --project=49205_sam_ *** BEGIN GPM DUMP *** Input files: : sim.ztautau.1000evts c5.01, size=0K, unbuffered yet : sim.ztautau.1000evts c5.02, size=0K, unbuffered yet : sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.267_1553, size=0K, unbuffered : sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.276_1152, size=0K, unbuffered Cached (not buffered) files: (none) Buffered files: (none) External files with delivery problems: (none) Umer contexts (name, state, join time, nSeen): (no umers) Proc contexts (ID: name, state, join time [, current|last]): (no procs) Processes waiting for call back:(none) *** END GPM DUMP *** sam dump project –project=

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP Optimizer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker encp

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker encp Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker encp Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker encp Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) PSUSP eworker encp Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid SAMManager:sam Getting next input file... SAMManager:sam Project master will call back.

11 March 2004Getting Ready for the Grid sam dump project --project=49225_sam_ *** BEGIN GPM DUMP *** Input files: : d0g.test_file_1G_a_dev.0001_001, size=0K, unbuffered yet : d0g.test_file_1G_a_dev.0002_001, size=0K, unbuffered yet Cached (not buffered) files: (none) Buffered files: (none) External files with delivery problems: (none) Umer contexts (name, state, join time, nSeen): 36422: jozwiak(test-harness:1), active, 05 Nov 13:59:09, 31 Proc contexts (ID: name, state, join time [, current|last]): : wait, 05 Nov 13:59:10, Processes waiting for call back: CID=36422: (05 Nov 20:53:43) *** END GPM DUMP *** Sam dump project –project=49225_sam_

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN eworker encp Enstore samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer rm

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer Optimizer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer eworker

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer eworker rcp

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer eworker rcp Other Cache

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer eworker rcp Other Cache

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer eworker rcp

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Job is submitted to queue. Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF) RUN samscript.sh userscript consumer

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache Stager stagerng Project pmaster Batch (LSF)

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache Stager stagerng Batch (LSF)

11 March 2004Getting Ready for the Grid fcdfdata016 sam submit…. sam run project… Station central-analysis smaster Disks/Cache Stager stagerng Batch (LSF)

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache Stager stagerng Batch (LSF) RUN RUN PSUSP Project pmaster Project pmaster Project pmaster samscript.sh userscript consumer eworker rcp encp Other Cache Enstore

11 March 2004Getting Ready for the Grid fcdfdata016 Station central-analysis smaster Disks/Cache Stager stagerng Batch (LSF) RUN RUN PSUSP Project pmaster Project pmaster Project pmaster samscript.sh userscript consumer eworker rcp encp Other Cache Enstore

11 March 2004Getting Ready for the Grid SAM Animation worldScenerio.html

11 March 2004Getting Ready for the Grid Storing Files Getting things to tape from Glasgow

11 March 2004Getting Ready for the Grid fcdfdata016 Disks FSS Central-analysis fss Stager stagerng

11 March 2004Getting Ready for the Grid sam dump fss FSS version v3_2_2 at station central-analysis running on fcdfdata016.fnal.gov 6 hours 57 minutes 34 seconds No routing (all transfers are direct) Configuration for operation retrial (count, interval/timeout) DBS contact: 3, 1 hours Opter contact: 1, 1 hours Authorization receipt:1, 1 hours Stager contact: 1, 1 hours Transfer (retrials upon timeout and upon failure): 3, 6 hours Relay (multi-stage routing only): 3, 1 hours File Storage Server Dump: Stagers are known at nodes: fcdfdata016.fnal.gov No requests ever submitted Sam dump fss

11 March 2004Getting Ready for the Grid fcdfdata016 sam store descrip.py --source= [--dest=/pnfs…..] Disks FSS Central-analysis fss Stager stagerng

11 March 2004Getting Ready for the Grid fcdfdata016 sam store descrip.py --source= [--dest=/pnfs…..] Disks FSS Central-analysis fss Stager stagerng Descrip.py Metadata Info about file Sam checks info, checks location,

11 March 2004Getting Ready for the Grid fcdfdata016 sam store descrip.py --source= [--dest=/pnfs…..] Disks FSS Central-analysis fss Stager stagerng eworker encp, rcp, bbftp

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From Really Far Away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss Routing: fcdfdata016 Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore sam store enstore

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss Routing: fcdfdata016 Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore eworker bbftp fcdfdata016

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore eworker bbftp fcdfdata016

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore eworker encp

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore eworker encp

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore rm

11 March 2004Getting Ready for the Grid Node from Really Far Away Disk Fss From really Far away Stager fcdfdata016 Fss central-analysis Stager Tmp Disk Enstore

11 March 2004Getting Ready for the Grid D0 Sam D0 relies entirely on SAM for analysis

11 March 2004Getting Ready for the Grid D0

11 March 2004Getting Ready for the Grid D0 Files Files/Day

11 March 2004Getting Ready for the Grid D0 Data Volume 1TB-3TB/day

11 March 2004Getting Ready for the Grid D0 Files Per Month By Year ,000 files Run II Start

11 March 2004Getting Ready for the Grid D0 Total Files 2.5Million Files Served

11 March 2004Getting Ready for the Grid D0 Data Per Month By Year 50 TB per month Run II Start

11 March 2004Getting Ready for the Grid D0 Total Data Moved 700TB moved

11 March 2004Getting Ready for the Grid Progress in SAM: CDF All 800,000 CDF data files are in SAM Sam is in beta testing on the CDF CAF (1200 cpus): passed 20TB/Day delivery Karlsruhe uses SAM routinely Minos uses SAM for its Data Handling Steve Mrenna (Phenomenology) depositing ALPGEN files in SAM for common CDF/D0 use.

11 March 2004Getting Ready for the Grid Florida workshop: 11 installations in about 2 hours. Integrated with dCAF in 2 cases in 2 days. 3 in Asia, 4 in Europe 6 sites committed to summer 2004 usage of their facilities for all of CDF (mostly MC) Sam installation now: initsam cdf Follow-up on April 1. Each site has a local user support person to reduce load on core development team. Generally: Security ate 80% of the effort! Now 20!

11 March 2004Getting Ready for the Grid CDF

11 March 2004Getting Ready for the Grid Florida Workshop: After 2 Days

11 March 2004Getting Ready for the Grid 2TB/Day: Karlsruhe

11 March 2004Getting Ready for the Grid CDF Dcache on CAF ALL CDF on CAF reads 25TB/Day NonGrid Running

11 March 2004Getting Ready for the Grid Karlsruhe: 1500 files/Day CDF Files in a Month

11 March 2004Getting Ready for the Grid Karlsruhe: 5-10M Evt/Day CDF Events Transfer in a Month

11 March 2004Getting Ready for the Grid All CDF Files Moved by SAM 300K Files D0: 2.5M files

11 March 2004Getting Ready for the Grid Total CDF Data Moved 200 TB D0:700TB

11 March 2004Getting Ready for the Grid Advantage of Local Processing Karlsruhe processes 2TB/day. Rest of CDF on Central Cluster processes 25TB/day. (450 processors, 8 experiments, 10/13TB disk filled.) 5 users actively at Karlsruhe. Make ntuple for bottom and top physics for 15 people. 100 users active for rest of CDF: They pin the datasets of interest; copy new ones automatically.

11 March 2004Getting Ready for the Grid Dcache and SAM Dcache shapes traffic into disk: If a SAM cache is large, need to use Dcache instead of nfs mounts Dcache gives the user what is requested. 1TB gets same priority as 1GB: CDF users must send requesting data to be staged. SAM examines consumption rate before staging next files – No needed. SAM uses Dcache for its Caching at FNAL. This needs further work with SRM

11 March 2004Getting Ready for the Grid In the near term future:JIM Adding Grid Standard Tools

11 March 2004Getting Ready for the Grid CDF Grid Strategy 25% of CDF Computing from external resources. All CDF computing on CDF Grid by April 15: Utilize resources fully controlled by CDF: Kerberos/fbsng: dCAF + SAM October 15, 2004: JIM to capture shared resources June 2005: 50% of Computing resources external

11 March 2004Getting Ready for the Grid Desktop Anywhere Condor centers SAM DB Condor Globus GK CAF Submitter SAM each site WN Private LAN dCache June 2004 testing June 2005 required Simple JIM

11 March 2004Getting Ready for the Grid Detailed JIM Site Resource Selector Info Collector Info Gatherer Match Making User Interface Submission Global Job Queue Grid Client Submission User Interface Global DH Services SAM Naming Server SAM Log Server Resource Optimizer SAM DB Server RCMetaData Catalog Bookkeeping Service SAM Stager(s) SAM Station (+other servs) Data Handling Worker Nodes Grid Gateway Local Job Handler (CAF, D0MC, BS,...) JIM Advertise Local Job Handling Cluster AAA Dist.FS Info Manager XML DB server Site Conf. Glob/Loc JID map... Info Providers MDS MSS Cache Site Web Serv Grid Monitoring User Tools Flow of: jobdata meta-data

11 March 2004Getting Ready for the Grid Conclusions CDF has embraced the need for the Grid to achieve its physics mission SAM is working for D0 and growing in CDF