High Energy Physics & Computing Grids TechFair Univ. of Arlington November 10, 2004

2 What’s the Point? High Energy Particle Physics is a study of the smallest pieces of matter. It investigates (among other things) the nature of the universe immediately after the Big Bang. It also explores physics at temperatures not common for the past 15 billion years (or so). It’s a lot of fun.

3 Fermilab Tevatron World’s Highest Energy proton-anti-proton collider –E cm =1.96 TeV (=6.3x10 -7 J/p  13M Joules on m 2 )  Equivalent to the kinetic energy of a 20t truck at a speed 80 mi/hr Chicago  Tevatron pp p CDF DØ

DØ Detector Weighs 5000 tons Can inspect 3,000,000 collisions/second Will record 50 collisions/second Records approximately 10,000,000 bytes/second Will record 4x10 15 (4,000,000,000,000,000) bytes in the current run (4 PetaByte). 30’ 50’ ATLAS Detector Weighs 10,000 tons Can inspect 1,000,000,000 collisions/second Will record 100 collisions/second Records approximately 300,000,000 bytes/second Will record 1.5x10 15 (1,500,000,000,000,000) bytes each year (1.5 PetaByte).

5 DØ Detector (cross sectional view)

6 Time “parton jet” “particle jet” “calorimeter jet” hadrons  CH FH EM Highest E T dijet event at DØ How does an Event Look in the DØ Detector?

7 How are computers used in HEP? Digital data Data Reconstruction pp p

8 650 Collaborators 78 Institutions 18 Countries DØ Collaboration

9

10 Data Challenges in HEP Enormous data need to be analyzed –1.5 – 2.0 PB / year for ATLAS (PB = Bytes) –Equivalent to 2,857,142 CD-ROMs Data is shared among world-wide collaboration Processing requirements exist for: –Reconstruction of captured data –Cataloging captured data and reconstructions –Sharing cataloged information –Analyzing data –Simulating events The Solution? –Grid Computing

11 What is a Computing Grid? Grid: Geographically distributed computing resources configured for coordinated use Physical resources & networks provide raw capability “Middleware” software ties it together

12 UTA-HEP Grid Collaborations Grid3 – US collaboration led by the International Virtual Data Grid Laboratory ( –Supports ATLAS, CMS, LIGO, SDSS and others –Testbed for large scale collaborations D0SAR- D0 Southern Analysis Region ( –Supports Analysis and Simulation needs in D0 for international group of collaborators –UTA acts as Regional Analysis Center THEGrid – Texas High Energy Grid ( –Sharing computing resources of several Texas universities

13 UTA has the first and the only US RAC UTA is the only US DØ RAC DØSAR formed around UTA Mexico/Brazil OU/ LU UAZ Rice LTU UTA KU KSU Ole Miss

14 UTA-DPCC 100 Pentium 4 Xeon 2.6GHz CPU 64TB of Disk space 84 Pentium 4 Xeon 2.4GHz CPU 7.5TB of Disk space Total CPUs: 193 Total disk: 73TB Total Memory: 189Gbyte

15 UTA Monitoring Applications Developed, implemented and improved by UTA Students Number of Jobs % of Total Available CPUs Time from Present (hours) Anticipated CPU Occupation Jobs in Distribute Queue Commissioned and being deployed

Supervisor –Executors (ATLAS DC2 Production System Powered by UTA Developers) Windmill numJobsWanted executeJobs getExecutorData getStatus fixJob killJob Jabber communication pathway executors Don Quijote (file catalog) Prod DB (jobs database) execution sites (grid) 1. lexor 2. dulcinea 3. capone 4. legacy supervisors execution sites (grid) Designed at UTA

17 Grid Production Statistics Figure : Pie chart showing the sites where DC1 single particle simulation jobs were processed. Only three grid testbed sites were used for this production in August Figure : Pie chart showing the number of pile-up jobs successfully completed at various U.S. grid sites for dataset 2001 (25 GeV dijets). A total of 6000 partitions were generated. These are examples of some datasets produced on the Grid. Many other large samples were produced, especially at BNL using batch.

18 Figure :Cumulative number of Monte-Carlo events produced since August, 2003 for the D0 collaboration by remote site.

19 Figure : Percentage contribution toward US-ATLAS DC2 production by computing site. Figure : Integrated CPU-days consumed by US-ATLAS computing sites. From 5/28/04 to 9/10/2004. UTA’s Contribution ~10 CPU years

20 UTA is a Major Player in HEP Grids Organizing regional and international grids DØSAR THEGrid Grid3 Developing production software and monitoring applications Windmill (ATLAS) McFarm (DØ) McPerm, McQue, Pippy, GridView Providing substantial computing resources –U–UTA is largest producer of monte carlo simulations for DØ collaboration –U–UTA is largest producer of data for US-ATLAS collaboration. Proposing the Southwest Tier2 Center for ATLAS computing with OU, LU, UNM (4x our current size)