Download presentation
Presentation is loading. Please wait.
Published byCurtis Flowers Modified over 9 years ago
1
Worldwide Data Processing with SAMGrid
2
As experiments refine their understanding of raw data, a point is reached where it becomes desirable to reanalyze the entire dataset with the latest techniques. For the D0 experiment, the datasets Involved are large: ~250 TB Equivalent to a stack of CDs nearly as tall as the Eiffel Tower
3
Processing such large datasets in a timely manner requires large scale compute resources. A single pass over the full dataset will involve: Reading ~250TB of input Writing ~ 70TB of output Processing ~1 Billion events To complete such a pass within 6 months requires ~3.5THz of PIII equivalent compute capacity
4
SAMGrid provides an ideal platform for mustering the large scale resources needed to do the D0 data reprocessing with over 20 production sites located across North America, Europe, Asia and South America
5
More than a dozen sites worldwide were able to participate in the D0 reprocessing effort by providing a peak compute capacity of over 3.5THz in PIII equivalent units: CCIN2P3 (Lyon)CMS (at FNAL) FermilabFZU (Prague) GridKa (Karlsruhe)Imperial (London) ManchesterOSCER (Oklahoma) SPRACE (Sao Paolo)D0SAR (Texas, Arlington) WestGrid (Vancouver BC)Wisconsin
6
Essential services provided by SAMGrid: Complete meta-computing environment including Grid-level job management based on Condor and Globus Delivery of executables to sites in an encapsulated compute environment suitable for operation on diverse Linux installations Delivery of raw data over WAN to remote installations Transport of output back to FNAL and storage in MSS Bookkeeping of processing history, job success/failure, and job recovery Monitoring facilities for job status, site availability, and error logging
7
Submission screen dump… Site screen dump… Job screen dump… Data flow (FNAL->remote->merge->FNAL) Conclusion, time spent, data processed etc.
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.