1. 瑞士 CMS 实验网格计算 站点建设与运行 陈志凌

2. CMS 实验物理目标 Large Hadron Collider (LHC): – Design values: E CM = 14 TeV Luminosity = 10 34 cm -2 s -1 Compact Muon Spectrometer (CMS) detector Main physics goals: – Search for Higgs – Search for new physics beyond Standard Model: e.g. Supersymmetry, Large extra dimensions, etc. First p-p collisions @7 TeV:30- 3-2010

3. Experimental Challenges At design luminosity: pile up of ~20 events per beam crossing High particle multiplicity: ~ 10 3 charged particles every 25 ns → Large number of channels (~100 Million) → Huge amount data: → 2 PB (2 ×10 6 GB)/year + Monte Carlo samples 3 Tracker: 75 million separate electronic read-out channels The Z+ end of the CMS Tracker with Tracker Outer Barrel completed.

4. WLCG: Distributed Tier Structure Tier 0 Raw data Tier 1 National centres Archival Datasets Re-Reco, Skim Tier 2 Regional groups Simulated event production Analysis Tier 3 Institutes Analysis by local physicist groups Data Acquisition Systems and Trigger Data Acquisition Systems and Trigger CERN Computer Center Germany CSCS DESY RWTH GSI France Italy UK USA PSIGenevaBern Huge collaboration ….

5. Data Transfers and storage are managed by WLCG automatically. Data Flow

6. S WISS T IER -2 AT CSCS

7. Tier-2: From Prototype to Production Swiss Tier-2 cluster ‘Phoenix’ (Phase B) ~1000 CPU cores and 490 TB storages

8. Tier-2: From Prototype to Production 1392 CPU Cores, 1111 TB Storage 10 Gbit/s connection to CERN Based on Infiniband QDR, All nodes are connected to a Sun M9 IB Switch: 40 Gb/s.

9. Swiss CMS Tier-2: Troubleshooting & Performance  2007: SE overload by LoadTests Transfer error/bad transfer performance to FZK Tier-1. problem solved  performance: 1TB/day  2008 Inconsistent monitoring with CMS Dashboard SE stability problem performance: > 6 TB/day  2009-2010: Scheduling and fair-share → new comprehensive policy performance:10-20TB/day The maximum transfer rate > 130 MB/s (10 TB/day) in 2009

10. Swiss CMS Tier-2: Commissioning and Operation Tests & Commissioning:  CMS Tier-2 can use all pledged resource for analysis jobs with high efficiency  The available monitoring capabilities understood Operation 2010:  Good performance  Number of events processed increasing fast, proportional to the luminosity of LHC. Good CPU Efficiency of CMS jobs during Scale Testing for the Experiment Program 2009 (STEP’09)

11. S WISS CMS T IER -3 AT PSI

12. Swiss CMS Tier-3 at PSI Dedicated to meet the analysis demands from the Swiss CMS groups: ETH Zürich, PSI and University of Zürich Optimized for developing and running jobs of end-user physics analysis Phase A: Since Sept.2008 64 CPU Cores 105 TB Storage Phase B: Since Jan.2010 224 CPU Cores 260 TB Storage

13. Scheme of Swiss CMS Tier-3  Similar hardware as Swiss CMS Tier-2  Benefit from the Tier- 2 experience  Optimized for end- user physics analysis:  Advanced local batch system: SGE  Shared-based file system: similar performance to local files. → Makes developing and debugging of analysis programs easier Full load test (CMS Analysis jobs) In 2009 2 weeks monitoring: Analysis jobs from Tier-3 users 2 weeks monitoring: Analysis jobs from Tier-3 users

14. Swiss CMS Tier-3 commissioning and operation SE performance: User analysis jobs read from 2 file servers at 800 MB/s (~100 connections) 6.7.2010 ~1.7 billion events were processed 1. April – 1. November 2010

15. Work Flow on Tier-3 Working on Swiss CMS Tier-3 CRAB CRAB module for SGE Simplify creation and submission of CMS analysis jobs Consistent way to submit jobs to Grid or Tier-3 Local Cluster Sun Grid Engine More flexible More powerful controls Priority Job Dependency… Command line and GUI

16. CMS Tier-3 Local Site Globe CMS Data Management Service CMS Analysis work flow Data Book- keeping DB DBS Data Book- keeping DB DBS Data Location DB DLS Data Location DB DLS LHC Grid Computing Tier-3 Local Cluster Tier-3 Local Cluster Tier-3 Storage Element PhEDEx Globe Data Transfer Agents and Database PhEDEx Globe Data Transfer Agents and Database PhEDEx Local Data Transfer Agents PhEDEx Local Data Transfer Agents File Transfer Service CRAB is a Python program to simplify the process of creation and submission of CMS analysis jobs into a grid environment.

17. A DMINISTRATION M ONITORING U SER S UPPORT

18. CMS Data Organization Physicist’s View Event collection Dataset A set of event collections that would naturally be grouped for analysis To process Events: Find Transfer Access System View Files File Blocks Files grouped into blocks of reasonable size or logical content. To operate files: files blocks Stored in Grid Transfer and Access files in different storage system Manage replicas 18

19. CMS Data Organization Physicist’s View Find “What data exist?” “Where are data located?” Transfer Access CMS Data management Data Bookkeeping System Standardized and queryable info of event data mapping from event collections to files/file blocks Data Location Service maps file blocks to locations PhEDEx Data Transfer and Placement System LCG commands SRM and POSIX-I/O 19 Map Physicist and system views

22. 一些想法： 高能物理计算发展的特点： – 需求推动 – 有限预算 – 广泛合作 工具 模式 经验 – 循序渐进，逐步探索 技术  ，性能  ，  认识  人员  – 集成简单成熟工具 – 国际交流

23. 谢谢！