HEP Grid Computing in China Gang Workshop on Future PRC-U.S. Cooperation in High Energy Physics

Gang Chen/IHEP-CC Agenda Computing Requirements Computing Requirements Grid Deployment Grid Deployment Applications Applications Networking Networking Prospects Prospects

Gang Chen/IHEP-CC BEPCII/BESIII BEPC: Beijing Electron Positron Collider Started in 1989 Started in ~5GeV/C 2~5GeV/C Being upgraded to Dual-Ring, (3~10)×10 32 cm -2 s -1 Being upgraded to Dual-Ring, (3~10)×10 32 cm -2 s Universities and Institutes from 27 Universities and Institutes from Korea, US, UK, Korea, US, UK, Japan, and China. Japan, and China.

Gang Chen/IHEP-CC BESIII Computing Requirements CPU ： ~2000 P-IV/3GHz CPU for data production/reconstruction and physics analysis CPU ： ~2000 P-IV/3GHz CPU for data production/reconstruction and physics analysis Storage ： ~5PB in 5 years. Storage ： ~5PB in 5 years. Network ： for international/domestic communications Network ： for international/domestic communications

Gang Chen/IHEP-CC YBJ-ARGO/AS  International cosmic ray observatories in Tibet. International cosmic ray observatories in Tibet. 200TB raw data per year. 200TB raw data per year. Data transferred to IHEP and processed with 400 CPUs. Data transferred to IHEP and processed with 400 CPUs. Rec. data accessible by collaborators. Rec. data accessible by collaborators.

Gang Chen/IHEP-CC LHC China is involved in all four LHC experiments, China is involved in all four LHC experiments, IHEP in CMS and ATLAS. IHEP in CMS and ATLAS.

Gang Chen/IHEP-CC More Collaborations Members of International Collaborations: Huge computing demands.

Gang Chen/IHEP-CC Grid Vision The GRID: networked data processing centres and ”middleware” software as the “glue” of resources. Researchers perform their activities regardless geographical location, interact with colleagues, share and access data Scientific instruments and experiments (and simulations) provide huge amount of data

Gang Chen/IHEP-CC HEP Grid in China Coordinated by IHEP Coordinated by IHEP Based on EGEE etc. Based on EGEE etc. To build a Tier-2 center at IHEP for LHC, WLCG MoU has been signed To build a Tier-2 center at IHEP for LHC, WLCG MoU has been signed Cooperating among institutes and universities Cooperating among institutes and universities

Gang Chen/IHEP-CC LCG Deployment in China UI CE_torque SE_classicl MON WN1 UI CE_torque SE_dcache BDII PX RB LFC_mysql WN2 SDU VO: ATLAS PC farm UI CE_torque SE_classicl WN1 WN2 PC farm UI WN1 WN2 PC farm PKU VO: CMS CSTNET\CERNET CNICIHEP

Gang Chen/IHEP-CC LCG Beijing Site Operation UI (User Interface) ： gLite UI (User Interface) ： gLite RB (Resource Broker) ： LCG RB (Resource Broker) ： LCG MyProxy ： gLite MyProxy ： gLite SE (SRM/Dcache) ： gLite SE (SRM/Dcache) ： gLite CE (OpenPBS->Torque) ： LCG CE (OpenPBS->Torque) ： LCG MON (R-GMA ) ： gLite MON (R-GMA ) ： gLite BDII ： gLite BDII ： gLite WNs ： gLite WNs ： gLite LFC ： LHC File Catalogue LFC ： LHC File Catalogue CA ： IHEP CA CA ： IHEP CA VOMS ： for BES/YBJ-ARGO VOs VOMS ： for BES/YBJ-ARGO VOs

Gang Chen/IHEP-CC CA and BES VOMS Operation IHEP CA is Unique CA accredited by EUGridPMA and APGridMPA, covering HEP and other grid applications. IHEP CA is Unique CA accredited by EUGridPMA and APGridMPA, covering HEP and other grid applications.

Gang Chen/IHEP-CC Grid Applications in HEP LHC LHC BES-III: Beijing Electron-Positron Collider/Beijing Electron Spectrometer: tau/charm physics BES-III: Beijing Electron-Positron Collider/Beijing Electron Spectrometer: tau/charm physics ARGO-YBJ: China-Italy cosmic ray observatories in Tibet ARGO-YBJ: China-Italy cosmic ray observatories in Tibet D0 D0 …

Gang Chen/IHEP-CC HEP Grid with BES Support

Gang Chen/IHEP-CC CMS PHeDex Data transfer Setup PHeDEX system for Dataset transfer. Setup PHeDEX system for Dataset transfer. Successful test for dataset subscription, data transfer, error recovery,etc. Successful test for dataset subscription, data transfer, error recovery,etc.

Gang Chen/IHEP-CC YBJ-ARGO Computing on LCG MC, Medea+ and Reconstruction in /opt/exp_soft/bes area. MC, Medea+ and Reconstruction in /opt/exp_soft/bes area. Submit jobs via BES VO to Beijing Site. Submit jobs via BES VO to Beijing Site. Results show everything is working. Results show everything is working.

Gang Chen/IHEP-CC Data Processing model Based on the sharing of the resources and the use of synchronized Data Catalogues Based on the sharing of the resources and the use of synchronized Data Catalogues A common ARGO Virtual Organization A common ARGO Virtual Organization A top ARGO-VO BDII and RB A top ARGO-VO BDII and RB The output files in the “local” Data Catalogue The output files in the “local” Data Catalogue Synchronization procedure for the Data Catalogues to have a copy of the files in each site Synchronization procedure for the Data Catalogues to have a copy of the files in each site All information in the experiment Data Base All information in the experiment Data Base

Gang Chen/IHEP-CC ARGO Achievements Use of Gilda infrastructure to test the computing model: GILDA elements installed at IHEP and INFN ROMA TRE Use of Gilda infrastructure to test the computing model: GILDA elements installed at IHEP and INFN ROMA TRE Most of the scripts implementing the data transfer model already designed and implemented Most of the scripts implementing the data transfer model already designed and implemented Soon extensive test Soon extensive test Many requirements specified Many requirements specified

Gang Chen/IHEP-CC Grid VPN between IHEP and USTC Motivation Motivation International bandwidth bottleneck of China universitiesInternational bandwidth bottleneck of China universities Academic users in universities have to use commercial service shared with millions of public users.Academic users in universities have to use commercial service shared with millions of public users. USTC d0ustc MC production remote farm to FNAL SAM Grid running at unstable 50kBps, which is the best can be provided by China Telecom.USTC d0ustc MC production remote farm to FNAL SAM Grid running at unstable 50kBps, which is the best can be provided by China Telecom. Grid won’t be function this way!Grid won’t be function this way!

Gang Chen/IHEP-CC Grid VPN between IHEP and USTC Solution Solution Virtual Private Network (VPN) channel between CSTNET (USTC) and CSTNET (IHEP). d0ustc gain kBps stable net connection.Virtual Private Network (VPN) channel between CSTNET (USTC) and CSTNET (IHEP). d0ustc gain kBps stable net connection. Problem Problem Grid CONDOR CLOBUS does not recognize a VPN IP farm, and there is no standard solution provided by Wisconsin Group yet.Grid CONDOR CLOBUS does not recognize a VPN IP farm, and there is no standard solution provided by Wisconsin Group yet. IHEP, USTC and FNAL computing experts are working on it now. If we success, then we can distribute university LCG Tiers.IHEP, USTC and FNAL computing experts are working on it now. If we success, then we can distribute university LCG Tiers kBps is far from acceptable for Grid, a P2P link between IHEP to FNAL etc is highly desired kBps is far from acceptable for Grid, a P2P link between IHEP to FNAL etc is highly desired.

Gang Chen/IHEP-CC Secure VPN

Gang Chen/IHEP-CC d0ustc is on via USTC-IHEP VPN, much faster and stable still have some problem with gridftp SAM GRID SUMMARY

Gang Chen/IHEP-CC To foster the creation of a intercontinental e-Science community To foster the creation of a intercontinental e-Science community Training peopleTraining people Supporting existing and new applications: LHC, ARGO, Biology…Supporting existing and new applications: LHC, ARGO, Biology… To support interoperable infrastructure for grid operations between Europe and China To support interoperable infrastructure for grid operations between Europe and China A EU funded project of Interconnection & Interoperation of Grid between Europe and China EUChinaGrid

Gang Chen/IHEP-CC Partners 1 Istituto Nazionale di Fisica Nucleare (IT) (coordinator) 2 European Organisation for Nuclear Research CERN (CH) 3 Dipartimento di Biologia - Università di Roma Tre (IT) 4 Consortium GARR (IT) 5 Greek Research & Technology Network (GR) 6 Jagiellonian University – Medical College, Cracow (PL) 7 School of Computer Science and Engineering – Beihang University Beijing (CN) 8 Computer Network Information Center, Chinese Academy of Sciences – Beijing (CN) 9 Institute of High Energy Physics, Beijing (CN) 10 Peking University – Beijing (CN)

Gang Chen/IHEP-CC GILDA sites

Gang Chen/IHEP-CC GILDA sites

Gang Chen/IHEP-CC Job Statistics

Gang Chen/IHEP-CC EUChinaGrid Workshop: June, 2006

Gang Chen/IHEP-CC EUChinaGrid Tutorial: June, 2006 Organized by IHEP Tutors from IHEP and INFN

Gang Chen/IHEP-CC Networking 10 G 155M 

Gang Chen/IHEP-CC Networking 1Gbps

Gang Chen/IHEP-CC GLORIAD Gloriad is an initiative from China, USA and Russia. And now has been extended with Korea, Netherlands, and Canada.Gloriad is an initiative from China, USA and Russia. And now has been extended with Korea, Netherlands, and Canada. Gloriad provides expanded capacity for science and education collaboration (10 Gbps).Gloriad provides expanded capacity for science and education collaboration (10 Gbps). Gloriad is open to more members, especially in Asia Pacific region!Gloriad is open to more members, especially in Asia Pacific region!

Gang Chen/IHEP-CC Users of Gloriad IHEP

Gang Chen/IHEP-CC Network Performance Test IHEP-Taipei IHEP-CERN

Gang Chen/IHEP-CC In near future expand computing power and storage capacity. expand computing power and storage capacity. Participate in LCG SC4 soon. Participate in LCG SC4 soon. Promote HEP applications on Grid platform. Promote HEP applications on Grid platform. Establish more collaborations Establish more collaborations Main issues: Main issues: - Interoperability among other grid projects: CNGrid, ChinaGrid … - Interoperability among other grid projects: CNGrid, ChinaGrid … - Network bandwidth upgrading. - Network bandwidth upgrading.

Gang Chen/IHEP-CC Resource Plan at IHEP Resources Planned Resources CPU (kSI2K) Disk (TB) Tape (TB) Tape (MB/s) WAN (Mb/s) ~50% for BES, ~50% LHC and others:

Gang Chen/IHEP-CC Thank You!