1. IHEP Computing Center Site Report Shi, Jingyan (shi.jingyan@ihep.ac.cn) Computing Center, IHEP

2. Shi,Jingyan/CC/IHEP 2015-11-12 - 2 IHEP Brief introduction The largest(~1000 stuffs) fundamental research center in China with research fields including: The largest(~1000 stuffs) fundamental research center in China with research fields including: Particle Physics ExperimentsParticle Physics Experiments Cosmic Ray/Astrophysics experimentsCosmic Ray/Astrophysics experiments Theoretical PhysicsTheoretical Physics … Scientific projects: Scientific projects: BESIII experiment running on BEPCBESIII experiment running on BEPC ARGO-YBJ experimentARGO-YBJ experiment Daya Bay reactor neutrino experimentDaya Bay reactor neutrino experiment ATLAS, CMS experiment on LHCATLAS, CMS experiment on LHC AMS, HMXT …AMS, HMXT … Beijing SpectrumARGO-YBJ Detector

3. CC-IHEP at a Glance The Computing Center was created in 1980’s The Computing Center was created in 1980’s Provided computing service to BES, the experiment on BEPCProvided computing service to BES, the experiment on BEPC Rebuilt in 2005 for the new projects: Rebuilt in 2005 for the new projects: BES-III on BEPC-IIBES-III on BEPC-II Tier-2 for ATLAS, CMSTier-2 for ATLAS, CMS Cosmic ray experimentsCosmic ray experiments 35 FTEs, half of them for computing facility 35 FTEs, half of them for computing facility Shi,Jingyan 2015-11-12 - 3

4. Shi,Jingyan 2015-11-12 - 4 10Gbit Ethernet 3+ PB disk storage ( Lustre,dpm, dCache) More than 8000 CPU cores 5PB tape storage Login, monitoring, scheduling,AFS,backup… Outside data farms: LHC, YBJ,DAYABAY Local data farm WAN

5. IHEP Campus Network Network upgrade finished last month Network upgrade finished last month IPv4/IPv6 enabled for all usersIPv4/IPv6 enabled for all users Campus wireless network coveredCampus wireless network covered Shi,Jingyan– Kan, Bowen/CC/IHEP 2015-11-12 - 5

6. Shi,Jingyan/CC/IHEP 2015-11-12 - 6 Network connection Daya Bay Shen Zhen Beijing CSTNet Hong Kong IHEP USA GLORIAD 10G ASGC TEIN3 IPv4 10G IPv6 Beijing Tsinghua YBJ EUR. 2.5G 1G 155M 2.5G Others EDU.CN 2.5G

7. Computing Resources Local Cluster & Grid Site Local Cluster & Grid Site Cpu: Cpu: Work node: More than 8000 cpu/coresWork node: More than 8000 cpu/cores Storage: Storage: 3+PB storage3+PB storage  Lustre for local cluster  dCache for CMS  DPM for ATLAS 5PB Tape storage5PB Tape storage Scheduler Scheduler Cluster: PBS + MauiCluster: PBS + Maui Shi,Jingyan 2015-11-12 - 7

8. File system - Lustre 4 MDSs, 39 OSSs, 376 OSTs, 800 client nodes, 135 million files Lustre Version: 1.8.5 ( upgraded in July) and 1.8.6 (with a new mount point) Capacity: 2.8PB IHEP is considering binding Lustre with CASTOR 1.7 using the HSM function provided by Lustre 2.x, however, it seems that the HSM feature has not totally landed in stable version of Lustre

9. Disk Error More than 3000 pieces of disks are used for the storage and 2/3 have expired guarantee period More than 3000 pieces of disks are used for the storage and 2/3 have expired guarantee period 1TB disks: 1972 pieces1TB disks: 1972 pieces 2TB disks: 752 pieces2TB disks: 752 pieces 3TB disks: 432 pieces3TB disks: 432 pieces Disk hard error happened once 3-4 days in average Disk hard error happened once 3-4 days in average Spare parts are limited, especially for 1TB disks Spare parts are limited, especially for 1TB disks Considering a way to upgrade those disks smoothly Considering a way to upgrade those disks smoothly Shi,Jingyan 2015-11-12 - 9

10. Shi,Jingyan– Kan, Bowen/CC/IHEP 2015-11-12 - 10 IHEP - CA Number of issued certificates Number of issued certificates Last update: Apr20 2012 UTC+8 User Certificate Host Certificate Service Certificate Total VALID 1121000212 Revoked 2542411496 EXPIRED 3443076657 IHEPPKUHKUBUAANJNUSEURSGS CNICTSINGHUACCNUSDUNNUNJUUSTC Cooperation Organizations Cooperation Organizations

11. CA need to be upgraded According to the GFD.125, emailAddress (or Email) now MUST NOT be used in subject or issuer DNs. According to the GFD.125, emailAddress (or Email) now MUST NOT be used in subject or issuer DNs. IHEP will follow the steps below to remove the emailAddress. IHEP will follow the steps below to remove the emailAddress. 1. Modify the CP/CPS to comply with GFD.125 2. Generate a new key pair (new root CA cert). Register it to the IGTF repository so that the new root CA cert and related files will be included in the IGTF CA distribution 3. Inform the relying parties that the subject DN of CA certs and EE certs would be changed. New root CA will issue new EE certs 4. Revoke the old CA certificate until all EE certificates issued by the old CA key will be expired Shi,Jingyan 2015-11-12 - 11

12. BEIJING-LCG2 Site report

14. Reliability and Availability

15. Site Operation Some new computing resources will be added 16 old WN will be replaced with new blade before JUL. 2012 HEPSPEC 2006 will rise up from 8000 to 9600 EGEE to EGI migration Testing new system Grid middleware upgrade Keep eyes on other sites’ migration.

16. Question & Thank You