Computing Research Center, High Energy Accelerator Organization (KEK) Site Status Report Go Iwai, KEK/CRC, Japan WLCG Tier-2 Workshop Dec. 1 ~ 4, 2006 Tata Institute of Fundamental Research, Mumbai, India

2006/12/1KEK/CRC, Japan, Go Iwai2Outline 1. Introduction1. Introduction –KEK –Network in Japan 2. Grid Deployment2. Grid Deployment –Current Status –Strategy –Hosted VOs: Belle, ILC, APDG and so on 3. Grid Inter-operability3. Grid Inter-operability –NAREGI: NAtional REsearch Grid Initiative –Relationship among LCG, NAREGI and KEK 4. Summary4. Summary

2006/12/1KEK/CRC, Japan, Go Iwai3 Next Topics IntroductionIntroduction –KEK –Network in Japan Grid DeploymentGrid Deployment –Current Status –Strategy –Hosted VOs: Belle, ILC, APDG and so on Grid Inter-operabilityGrid Inter-operability –NAREGI: NAtional REsearch Grid Initiative –Relationship among LCG, NAREGI and KEK SummarySummary

2006/12/1KEK/CRC, Japan, Go Iwai4 KEK Computing Research Center Introduction Overview of CRC Super Computer System Our mission related computing and networkingOur mission related computing and networking –Providing Computing Facility KEK-B/BelleKEK-B/Belle ILCILC J-PARCJ-PARC –Proton Synchrotron K2K, T2KK2K, T2K –long baseline neutrino detection Accelerator designAccelerator design Application at Synchrotron Radiation FacilityApplication at Synchrotron Radiation Facility –material science, life science and etc –Networking –Security –Support for university groups in the field As an Inter University Research Institute CorporationAs an Inter University Research Institute Corporation

2006/12/1KEK/CRC, Japan, Go Iwai5HEPnet-J Originally, KEK organized HEP institutes in Japan to provide the networking among themOriginally, KEK organized HEP institutes in Japan to provide the networking among them –We started from 9600bps DECnet in early 1980’s –KEK is one of the first Internet sites and the first web site in Japan (1983? and 1992) The current network infrastructure is the SuperSINET operated by NII (National Institute of Informatics)The current network infrastructure is the SuperSINET operated by NII (National Institute of Informatics) NII will be upgraded to the SINET3 in April 2007.NII will be upgraded to the SINET3 in April The SINET3 will provide multi-layered network service with Gbps backbone.The SINET3 will provide multi-layered network service with Gbps backbone.Introduction

2006/12/1KEK/CRC, Japan, Go Iwai6 SINET (44nodes) 100 Ｍｂｐｓ～ 1Gbps Super SINET (32nodes) 10Gbps International lines Japan~U.S.A 10Gbps (To:N.Y.) 2.4Gbps (To:L.A.) Japan~Singapore622Mbps Japan~Hong Kong 622Mbps NationalPublicPrivateJuniorCollegesSpecialized Training Colleges Inter-Univ. Res. Inst. Corp. OthersTotal Introduction Tab.: Number of SINET particular Organizations (Feb. 2006) Tab.: Line Speeds HEPnet-J (cont.) Network Topology Map of SINET/SuperSINET(Feb. 2006)

2006/12/1KEK/CRC, Japan, Go Iwai7 Next Topics IntroductionIntroduction –KEK –Network in Japan Grid DeploymentGrid Deployment –Current Status –Strategy –Hosted VOs: Belle, ILC, APDG and so on Grid Inter-operabilityGrid Inter-operability –NAREGI: NAtional REsearch Grid Initiative –Relationship among LCG, NAREGI and KEK SummarySummary

2006/12/1KEK/CRC, Japan, Go Iwai8 Strategy on GRID Grid Deployment Deployment at KEK for major groupsDeployment at KEK for major groups –BELLE Ongoing experimentOngoing experiment –ILC Near future targetNear future target University supportUniversity support –education and training –Deployment at smaller centers HEPNET-J VOHEPNET-J VO Overview of KEK-B accelerator Design of ILC accelerator/detector

2006/12/1KEK/CRC, Japan, Go Iwai9 Recent Events Nov. 2005: HEP Data Grid WorkshopNov. 2005: HEP Data Grid Workshop –training and cooperation in Asia-Pacific region –Lecturer: Dr. Marco La Rosa very good exercise for usvery good exercise for us Mar. 2006: First meeting on NAREGI/EGEE InteroperabilityMar. 2006: First meeting on NAREGI/EGEE Interoperability –launched Inter-OP projects between NAREGI and EGEE talk about NAREGI latertalk about NAREGI later –at CERN Aug. 2006: Belle workshop on GridAug. 2006: Belle workshop on Grid –to share the information among Belle collaborations –at Nagoya Univ. Sep. 2006: Japan-France Workshop on Grid ComputingSep. 2006: Japan-France Workshop on Grid Computing –at IN2PS/Lyon Univ. Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai10 Summary of LCG Deployment JP-KEK-CRC-01 (Pre-Production System)JP-KEK-CRC-01 (Pre-Production System) –since Nov –is registered to GOC, is ready to WLCG (World wide LCG). –is operated by KEK staffs. –Site Role: practice for production system JP- KEK-CRC-02.practice for production system JP- KEK-CRC-02. test use among university groups in Japan.test use among university groups in Japan. –Resource and Component: SL w/ LCG-2.7SL w/ LCG-2.7 –upgrade to gLite-3.0 is done. CPU: 14, Storage: 1TBCPU: 14, Storage: 1TB –Supported VOs: belle, apdg, dteam and opsbelle, apdg, dteam and ops JP-KEK-CRC-02 (Production System)JP-KEK-CRC-02 (Production System) –since early –is registered to GOC, is ready to WLCG. –is outsourced to IBM Co.,Ltd. –Resource and Component: SL or SLC w/ LCG-2.7 –upgrade to gLite-3.0 is done. CPU: 48, Storage: 6TB (w/o including HPSS) –Supported VOs: belle, apdg, atlasj, ilc, dteam and ops JP-KEK-CRC-00 (Testbed System)JP-KEK-CRC-00 (Testbed System) –since Jun –is closed environment in comparison with other sites. easy to access and configure. –Resource and Component: SL w/ gLite-3.0 (100% pure) –Supported VOs: belle, apdg, atlasj and g4med Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai11 Other Grid-related Services We have our own GRID CAWe have our own GRID CA –is started on Feb. 2006, and is recognized by LCG. –is accredited by APGRID PMA –# of issued user certificates: 32 –# of issued host certificates: 77 – VO Membership ServiceVO Membership Service –Supported VOs: apdg is the VO for Asia-Pacific Data Grid.apdg is the VO for Asia-Pacific Data Grid. belle is the VO for Belle experiments.belle is the VO for Belle experiments. atlasj is the VO for Atlas experiments in Japan.atlasj is the VO for Atlas experiments in Japan. g4med is the VO for Geant4 medical application.g4med is the VO for Geant4 medical application. Local Mirror ServiceLocal Mirror Service –SL, SLC, LCG, gLite –It takes ~30 minutes to update by using apt-get with CERN or FNAL repositories. ~3 minutes with KEK repository~3 minutes with KEK repository – Semi-automatic Installation ServiceSemi-automatic Installation Service –WNs can be installed semi-automatically by PXE (Preboot eXecution Environment) and kickstart configuration file. – Site PortalSite Portal – Grid Deployment KEK Grid CA Web Repository

2006/12/1KEK/CRC, Japan, Go Iwai12 Belle VO Started using SRB and LCGStarted using SRB and LCG –LCG site: JP-KEK-CRC-01/02 Data distribution service using SRB-DSIData distribution service using SRB-DSI –Belle already have a few PBs data in total including 100s TB DST and MC Bulk file register helps us: SregisterBulk file register helps us: Sregister we do not move any of themwe do not move any of them –Benefits both for native SRB users and LCG users VO is supported by KEKVO is supported by KEK –Nagoya (JP), Melbourne (AU), Academia Sinica (TW), Krakow (PL) and etc Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai13 New B Factory Computer System Performance(Year)1997~(4years)2001~(5years)2006~(6years) Computing Server （ SPECint2000 rate ） ~100(WS)~1,250(WS+PC)~42,500(PC) Disk Capacity （ TB ） ~4~91,000(1PB) Tape Library Capacity （ TB ） ,500(3.5PB) Work Group server （ # of hosts ） 3+(9) FS User Workstation （ # of hosts ） 25WS +68X 23WS +100PC 128PC Grid Deployment - New B Factory Computer System since March History of B Factory Computer System Moore’s Law: 1.5y=x2.0 4y=x~6.3 5y=x~10

2006/12/1KEK/CRC, Japan, Go Iwai14 Belle Grid Deployment Plan We are planning a 2-phased deployment for BELLE experiments.We are planning a 2-phased deployment for BELLE experiments. –Phase-1: BELLE user uses VO in JP-KEK-CRC-02 sharing with other VOs. JP-KEK-CRC-02 consists of “Central Computing System” maintained by IBM corporation.JP-KEK-CRC-02 consists of “Central Computing System” maintained by IBM corporation. Available resources:Available resources: –CPU: 72 processors (opteron), SE: 200TB (with HPSS) –Phase-2: Deployment of JP-KEK-CRC-03 as BELLE Production System JP-KEK-CRC-03 uses a part of “B Factory Computer System” resources.JP-KEK-CRC-03 uses a part of “B Factory Computer System” resources. Available resources (maximum estimation)Available resources (maximum estimation) –CPU: 2200 CPU, SE: 1PB (disk), 3.5 PB (HSM) This system will be maintained by CRC and NetOne corporation.This system will be maintained by CRC and NetOne corporation. Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai15 Belle Grid Deployment Plan (cont.) We are planning to federate with Japanese universities.We are planning to federate with Japanese universities. –KEK hosts the BELLE experiment and behaves as Tier-0. –Univ. with reasonable resources: full LCG (Tier-1) –Univ. without resources: UI –The central services such as VOMS, LFC and FTS are provided by KEK. –KEK also covers web Information and support service. –Grid operation is co- operated with 1~2 staffs in each full LCG site. Grid Deployment JP-KEK-CRC-02 JP-KEK-CRC-03 University UI University UI University UI University UI University UI University UI University UI University UI University UI Tier-0 Tier-1 deploy at phase-2 preliminary design

2006/12/1KEK/CRC, Japan, Go Iwai16 Other VOs ILCILC –The initial goal is data sharing among institute in Japan, Asia and world-wide. 1-10TB in total1-10TB in total APDGAPDG –is the VO for DataGrid in Asia-Pacific region. Atlas JapanAtlas Japan –is the VO for the ATLAS collaborations in Japan not for world wide ATLAS.

2006/12/1KEK/CRC, Japan, Go Iwai17 Next Topics IntroductionIntroduction –KEK –Network in Japan Grid DeploymentGrid Deployment –Current Status –Strategy –Hosted VOs: Belle, ILC, APDG and so on Grid Inter-operabilityGrid Inter-operability –NAREGI: NAtional REsearch Grid Initiative –Relationship among LCG, NAREGI and KEK SummarySummary

2006/12/1KEK/CRC, Japan, Go Iwai18 NAREGI: NAtional REsearch Grid Initiatives National Research Grid Initiatives (NAREGI)National Research Grid Initiatives (NAREGI) –Apr MEXT funded NAREGI 5 years Project –Lead by Prof. Ken Miura (NII) –Development of Grid infrastructure and an application for promotion of national economy –Target application is nano science and technology for new material design – Players Computing & networking: NII, AIST, TITECComputing & networking: NII, AIST, TITEC Material scientists :IMS, U. Tokyo, Tohoku U., Kyushu U., KEK,..Material scientists :IMS, U. Tokyo, Tohoku U., Kyushu U., KEK,.. Companies: Fujitsu, Hitachi, NECCompanies: Fujitsu, Hitachi, NEC –Distributed facility: Computing Grid up to 100 TFLOPS in total –Extended to 2010 as a part of National Peta-scale Computing Project Grid Inter-operability SuperSINET NII 10 TFLOPS (1,618 CPU) Application Testbed 5 TFLOS (896 CPU) Software Testbed * As of 2004 TokyoNagoyaIMS

2006/12/1KEK/CRC, Japan, Go Iwai19 Collaboration with NAREGI What we expect for NAREGIWhat we expect for NAREGI –Better quality –Easier deployment –Better support in the native language What we need but still looks not in NAREGIWhat we need but still looks not in NAREGI –File/replica catalogue and data GRID related functionalities Need more assessmentsNeed more assessments Comes a little bit lateComes a little bit late –Earlier is better for us We need something working today!We need something working today! Require commercial version of PBS for beta-1Require commercial version of PBS for beta-1 LCG (LHC Computing GRID) is now based on gLite 3.LCG (LHC Computing GRID) is now based on gLite 3. Only middleware available today to satisfy HEP requirementsOnly middleware available today to satisfy HEP requirements –US people are also developing their own DifficultyDifficulty –Support Language gapsLanguage gaps –Quality assurance –Assumes rich man power Grid Inter-operability

2006/12/1KEK/CRC, Japan, Go Iwai20 LCG and NAREGI Inter-operability NAREGI has much interests on interoperability because they came late and they decided to establish in their sideNAREGI has much interests on interoperability because they came late and they decided to establish in their side First meeting at CERNFirst meeting at CERN –March 2006 –NAREGI, LCG and people from KEK Second meeting at GGF TokyoSecond meeting at GGF Tokyo Grid Inter-operability

2006/12/1KEK/CRC, Japan, Go Iwai21 KEK Plan on GRID Inter-op NAREGI will implement LFC on their middlewareNAREGI will implement LFC on their middleware –We assume job submission between them will be realized soon –Share the same file/replica catalogue space between LCG and NAREGI Move data between them using GridFTPMove data between them using GridFTP NAREGI SRB LCG will be tried alsoNAREGI SRB LCG will be tried also –using SRB-DSI Assessments will be done forAssessments will be done for –Command level compatibility (syntax) between NAREGI and LCG –Job description languages –Software in experiments ILC, International Linear Collider, will be a targetILC, International Linear Collider, will be a target – interoperability among LCG, OSG and NAREGI will be required Grid Inter-operability

2006/12/1KEK/CRC, Japan, Go Iwai22 Next Topics IntroductionIntroduction –KEK –Network in Japan Grid DeploymentGrid Deployment –Current Status –Strategy –Hosted VOs: Belle, ILC, APDG and so on Grid Inter-operabilityGrid Inter-operability –NAREGI: NAtional REsearch Grid Initiative –Relationship among LCG, NAREGI and KEK SummarySummary

2006/12/1KEK/CRC, Japan, Go Iwai23Summary We, KEK Computing Research Center, are working for Belle and ILC mainly related GRIDWe, KEK Computing Research Center, are working for Belle and ILC mainly related GRID –Belle has started to use LCG –ILC soon –University supports also 3 LCG sites are in operation at KEK.3 LCG sites are in operation at KEK. Other Grid-related services are in operation also.Other Grid-related services are in operation also. –CA, VOMS, Mirror, Installation and Documentation Grid inter operability between NAREGI and LCG will be established.Grid inter operability between NAREGI and LCG will be established.

Computing Research Center, High Energy Accelerator Organization (KEK) Thank You K. Amako 1,2, J. Ebihara 3, Y. Iida 1, K. Inami 4, K. Ishikawa 5, M. Kaga 4, S. Kameoka 1,2, S. Kawabata 1, K. Kawagoe 6, A. Kimura 7, Y. Kiyamura 6, M. Matsui 5, K. Murakami 1,2, H. Sakamoto 8, T. Sasaki 1,2, S. Suzuki 1, Y. Watase 1, S. Yashiro 1 and H. Yoshida 9 1 High Energy Accelerator Organization (KEK) 2 Japan Science and Technology Agency (JST) 3 SOUM Co.,Ltd. 4 Nagoya University 5 IBM Japan Systems Engineering Co.,Ltd. 6 Kobe University 7 Ashikaga Institute of Technology 8 ICEPP, University of Tokyo 9 Naruto University of Education

Computing Research Center, High Energy Accelerator Organization (KEK) Backup

2006/12/1KEK/CRC, Japan, Go Iwai26 ILC VO GRID for ILC is sponsored byGRID for ILC is sponsored by –GAKUJYUTSU SOUSEI budget (a grant from MEXT) –French-Japan Joint Lab. Program Initial goalInitial goal –As a tool to share data of total size 1~10TB among Institutes in Japan, Asia, and World Wide. Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai27 APDG VO Asia Pacific Data GRIDAsia Pacific Data GRID Collaboration among Academia Sinica(TW), Center for HEP-Korea, University of Melbourne and KEKCollaboration among Academia Sinica(TW), Center for HEP-Korea, University of Melbourne and KEK Regular meetings, workshops and conferencesRegular meetings, workshops and conferences We are seeking tighter collaboration with ASGCWe are seeking tighter collaboration with ASGC –GOC in Asia Grid Deployment

2006/12/1KEK/CRC, Japan, Go Iwai28 ATLAS JAPAN VO federation among ICEPP, Kobe Univ., Nagoya Univ. and KEK.federation among ICEPP, Kobe Univ., Nagoya Univ. and KEK. –Okayama Univ. and Hiroshima-IT are also potential sites VO UsageVO Usage –Testing inter-connectivity among ICEPP, Kobe Univ. and KEK –Testing Function of middleware –Measuring performance of data sharing ATLAS RC is hosted by ICEPP not usATLAS RC is hosted by ICEPP not us Grid Deployment