1. 23 Sept. 2010 Jonghu Lee ( 李鍾厚 ) KISTI GSDC jlee206ATkisti.re.kr

2. Content Introduction to KISTI and GSDC 1 1 Current Status of SACC 2 2 Future Plan of SACC 3 3 2

3. 1. Introduction to KISTI and GSDC Jan. 1962 Jan. 1980 Jan. 1991 Feb. 1991 Apr. 1993 Sep. 1999 Jan. 2001 Jan. 1962 Jan. 1980 Jan. 1991 Feb. 1991 Apr. 1993 Sep. 1999 Jan. 2001 Organized KORSTIC (Korea Scientific & Technology Information Center) Reorganized to KIET (Korea Institute for Industrial Economics and Technology) Merged with KIEI (Korea International Economics Institute) KINITI (Korea Institute of Industry and Technology Information) spun off from KIET Established SERI (System Engineering Research Institute) division under KINITI Founded KORDIC (KORea Research and Development Information Center) Acquisition Super Computing Center from ETRI (Electronics and Telecommunications Research Institute) Established KISTI (Korea Institute of Science and Technology Information) 3

4. Three Main Functions of KISTI Super Computing Center Supercomputing Management and Operation Supercomputing High Speed Research Network National Grid Infrastructure Supercomputing Management and Operation Supercomputing High Speed Research Network National Grid Infrastructure Knowledge Information Center Developing National Portal Systems for Information Resources Developing Next Generation Technology in Information Services Developing National Portal Systems for Information Resources Developing Next Generation Technology in Information Services Information Analysis Center Core Technologies Analysis Core Technologies Feasibility study Foreign Information Trend Analysis Information Analysis System Development Core Technologies Analysis Core Technologies Feasibility study Foreign Information Trend Analysis Information Analysis System Development 4

5. Organization of KISTI 5

6. Location SEOUL Daejeon 160km KISTI 6 Universities 20 government research institutes 10 government-invested institutes 33 private R&D labs 824 high-tech companies 6 Universities 20 government research institutes 10 government-invested institutes 33 private R&D labs 824 high-tech companies Located in the Heart of Science Valley, “Daedeok” Innopolis The DAEDEOK INNOPOLIS complex consists of a cluster of firms that represents a cross-section of Korea's cutting-edge industries, including information technology, biotechnology and nanotechnology. 6

7. Super Computing Center at KISTI RankSiteComputer/Year VendorCoresR max R peak Power 1 Oak Ridge National Laboratory United States Jaguar - Cray XT5-HE Opteron Six Core 2.6 GHz / 2009 Cray Inc. 2241621759.002331.006950.60 2 National Supercomputing Centre in Shenz hen (NSCS) China Nebulae - Dawning TC3600 Blade, Intel X5650, NVidia Tesla C 2050 GPU / 2010 Dawning 1206401271.002984.30 3 DOE/NNSA/LANL United States Roadrunner - BladeCenter QS22/LS21 Cluster, PowerXCell 8i 3.2 Ghz / Opteron DC 1.8 GHz, Voltaire Infiniband / 2009 IBM 1224001042.001375.782345.50 4 National Institute for Computational Scienc es/University of Tennessee United States Kraken XT5 - Cray XT5-HE Opteron Six Core 2.6 GHz / 2009 Cray Inc. 98928831.701028.85 5 Forschungszentrum Juelich (FZJ) Germany JUGENE - Blue Gene/P Solution / 2009 IBM 294912825.501002.702268.00 15 KISTI Supercomputing Center Korea, South TachyonII - Sun Blade x6048, X6275, IB QDR M9 switch, Sun HPC stack Linux edition / 2009 Sun Microsystems 26232274.80307.441275.96 16 University of Edinburgh United Kingdom HECToR - Cray XT6m 12-Core 2.1 GHz / 2010 Cray Inc. 43660274.70366.74 1988: - The 1 st Super Computer in Korea - Cray-2S (2GFlops) 1993: - Cray C90 (16GFlops) 2001: - IBM p690(655.6GFlops) - NEC SX-6(160GFlope) - IBM p690+ (2,655GFlops) 2009: - IBM p595 (5.9TFlops) - SUN B6048 (24TFlops) 7

8. KISTI Super Computing Center Mission Test-bed Center Value-adding Center Resource Center Technology Center Help Korea research communities to be equipped with proper knowledge of super computing Keep securing and providing world-class super computing systems Keep securing and providing world-class super computing systems Make the best use of What the center has to create the new values Make the best use of What the center has to create the new values Validate newly emerging concepts, ideas, tools and system Validate newly emerging concepts, ideas, tools and system 8

9. Global Research Networks ■GLORIAD  GLObal RIng Network for Advanced Applications Development with 10/40Gbps Optical lambda networking  Consortium of 11 Nations: Korea, USA, China, Russia, Canada, the Netherlands and 5 Nordic Countries  Supporting Advanced Application Developments such as HEP, Astronomy, Earth System, Bio-Medical, HDTV etc.  Funded by MEST (Ministry of Education, Science and Technology) of KOREA ■KREONET  Korea Research Environment Open NETwork  National Science & Research Network of Korea, Funded by Government since 1998  20Gbps Backbone, 1 ~ 20Gbps Access Networks 9

10. MISSION Physics 2009 ~Next ~ ALICE STAR Belle CDF Various Bio-informatics Earth Science Astrophysics 10

11. Current Status ALICE Tier-1 Test-bed ALICE Tier-2 KIAF (KIsti Analysis Farm) RAW Data Tier-1 RAW Data Computing and Storage Infrastructure Technology Development Apply Grid Technology to legacy app. support Korean Gov. 11

12. Activities ALICE Tier-1 Test-bed Completed set-up ALICE Tier-1 test-bed this year Will provide official service in a few years Completed set-up ALICE Tier-1 test-bed this year Will provide official service in a few years ALICE Tier-2 Site availability: 98% since Feb. 2009 Belle Providing computing resources for Belle MC production (Grid) Bell to provide their data to KISTI GSDC Providing computing resources for Belle MC production (Grid) Bell to provide their data to KISTI GSDC CDF Providing computing resources under NAMCAF Supporting CDFSoft development Providing computing resources under NAMCAF Supporting CDFSoft development LIGO Set-up LIGO cluster test-bed GBrain Planning to cooperate with global brain research project (mainly from McGill Univ. Canada) 12

13. Members NoNameRole 1Dr. Haengjin JangHead of GSDC 2Dr. Hyungwoo ParkProject Management 3Mr. Jinseung YuTechnical Staff (Network) 4Mr. Heejun YoonTechnical Staff (DBA) 5Dr. Beokyun KimTechnical Staff (Grid) 6Dr. Christophe BonnaudTechnical Staff (Admin) 7Dr. Seokmyun KwonTechnical Staff 8Dr. Seo-Young NohTechnical Staff 9Dr. Jonghu LeeTechnical Staff (STAR contact) 10Dr. Seungyun YuPlanning 11Mr. Kyungyun KimTechnical Staff 12Mr. Seunghee LeeTechnical Staff 13Ms. Tajia HanTechnical Staff 13

14. Current Computing Resources: Cluster Server ClusterSpecMem.NodeCorekSI2k Ce-alice Dell Intel Xeon E5405 2.0 GHz Quad 2 CPU 16GB648 Ce01 HP Intel Xeon E5420 2.5GHz Quad 2 CPU 16GB16128219 Ce02 IBM Intel Xeon E5450 3.0GHz Quad 2 CPU 16GB38304650 This Year IBM Intel Xeon X5650 2.66GHz 6 Core 2 CPU 24GB364321,2 14

15. Current Computing Resources: Storage YearModelDisk/TapePhysical SizeUsable Size 2008 NetApp FAS2050 Disk(SAN)48TB30TB 2009 NetApp FAS6080 Disk(SAN, NAS)334TB200TB This Year Hitachi USP-V Disk(SAN)960TB600TB This Year-Tape100TB Total1442TB930TB 15

16. Some Pictures Newly delivered storage system in June 16

17. Network Architecture 8Gb SAN S/W …. 10GbE S/W … SAN U600TB FC Controller #1Controller #2 PFS Servers 10GbE S/W GLORIAD FC Controller 1,2 SAN U160TB 2009 : 8Gb FC : 1GbEthernet : 10GbEthernet 1G/10GbE S/W FC Controller 1,2 NAS U40TB NAS WN Servers … 1G Grid Servers 10G Grid Servers 1G Grid Servers …. Public Private 2010 IBRIX Fusion Cluster File System 17

18. 2. Current Status SACC Collaboration Provide Service Provide Computing Resources STAR Asian Users (China, India, etc.) 18

19. Network Architecture 1F 신규 2 전산실 S/W 1F 신규 2 전산실 S/W DJ-F10 대전 OME6500 대전 OME6500 시애틀 OME6500 시애틀 OME6500 시애틀 OSR 시애틀 OSR DJ_7609 Cisco7609 DJ_7609 Cisco7609 1F –FDF(3,4) Foundry 2402CF Foundry 2402CF 2F 대용량 데이터팀 1G FDF(3,4) VLAN 124 IP : 134.75.238.1 / 26 VLAN 233 IP : 150.183.233.1 / 24 G3/18 G2/5 G8/21 G0/26 T2/3 G2/11 1G Starproject to BNL KISTI 2F 대용량데이터팀 LAB 3F –FDF(3,4) Foundry SuperX Foundry SuperX Gloriad-KR 10G Trunk Vlan 123, Vlan124 VLAN 123 IP : 134.75.123.1 / 24 10G 1G Lightpath Vlan124 10G Trunk Vlan 123 10G 시애틀 F-10 10G 1G 19

20. Daejeon – Seattle Performance Test(UDP) [root@seattle ~]# iperf -s -u -i 1 ------------------------------------------------- Server listening on UDP port 5001 Receiving 1470 byte datagrams UDP buffer size: 107 KByte (default) ------------------------------------------------- [3] local 134.75.205.21 port 5001 connected with 134.75.204.20 port 41490 [3] 0.0- 1.0 sec 233 MBytes 1.96 Gbits/sec 0.006 ms 0/166343 (0%) [3] 1.0- 2.0 sec 233 MBytes 1.96 Gbits/sec 0.005 ms 0/166533 (0%) [3] 2.0- 3.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166643 (0%) [3] 3.0- 4.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166659 (0%) [3] 4.0- 5.0 sec 234 MBytes 1.96 Gbits/sec 0.007 ms 0/166652 (0%) [3] 5.0- 6.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166664 (0%) [3] 6.0- 7.0 sec 233 MBytes 1.96 Gbits/sec 0.006 ms 0/166363 (0%) [3] 7.0- 8.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166644 (0%) [3] 8.0- 9.0 sec 234 MBytes 1.96 Gbits/sec 0.006 ms 0/166663 (0%) [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec 0.008 ms 0/1665755 (0%) [root@seattle ~]# iperf -s -u -i 1 ------------------------------------------------- Server listening on UDP port 5001 Receiving 1470 byte datagrams UDP buffer size: 107 KByte (default) ------------------------------------------------- [3] local 134.75.205.21 port 5001 connected with 134.75.204.20 port 41490 [3] 0.0- 1.0 sec 233 MBytes 1.96 Gbits/sec 0.006 ms 0/166343 (0%) [3] 1.0- 2.0 sec 233 MBytes 1.96 Gbits/sec 0.005 ms 0/166533 (0%) [3] 2.0- 3.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166643 (0%) [3] 3.0- 4.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166659 (0%) [3] 4.0- 5.0 sec 234 MBytes 1.96 Gbits/sec 0.007 ms 0/166652 (0%) [3] 5.0- 6.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166664 (0%) [3] 6.0- 7.0 sec 233 MBytes 1.96 Gbits/sec 0.006 ms 0/166363 (0%) [3] 7.0- 8.0 sec 234 MBytes 1.96 Gbits/sec 0.005 ms 0/166644 (0%) [3] 8.0- 9.0 sec 234 MBytes 1.96 Gbits/sec 0.006 ms 0/166663 (0%) [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec 0.008 ms 0/1665755 (0%) [root@localhost ~]# iperf -c 134.75.205.21 -u -b 1.8g -i 1 -t 10 ------------------------------------------------------------ Client connecting to 134.75.205.21, UDP port 5001 Sending 1470 byte datagrams UDP buffer size: 64.0 KByte (default) ------------------------------------------------------------ [3] local 134.75.204.20 port 41490 connected with 134.75.205.21 port 5001 [ID] Interval Transfer Bandwidth [3] 0.0- 1.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 1.0- 2.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 2.0- 3.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 3.0- 4.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 4.0- 5.0 sec 234 MBytes 1.96 Gbits/sec [D] Interval Transfer Bandwidth [3] 5.0- 6.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 6.0- 7.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 7.0- 8.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 8.0- 9.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec [3] Sent 1665755 datagrams [3] Server Report: [ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec 0.008 ms 0/1665755 (0%) [root@localhost ~]# iperf -c 134.75.205.21 -u -b 1.8g -i 1 -t 10 ------------------------------------------------------------ Client connecting to 134.75.205.21, UDP port 5001 Sending 1470 byte datagrams UDP buffer size: 64.0 KByte (default) ------------------------------------------------------------ [3] local 134.75.204.20 port 41490 connected with 134.75.205.21 port 5001 [ID] Interval Transfer Bandwidth [3] 0.0- 1.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 1.0- 2.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 2.0- 3.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 3.0- 4.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 4.0- 5.0 sec 234 MBytes 1.96 Gbits/sec [D] Interval Transfer Bandwidth [3] 5.0- 6.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 6.0- 7.0 sec 233 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 7.0- 8.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 8.0- 9.0 sec 234 MBytes 1.96 Gbits/sec [ID] Interval Transfer Bandwidth [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec [3] Sent 1665755 datagrams [3] Server Report: [ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [3] 0.0-10.0 sec 2.28 GBytes 1.96 Gbits/sec 0.008 ms 0/1665755 (0%) 20

21. Daejeon – Seattle Performance Test(TCP) [root@seattle ~]# iperf -c 134.75.204.20 -i 1 -w 30m -t 60 ------------------------------------------------------------ Client connecting to 134.75.204.20, TCP port 5001 TCP window size: 60.0 MByte (WARNING: requested 30.0 MByte) ------------------------------------------------------------ [ 3] local 134.75.205.21 port 46831 connected with 134.75.204.20 port 5001 [ 3] 0.0- 1.0 sec 52.3 MBytes 439 Mbits/sec [ 3] 1.0- 2.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 2.0- 3.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 3.0- 4.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 4.0- 5.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 5.0- 6.0 sec 20.9 MBytes 176 Mbits/sec [ 3] 54.0-55.0 sec 267 MBytes 2.24 Gbits/sec [ 3] 55.0-56.0 sec 238 MBytes 1.99 Gbits/sec [ 3] 56.0-57.0 sec 270 MBytes 2.26 Gbits/sec [ 3] 57.0-58.0 sec 242 MBytes 2.03 Gbits/sec [ 3] 58.0-59.0 sec 239 MBytes 2.01 Gbits/sec [ 3] 59.0-60.0 sec 270 MBytes 2.27 Gbits/sec [ 3] 0.0-60.1 sec 8.84 GBytes 1.26 Gbits/sec [root@seattle ~]# iperf -c 134.75.204.20 -i 1 -w 30m -t 60 ------------------------------------------------------------ Client connecting to 134.75.204.20, TCP port 5001 TCP window size: 60.0 MByte (WARNING: requested 30.0 MByte) ------------------------------------------------------------ [ 3] local 134.75.205.21 port 46831 connected with 134.75.204.20 port 5001 [ 3] 0.0- 1.0 sec 52.3 MBytes 439 Mbits/sec [ 3] 1.0- 2.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 2.0- 3.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 3.0- 4.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 4.0- 5.0 sec 0.00 Bytes 0.00 bits/sec [ 3] 5.0- 6.0 sec 20.9 MBytes 176 Mbits/sec [ 3] 54.0-55.0 sec 267 MBytes 2.24 Gbits/sec [ 3] 55.0-56.0 sec 238 MBytes 1.99 Gbits/sec [ 3] 56.0-57.0 sec 270 MBytes 2.26 Gbits/sec [ 3] 57.0-58.0 sec 242 MBytes 2.03 Gbits/sec [ 3] 58.0-59.0 sec 239 MBytes 2.01 Gbits/sec [ 3] 59.0-60.0 sec 270 MBytes 2.27 Gbits/sec [ 3] 0.0-60.1 sec 8.84 GBytes 1.26 Gbits/sec 21

22. Current Resources CPUs Intel Xeon E5450 300cores ≈ 960kSI2K (1core = 3.2kSI2K) shared queue Intel Xeon E5450 300cores ≈ 960kSI2K (1core = 3.2kSI2K) shared queue Storages Disk: 100TB Tape: 100TB Disk: 100TB Tape: 100TB Network 10G GLORIAD: Active Software SL10i on the Scientific Linux 5.5 64bits 22

23. 100TB Disks 23

24. Resource Plan 201020112012 CPU (kSI2K)9602,0803,100 Storage Disk (TB)100200300 Tape (TB)100200300 Network~10G (GLORID) 24

25. 3. Future Plan of SACC Before 2010 Jan. 2011 Summer 2011 Before 2010 Jan. 2011 Summer 2011 Installation of additional CPUs (+430 cores) & Tests MOU between KISTI GSDC and STAR Collaboration Start user service for Asian users Collaborations between KISTI GSDC and Asian institutes RUN11 – online data transfer from BNL to KISTI GSDC Installation of additional storages (100TB disk + 100TB tape) 25

26. Our Vision 26

27. Do You Have Any Questions? 27