ASCC Site Report Simon C. Lin Computing Centre, Academia Sinica

Slides:



Advertisements
Similar presentations
1 International IP Backbone of Taiwan Academic Networks Wen-Shui Chen & Yu-lin Chang APAN-TW APAN 2003 Academia Sinica Computing Center, Taiwan.
Advertisements

Current Testbed : 100 GE 2 sites (NERSC, ANL) with 3 nodes each. Each node with 4 x 10 GE NICs Measure various overheads from protocols and file sizes.
GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.
Grid Computing for High Energy Physics in Japan Hiroyuki Matsunaga International Center for Elementary Particle Physics (ICEPP), The University of Tokyo.
1 A Basic R&D for an Analysis Framework Distributed on Wide Area Network Hiroshi Sakamoto International Center for Elementary Particle Physics (ICEPP),
Status Report on Tier-1 in Korea Gungwon Kang, Sang-Un Ahn and Hangjin Jang (KISTI GSDC) April 28, 2014 at 15th CERN-Korea Committee, Geneva Korea Institute.
BINP/GCF Status Report BINP LCG Site Registration Oct 2009
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Simply monitor a grid site with Nagios J.
ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID Project Phase2 of the ScotGRID Project Phase2 of the.
LCG-2 Plan in Taiwan Simon C. Lin and Eric Yen Academia Sinica Taipei, Taiwan 13 January 2004.
Service Challenge Meeting - ISGC_2005, 26. Apr Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH.
Infrastructure and Applications Development of IPv6 in Taiwan 22nd APAN Meeting Singapore, July 18, 2006 Infrastructure and Applications Development of.
14 Aug 08DOE Review John Huth ATLAS Computing at Harvard John Huth.
KISTI-GSDC SITE REPORT Sang-Un Ahn, Jin Kim On the behalf of KISTI GSDC 24 March 2015 HEPiX Spring 2015 Workshop Oxford University, Oxford, UK.
BNL Facility Status and Service Challenge 3 HEPiX Karlsruhe, Germany May 9~13, 2005 Zhenping Liu, Razvan Popescu, and Dantong Yu USATLAS/RHIC Computing.
Network to and at CERN Getting ready for LHC networking Jean-Michel Jouanigot and Paolo Moroni CERN/IT/CS.
USATLAS dCache System and Service Challenge at BNL Zhenping (Jane) Liu RHIC/ATLAS Computing Facility, Physics Department Brookhaven National Lab 10/13/2005.
National HEP Data Grid Project in Korea Kihyeon Cho Center for High Energy Physics (CHEP) Kyungpook National University CDF CAF & Grid Meeting July 12,
NICI IPv6 Infrastructure Development Status IPv6 Summit in Taiwan 2005 Aug. 23 rd, 2005 Jing-Jou Yen National Center for High-Performance Computing.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
Derek Ross E-Science Department DCache Deployment at Tier1A UK HEP Sysman April 2005.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.
INFSO-RI Enabling Grids for E-sciencE The gLite File Transfer Service: Middleware Lessons Learned form Service Challenges Paolo.
BNL Service Challenge 3 Status Report Xin Zhao, Zhenping Liu, Wensheng Deng, Razvan Popescu, Dantong Yu and Bruce Gibbard USATLAS Computing Facility Brookhaven.
Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.
Participation of JINR in CERN- INTAS project ( ) Korenkov V., Mitcin V., Nikonov E., Oleynik D., Pose V., Tikhonenko E. 19 march 2004.
DataGrid is a project funded by the European Commission under contract IST rd EU Review – 19-20/02/2004 The EU DataGrid Project Three years.
Status of Tokyo LCG tier-2 center for atlas / H. Sakamoto / ISGC07 Status of Tokyo LCG Tier 2 Center for ATLAS Hiroshi Sakamoto International Center for.
Enabling Grids for E-sciencE INFSO-RI Enabling Grids for E-sciencE Gavin McCance GDB – 6 June 2007 FTS 2.0 deployment and testing.
ASCC Site Report Eric Yen & Simon C. Lin Academia Sinica 20 July 2005.
QI Fazhi / IHEP CC HEPiX IPv6 F2F Meeting IPv6 Network Status in IHEP/China QI Fazhi Computing Center, IHEP July 4, 2013.
BNL dCache Status and Plan CHEP07: September 2-7, 2007 Zhenping (Jane) Liu for the BNL RACF Storage Group.
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
ALICE Physics Data Challenge ’05 and LCG Service Challenge 3 Latchezar Betev / ALICE Geneva, 6 April 2005 LCG Storage Management Workshop.
The status of IHEP Beijing Site WLCG Asia-Pacific Workshop Yaodong CHENG IHEP, China 01 December 2006.
SRB at KEK Yoshimi Iida, Kohki Ishikawa KEK – CC-IN2P3 Meeting on Grids at Lyon September 11-13, 2006.
Operations Workshop Introduction and Goals Markus Schulz, Ian Bird Bologna 24 th May 2005.
Service Challenge, 2. Dec Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Bruno Hoeft Progress at GridKa Forschungszentrum Karlsruhe GmbH.
Availability of ALICE Grid resources in Germany Kilian Schwarz GSI Darmstadt ALICE Offline Week.
Servizi core INFN Grid presso il CNAF: setup attuale
Bob Jones EGEE Technical Director
Performance measurement of transferring files on the federated SRB
WLCG IPv6 deployment strategy
ALICE Computing Data Challenge VI
WLCG Tier-2 Asia Workshop TIFR, Mumbai 1-3 December 2006
Measurement team Hans Ludwing Reyes Chávez Network Operation Center
Regional Operations Centres Core infrastructure Centres
The Beijing Tier 2: status and plans
“A Data Movement Service for the LHC”
2nd Asia Tier Centre Forum Summary report 4th April 2017 edoardo
LCG Service Challenge: Planning and Milestones
Status Report on LHC_2 : ATLAS computing
LCG 3D Distributed Deployment of Databases
Ian Bird GDB Meeting CERN 9 September 2003
LCG Deployment in Japan
Status and Plans on GRID related activities at KEK
Service Challenge 3 CERN
Database Readiness Workshop Intro & Goals
Update on Plan for KISTI-GSDC
The INFN TIER1 Regional Centre
Olof Bärring LCG-LHCC Review, 22nd September 2008
Conditions Data access using FroNTier Squid cache Server
The INFN Tier-1 Storage Implementation
Bernd Panzer-Steindel CERN/IT
Grid Canada Testbed using HEP applications
Interoperability of Digital Repositories
R. Graciani for LHCb Mumbay, Feb 2006
LHC Data Analysis using a worldwide computing grid
Summer 2002 at SLAC Ajay Tirumala.
The LHCb Computing Data Challenge DC06
Presentation transcript:

ASCC Site Report Simon C. Lin Computing Centre, Academia Sinica Service Challenge Workshop Apr. 26, 2005

Outline Networking Status ASCC Service Challenge Status Tentative Resource Plan for Tier1 Other Related Activities Site Visit

Taipei Status ASNet* runs one STM16 IPLC from Taipei to Amsterdam One GE Local loop from Amsterdam to Geneva (via NetherLight LightPath Service), second GE is waiting to turn on (in process) will double the IPLC on July 1st (contract will be signed on May 27th) ASCC LCG facilities now have multiple GE uplinks to ASNet’s core router; will be replaced by one or two 10GE during this summer. ASNet connects (10GE*2 + STM64*n) to domestic backbone, a.k.a. TANet/TWAREN joint backbone, to reach T2’s in Taiwan. Every T2 in Taiwan has it’s own 10GE link to the domestic backbone * ASNet (Academic Services Network, as#9264) is the network division and is also the network name that registered in APNIC.

Asia Pacific Link Status How many T2s in Asia Pacific region? Taipei – Tokyo STM-4 IPLC 2*GE to APAN-JP (for JP Universities) Will have GE to SiNet/SuperSiNet (for Tokyo Univ. & KEK) IPLC should be increased before LCG production run. Taipei – Singapore STM-1 IPLC to SingAREN (National Grid Office of Singapore)

Asia Pacific Link Status Taipei – Hong Kong STM-4 IPLC, will upgrade to STM-16 on Feb 1st, 2006 (contract already been signed on March 30th) GE link to CERnet, GE link to CSTnet CERnet : China Education and Research network CSTnet : China Science and Technology network Taipei – Korea Maybe have GE link with KOREN (still work with KOREN engineer)

Technical Issues LightPath How urgent is it for the lightpath between T1 and T2 ? Today, there is no lambda can be brought in AP region (there is only SDH circuits) because of the design of cable systems Can MPLS L2 VPN and/or premium IP services provide same (or almost the same) quality compares to LightPath in the LCG environment ? Even though, ASNet is still surveying the WDM equipments

Networking Topology for LCG (New)

Service Challenge

ASCC-CERN Service Challenge Objectives Ensure the baseline services, framework for problem resolution, and operational/emergency procedures Reliable data transfer, job flooding/exerciser, incident response, interoperability Should reach ~ 200MB under current internetworking status Still working with CERN to get better performance

SC test bed @ ASCC

Testbed 4 nodes 2 GByte Memory ~200GB HD Scientific Linux 3.0.4 ClassicSE configuration / GridFTP Each has 1Gb connectivity (Intel Pro1000/MT NIC)

LAN Performance Test without system tuning [root@lcg00115 sc]# iperf -c lcg00116.grid.sinica.edu.tw -i 5 -w 64M -t 36000 | tee iperf-0426.txt ------------------------------------------------------------ Client connecting to lcg00116.grid.sinica.edu.tw, TCP port 5001 TCP window size: 128 MByte (WARNING: requested 64.0 MByte) [ 3] local 140.109.248.4 port 32819 connected with 140.109.248.5 port 5001 [ 3] 0.0- 5.0 sec 562 MBytes 843 Mbits/sec [ 3] 5.0-10.0 sec 486 MBytes 816 Mbits/sec [ 3] 10.0-15.0 sec 486 MBytes 815 Mbits/sec [ 3] 15.0-20.0 sec 474 MBytes 794 Mbits/sec [ 3] 20.0-25.0 sec 467 MBytes 784 Mbits/sec [ 3] 25.0-30.0 sec 478 MBytes 802 Mbits/sec [ 3] 30.0-35.0 sec 499 MBytes 837 Mbits/sec [ 3] 35.0-40.0 sec 493 MBytes 827 Mbits/sec [ 3] 40.0-45.0 sec 465 MBytes 780 Mbits/sec [ 3] 45.0-50.0 sec 467 MBytes 784 Mbits/sec [ 3] 50.0-55.0 sec 486 MBytes 815 Mbits/sec [ 3] 55.0-60.0 sec 485 MBytes 813 Mbits/sec [ 3] 60.0-65.0 sec 487 MBytes 817 Mbits/sec Up to 800 Mbits/sec with single stream test

WAN Test Taipei-to-CERN without system tuning [root@lcg00115 sc]# iperf -c 128.142.224.200 -i 5 -w 4M -t 36000 ------------------------------------------------------------ Client connecting to 128.142.224.200, TCP port 5001 TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte) [ 3] local 140.109.248.4 port 32774 connected with 128.142.224.200 port 5001 [ 3] 0.0- 5.0 sec 18.3 MBytes 30.7 Mbits/sec [ 3] 5.0-10.0 sec 28.3 MBytes 47.4 Mbits/sec [ 3] 10.0-15.0 sec 20.9 MBytes 35.1 Mbits/sec [ 3] 15.0-20.0 sec 2.00 MBytes 3.36 Mbits/sec [ 3] 20.0-25.0 sec 14.7 MBytes 24.7 Mbits/sec [ 3] 25.0-30.0 sec 14.6 MBytes 24.5 Mbits/sec [ 3] 30.0-35.0 sec 14.0 MBytes 23.4 Mbits/sec [ 3] 35.0-40.0 sec 16.0 MBytes 26.9 Mbits/sec [ 3] 40.0-45.0 sec 14.1 MBytes 23.7 Mbits/sec [ 3] 45.0-50.0 sec 16.3 MBytes 27.3 Mbits/sec [ 3] 50.0-55.0 sec 14.2 MBytes 23.8 Mbits/sec [ 3] 55.0-60.0 sec 16.4 MBytes 27.6 Mbits/sec [ 3] 60.0-65.0 sec 14.4 MBytes 24.2 Mbits/sec [ 3] 65.0-70.0 sec 14.5 MBytes 24.3 Mbits/sec Up to 40Mbits/s in single stream test with small window size & system buffer & NIC buffer

Performance Tuning on High Bandwidth/Latency Production Network

Enabling High Performance Data Transfers High Latency Network RTT is above 300ms Tuning Parameter TCP Window Size System Buffer Size NIC Queue length (txqueuelen)

WAN Test Taipei-to-CERN ------------------------------------------------------------ Client connecting to 128.142.224.200, TCP port 5001 TCP window size: 128 MByte (default) [ 3] local 140.109.248.4 port 33812 connected with 128.142.224.200 port 5001 [ 3] 0.0- 5.0 sec 95.4 MBytes 160 Mbits/sec [ 3] 5.0-10.0 sec 485 MBytes 814 Mbits/sec [ 3] 10.0-15.0 sec 528 MBytes 886 Mbits/sec [ 3] 15.0-20.0 sec 450 MBytes 755 Mbits/sec [ 3] 20.0-25.0 sec 500 MBytes 839 Mbits/sec [ 3] 25.0-30.0 sec 502 MBytes 842 Mbits/sec [ 3] 30.0-35.0 sec 424 MBytes 711 Mbits/sec [ 3] 35.0-40.0 sec 500 MBytes 840 Mbits/sec [ 3] 40.0-45.0 sec 502 MBytes 842 Mbits/sec [ 3] 45.0-50.0 sec 427 MBytes 716 Mbits/sec [ 3] 50.0-55.0 sec 503 MBytes 843 Mbits/sec [ 3] 55.0-60.0 sec 500 MBytes 840 Mbits/sec TCP test reaches to 800 Mbits/s, ~77% of max. PCI speed. 128M window size 256M buffersize txqueuelen 1000000

WAN Test Taipei-to-CERN (Cont.) Mbps sec Not stable enough! Need further investigation.

To Do Work with CERN Node Architecture and CPU effects for the performance data from CERN to ASCC Tuning for stable state Testing router by router Node Architecture and CPU effects SMP nodes Itanium CPUs Middleware Testing GridFTP SRM

Resource Plan for T1

LCG in ASCC (35TB) (25TB) (138 CPUs)

Castor Deployment Installed at Apr. 2004 Why Castor Current Status 20 TB SATA Disk Array 35 TB IBM LTO Tape Library Why Castor Support and sharable with TSM, IBM 3494, IBM 3584 SRM interface is ready, but is not good enough in both performance and stability Current Status Version Management of disk cache and tape library

Tentative Resource Plan

Extending Grid Applications Atmospheric Science NTU, NCU and NTNU SRB-based Data Grid, will integrate analysis applications in the next phase Data Grid for Digital Archives Bioinformatics: mpiBLAST-g2 Grid for Medical Imagery (planning) Island-wide Grid Testbed Geospatial Information Infrastructure Biodiversity Informatics

Education and Training Event Date Attendant Remarks China Grid LCG Training May 16-18,’04 40 Beijing, China ISGC’04 Jul.26,’04 50 AS, Taipei Grid Workshop Aug.16-18,’04 Shang-Dong, CN NTHU Dec.22-23,’04 110 Shin-Chu NCKU Mar.9-10,’05 80 Tainan ISGC’05 Apr.25,’05 ~80 Tung-Hai U June, 2005 Tai-chung Hua-Lian/ Tai-Tung Sep., 2005 Planning

IPAS Computer Room Space Plan Resource Plan Movement Plan Layout 60% CPU & Networking 30% Disk 10% Tape Resource Plan Movement Plan Apr. 14 ~ Apr.20 Layout

Layout of Computer Room in IPAS

LCG/EGEE Technology Involvement

ASCC in ARDA (1) gLite Testing gLite WMS monitoring Program has been started to detect WMS efficiency and stability through the heart-beat job operations since Feb. 2005 Realtime report is available at here on ARDA webpage gLite IO monitoring The same philosophy of WMS monitoring The program has been started to collect data since Mar. 2005 (web report is under construction) gLite certification test suite Help on porting 3 testing items from LCG to gLite

ASCC in ARDA (2) ARDA Group in Taiwan ATLAS Distributed Analysis Evaluating and testing the frameworks for ATLAS distributed analysis on LCG and gLite Grid environments Still on the early phase of enabling the frameworks on LCG and gLite ARDA Group in Taiwan Middleware (gLite) Testing & Integration ALTAS CMS

3D -- Distributed Deployment of Databases Objectives Define distributed databases services and application Help to avoid the costly parallel development of data distributions, backup and HA features Minimize number of LCG database administration personnel ASCC Involvement is one of the 3D T1 site. Service Definition and Implementation Group intend to support 3D software development and DB design Incorporation of CA with 3D

CIC and ROC Services Objectives VOMS: CIC Responsibilities Centralized service operations (RB, BDII, VOMS, RLS) Monitoring and Operations Support ROC Responsibilities Monitoring and Operations Support: Deployment support Testbed Operations User support General Management Tools development VOMS: System Establishment AUP reviewing