Challenge in Building National Scale Grid Infrastructure Somsak Sriprayoonsakul Thai National Grid Center Software Industry Promotion Agency (SIPA) Ministry of Information and Communication

Thai National Grid Project A 4 years project under Ministry of Information and Communication Technology 5.5 Million US$ program Start from 13 universities and 1 government organizations Extending rapidly

Partner Institutes Bangkok Chaingmai University (CMU) Chulalongkorn University (CU) Kasetsart University (KU) King Mongkut’s Institute of Technology North Bangkok (KMITNB) King Mongkut’s University of Technology Thonburi (KMUTT) King Mongkut’s Institute of Technology Ladkrabang (KMITL) Mahidol Unversity (MU) Silpakorn University (SU) Meteorological Department of Thailand (TMD) Walailak University (WU) Prince of Songkla University (PSU) Khonkhan University (KKU) Suranaree University of Technology (SUT) Nakhon Ratchasima Khonkhan Chaingmai Nakhonsrithammarat Songkla

Potential Application Domain Life-science Grid Cheminformatics Grid Geoinformatics Grid Digital Archive and E-learning Computational Fluid Dynamics Grid Simulation and Modeling Grid Digital Media and Animation

Pilot Grid R&E Grid Enterprise Grid Pilot Grid and Cluster infrastructure Interoperable Middleware Basic computational Grid Basic Data Grid POC applications Stable Grid and Cluster infrastructure Interoperable Middleware and basic services Basic computational Grid Basic Data Grid Grid Portal Grid applications R&E Non production enterprise app Broad academic adoption Limited commercial adoption Production Grid and Cluster infrastructure Rich set of services Computational Grid Enterprise analysis Grid Data and knowledge Grid Collaborative Grid Broad commercial adoption Capability Time

ปัตตานี ยะลา นราธิวาส แม่ฮ่องสอน เชียงราย ลำปาง พะเยา น่าน แพร่ อุตรดิตถ์ สุโขทัย ลำพูน ตาก กำแพงเพชร เพชรบูรณ์ เลย ชัยภูมิ พิจิตร หนองบัว ลำภู อุดรธานี หนองคาย สกลนคร นครพนม กาฬสินธุ์ มหาสารคาม มุกดาหาร ร้อยเอ็ด ยโสธร อำนาจ เจริญ บุรีรัมย์สุรินทร์ ศรีสะเกษ นครสวรรค์ อุทัยธานี กาญจนบุรี สุพรรณ บุรี ลพบุรี สระบุรี ปราจีนบุรี สระแก้ว ฉะเชิงเทรา จันทบุรี ระยอง ราชบุรี เพชรบุรี ตราด พระนครศรี สิงห์ บุรี ปทุมธานี นนทบุรี สมุทรสงคราม สมุทรสาคร สมุทรปราการ ประจวบ คีรีขันธ์ ชุมพร ระนอง สุราษฏร์ธานี พังงา ภูเก็ต กระบี่ ตรัง สตูล พัทลุง นครนายก เชียงใหม่ นครราชสีมา อุบลราชธานี สงขลา นครศรีธรรมราช Inter-University Network (UniNet) Commission on Higher Education (Center) 24 Public Universities 39 Rajabhat Universities 29 IT Campus 8 Private Universities 13 Mahachulalongkorn Rajawitayalai 9 Other agencies 9 Rajamangala Institue of Technology (44 Campus) Ministry of Education (for Under Higher Education) 155 Mbps- STM1 34 Mbps - E3 8 Mbps - 4*E1 2 Mbps - E1 64 k – 2Mbps CMU SUT MUA SU_NMJ BU PSU Domestic1Gbps Internet Mbps (to Internet Mbps) 2.5 Gbps- STM16 1 Gbps Update on 24 Jan 2006 THAISARN1Gbps MOEnet (K-12) 1Gbps (coming soon)

Grid Infrastructure ThaiRen (Uninet/ThaiSarn) KMITNB KMUTT KMITL TMD WU KU PSU CMU SUT KKUSU MU CU AIT Teraflop Cluster at TNGC 2.5Gb NECTEC

Implementation

CCCC H CCCC H CCCC H CCCC H Network GH Grid Scheduler Super Cluster Grid Model Super Cluster Grid Model Users see grid as a large single cluster

Thai National Grid Standard Software Stack Applications Core:Gamess,Autodock Option:Blast Portal:GridSphere Option:GridPort Management Core:SCMS Option:MOGAS Scheduler Core:SQMS,SGE Option:PBS,Condor,CSF Programming Ninf-G MPI-G2 Grid Middleware Globus Toolkit4.0 Fabric Core:Linux/Rocks on x86_64/32 Option:Windows

Network Building the Grid Grid Middleware Bioinformatics Digital Media HEP Medical

Technical Contact Each ThaiGrid member institute need to assign at least one person as technical contact point for all ThaiGrid activity Technical contact tasks –Software setup (both cluster and Grid) –Contact point for people inside institute Our communication medium are mailing-list, webboard, wiki, skype, and telephone – are the central point for all technical contactshttp://rswg.thaigrid.or.th

ThaiGrid deployment step 1.Site firewall compliance test 2.Site deployment Install NPACI Rocks 4.1 Install ThaiGrid Roll 3.Site setup Grid CA setup Setup of Grid services Register site with Grid Control Center node 4.Site Certify

Site firewall compliance test ThaiGrid using Globus 4.0 software that comes with NAPCI Rocks Focus on using Pre-WS Grid component –2119 – Globus Gatekeeper –2811 – Grid FTP –2135 – Globus MDS –80 – For web access –22 – For SSH access –10001 – – reserved for applications Firewall is a nightmare for Grid deployment –A Firewall testing tools is needed to identify firewall problem

Firewall testing tool A firewall tester program “TGCheckport” is developed. TGcheckport-1.3]$./TGcheckport.sh maeka.ku.ac.th Start testing maeka.ku.ac.th Tue May 16 16:48:31 ICT 2006 determining globus-gatekeeper service [2119]... Pass determining gsiftp service [2811]... Pass determining globus-mds service [2135]... Pass determining scmsweb service [80]... Pass determining globus-mpi-required-spare-port service [ ]... Pass Congratulation, you pass all the test. Your system is ThaiGrid Firewall Compliance. Finish testing Tue May 16 16:48:32 ICT 2006 TGcheckport-1.3]$ maeka.ku.ac.th

Site Deployment Site deployment process –OS installation using –Grid core middleware (Globus) installation –Grid additional middleware installation MPICH-G2, ninf-G, GSI-ssh

Challenge All these must be done on every cluster nodes to ensure compatibility for all components Manual installation is time consuming, costly, and error prone Rocks simplify all of these process due to the roll concept –OS and Middleware installation are highly automatic

Challenge Many Grid middleware components are needed –SGE/NINF-G/GLOBUS/MPICH/SCE –Each has already been packed into Rocks Roll –Nevertheless, to completely install all software require many rolls Solution: every software are bundled to a new ThaiGrid Roll –A Jumbo Roll bundle all standard Grid software together ThaiGrid Roll HPC Grid SGE Java Ninf-G SCE

Site Setup for ThaiGrid Setup host certificates for Grid Metadata Service Setup Enable some services –Jobmanager-sge –Grid-FTP –GSI-SSH Joining with Grid Control Center –Currently using SCMSWeb for central web-base monitoring –Technical contact of each site will send an with URL of cluster monitoring nodes –Grid meta-scheduler on GCC will distribute jobs to each institutes cluster Setup Grid user

Site Setup Traditional Grid setup are based on manual setup –Complicate –Error prone ROCKS 4.1 provide mechanism for automatic grid installation and configuration –Rocks has already configured almost every services for basic Grid system Anyway, some additional manual setup is needed We currently use a well-documented admin guide to provide a step-by-step to join ThaiGrid

Thai Grid Control Center Grid Control Center Software –Monitoring Automatic testing Service Monitoring Accounting System monitoring Netwwork Bandwidth –Grid-level Job scheduling

Thai Grid Control Center (con’t) Automatic Grid services status monitoring Command line for immediate status checking

Site Certify Site certification program –Level 1: Network firewall compliance –Level 2: Middleware compliance (MPI/G2, Ninf-g) –Level 3: Job scheduling compliance –Level 4: Data Grid compliance Standard –Define clear software stack

ThaiGrid Firewall Status Overall : 15 Pass : 12 Partial : 1 Fail : 2

ThaiGrid Middleware Status Overall: 15 Pass : 4 Partial : 1 Fail : 10

Grid Applications ThaiGrid Drug Discovery Infrastructure Grid based data assimilation using RS data Implicit Narrowband Level Set Method for Fire Spread Model The Parallel Simulation Model for Thin Film Deposition Using the DSMC Method Medical Grid Access Grid

ThaiGrid Drug Discovery Infrastructure Partners: –Dr. Jak Saengma, KU –Somsak Sriprayoonsakul, TNGC –IBM Thailand Challenges –From over active compounds available from Thai medicinal plants database, find a smallest set of compounds that has a potential to be used as a drug –Very compute intensive. Several month of computing time Solution: Use grid to increase computing power to times –Distributed screening process to hundred of hosts on the Grid Benefit –Speeding up time to results –Allow scientists to spend more time analyzing results

Grid based data assimilation using RS data Dr. Kiyoshi Honda, AIT Dr. Putchong Uthayopas, THNC Challenges –Using GA based data assimilation for agricultural monitoring –Very long computation time from months to years –Large amount of RS data needed to be moved around and process Solution: –Using Grid to harvest more computing power –Hide data assimilation process behind the service using MPI, GridRPC (ninf-g) Benefit –Speeding up the calculation

Implicit Narrowband Level Set Method for Fire Spread Model Dr. Pallop Huabsomboon Mahidol University Challenges –Simulation of large scale firespred for forest fire –Very time consuming process Solution –Distribute the computation on the Grid Benefit –Expect 10x faster simulation speed –Ability to model much larger problem within a reasonable time Part of TNGP/ Hewlett Packard Join Research Program

The Parallel Simulation Model for Thin Film Deposition Using the DSMC Method Dr. Rardchawadee Silapunt Dr. Tiranee Achalakul KMUTT, Thailand Thin film deposition process is used for many manufacturing industries including very large scale integrated circuits, optics, magnetic, hard and corrosion resistant coatings, micro- mechanics. Grid system will be used to accelerate the computation and simulation process Part of TNGP/ Hewlett Packard Join Research Program

Medical Grid Suwachai Sieanoon, Somkiat Chatchuenyot, Dr. Panjai Tantasnawong Computer Science Department, Silpakorn University To build and develop a distributed information management system using DataGrid technology. Apply the distributed information management system to test on exchanging and managing the Patient Records System, which is record complexity and, different format and large size of data. GRID Services Data Agent Service Registry Meta Data (HL7) XML IMAGE

Access Grid First APEC Virtual Symposium on Pandemic Preparedness using Access Grid, January 20, 2006 Presentation from Medical Organization in many economies –China, Taiwan, USA, Vietnam, Thailand, Singapore, Philippine Access Grid are used as a key communication infrastructure

Future Work Grid Infrastructure –Include “Grid ZeroConf” roll to auto configure the cluster to join ThaiGrid Install additional CA certificates Generate host certificate request Modify yum repository Grid Application –Drug Design application will be deployed fully on ThaiGrid clusters before July –GIS-base Grid-RPC application due on August –More application coming

End

Additional Slides

Technical Contact Each ThaiGrid member institute need to assign at least one person as technical contact point for all ThaiGrid activity Technical contact tasks –Software setup (both cluster and Grid) –Contact point for people inside institute Our communication medium are mailing-list, webboard, wiki, skype, and telephone – are the central point for all technical contactshttp://rswg.thaigrid.or.th

Grid: What and Why Grid is a technology that incorporates distributed computing systems into a massive computing and storage pool with efficient resource sharing. Grid can strengthen national competitiveness in ICT industries as well as promote sciences and engineering research. Grid benefits several application domains: –E-commerce, E-government, E-learning –Digital Media and Animation –Geoinformatics, Bioinformatics –Financial Analysis –Etc.

Solving network problem General problem –Firewall Compliance –Dealing with smart intrusion detection system –Running Grid on Shared REN/Service network –Network underutilized, unbalance bandwidth Develop a network measurement infrastructure –Collaboration PSU/TNGC/KU/Uninet Collaboration with network provider (Uninet)

ThaiGrid Central Monitoring ThaiGrid currently use OpenSCE to monitor the whole ThaiGrid OpenSCE is installed during Rocks installation using SCE Roll Each cluster provides monitored information through HTTP protocol to central monitoring site