PRAGMA19, Sep. 15 Resources breakout Migration from Globus-based Grid to Cloud Mason Katz, Yoshio Tanaka

Let’s review the past 8 years Good things – It was exciting to build a real Grid infrastructure across pacific rim. – Experiences on building PRAGMA Grid gave us a lot of insights. – Experiments on PRAGMA Grid gave us a lot of valuable input. Bad things – Who used, who are using PRAGMA Grid? – Globus-based Grid is still troublesome to use Need to check application run on every site. – Resources (e.g. network, computing resources) are not enough to motivate users to use PRAGMA Grid.

Proposed direction - migration to Cloud - Build once, run everywhere! What a user should do? – Build and test applications on his local machine. – Create a VM image which is able to run on any cloud resource. What a resource provider should do? – Build VM hosting servers.

VM Replication Experiment SDSC VM hosting server AIST VM hosting server AFG VM (original) AFG VM (copy) VM hosting server: Rocks 5.3 Xen roll Avian Flu Grid VM Rocks VM Globus/SGE Autodock Replication updates hostname and IP Compute nodes Network configurations Globus configuration SGE configuration NBCR VM hosting server AFG VM (copy) VM replication

5 Data Service zone GEO Grid System Zones Laboratory zone software develop application develop Cloud Service zone home archive applications, libraries DEM, GRASS, GDAL, WMS, WCS, WFS, CS-W? WPS Google Service map Network SW Internet CS-W? Frontend service portal, workflow,

Agenda of breakouts Have consensus of migration Figure out possible problems on migrating to Cloud – Does anybody need Globus-based Grid?, i.e. is Globus still mandatory? Figure out missing pieces. What do we need to make PRAGMA Cloud a practical infrastructure? – How make it easy to create VM images? Start with creating baseline VM images – Interaction with data cloud – Meta scheduling / resource brokering – Security – Interoperation between VM hosting servers – … Rough schedule and milestones Detailed discussions will be in this afternoon

Steps 1.Identify sites 1.AIST, UCSD, NCHC, JILIN, Osaka, Indiana 2.Identify what baseline images should be published 1.Publish a baseline Condor image (Phil UCSD by PRAGMA20) 2.Gfarm client in baseline image 3.Independent test is necessary 3.Automatic replication (Solve re-configuration / re-forming problems) (Mason by PRAGMA 20) 1.Hopefully by 4.Consider how to share VM images 1.Build a stable Gfarm filesystem. (UCSD, AIST, Tsukuba) 2.Build and use Gfarm image on Virtual clusters at SDSC and AIST. (by PRAGMA20) 1.physical clusters provided by SDSC/AIST. Tatebe will lead deployment) 5.Interoperability testing 1.Future issues 6.Other research issues to be done (by each institution) 1.Security, meta-scheduling, data, …