1. 1 Data Management for Internet Backplane Protocol by Tang Ming Assoc/Prof. Francis Lee School of Computer Engineering, Nanyang Technological University, Singapore

2. 2 0. Outline Introduction The Logistical Networking & Globus Replica Catalog Data Management for IBP

3. 3 1. Introduction Backgrounds of IBP Univ. of Tennessee Base layer of the Logistical Networking A technology to share network storages distributed across the globe Our work Integrate IBP and Globus Replica Catalog, develop the ExNode File Warehouse, and implement a data management system for IBP.

4. 4 2. Logistical Networking IBP: a middleware to allow the sharing of distributed storage resources over the network L-Bone: a resource discovery service that maintains a list of public depots and metadata about those depots exNode: a data structure for aggregation storage allocations LoRS: provide API and command line tools

5. 5 2.1 Examples: Upload / Download a File to IBP with LoRS Tools $ lors_upload blues.mp3 –f –c 1 –F 1 File “blues.mp3” is uploaded to IBP and the exNode file “blues.mp3.xnd” is generated. $ lors_download blues.mp3.xnd –f File “blues.mp3” is downloaded The data resources are not shareable to public Need a data management service for data sharing!

6. 6 3. Globus Replica Catalog Provide the replica location information for Data Grid Logical File Name  Physical File Name LFN PFN File1gsiftp://ssim1.sce.ntu.edu.sg/dir/ File2ftp://ssim4.sce.ntu.edu.sg/dir2/ File3irm://vista.sas.ntu.edu.sg/tmp/

7. 7 3.1 Motivation of using GRC Globus is the de facto standard of the Grid system The GRC has been used by many Data Grid projects It’s a way to integrate IBP and the existed Data Grids

8. 8 4. Data Management for IBP System architecture Components The Logistical Networking Globus Replica Catalog (GRC) ExNode File Warehouse (EFW) IBP Data Management (IDM) Workflow examples (upload & download) Application

9. 9 4.1 System Architecture

10. 10 4.2 ExNode File Warehouse (EFW) EFW stores and maintains the exNode files for the data in IBP Also keeps the metadata about the data files, e.g., the expiration dates and time stamps EFW interacts with IDM and GRC client exNode info. (in memory) Storage Main thread Communication thread Expiration checking thread EFW Communication thread client Globus RC …

11. 11 4.3 IBP Data Management (IDM) IDM interacts with the Logistical Networking, GRC, and EFW, and it provides APIs to applications. Functions: upload, download, refresh, delete, query a file or a file set

12. 12 4.4 Workflow – Uploading a File IBP Data Management (IDM) IBP Data Management (IDM) IBP 1 Globus Replica Catalog (GRC) exNode file warehouse (EFW) 8 23 4 5 6 7 User

13. 13 4.5 Workflow – Downloading a File IBP Data Management (IDM) IBP Data Management (IDM) IBP 1 exNode file warehouse (EFW) 8 67 5 4 2 3 User Globus Replica Catalog (GRC)

14. 14 4.6 Support Grid Computing The system provides a global data cache to support Grid computing

15. 15 5. Future Works Develop data discovery and replica selection service Support Globus new service: Replica Location Service (RLS) For further information: http://www.ntu.edu.sg/home5/p148943492/idm.htm Email: mtang@pmail.ntu.edu.sgmtang@pmail.ntu.edu.sg ebslee@ntu.edu.sg

16. 16 Thanks!