Gfarm v2 and CSF4 Osamu Tatebe University of Tsukuba Xiaohui Wei Jilin University SC08 PRAGMA Presentation at NCHC booth Nov 19, 2008, Austin

Motivation PRAGMA Life Science Group requires worldwide distributed data analysis SDSC in US, KISTI in Korea, Academia Sinica in Taiwan,... Generate simulated data using available compute resources Analyze them depending on site-own interests

Gfarm v2 and CSF4 Open source project Gfarm v2 – worldwide distributed file system CSF4 – metascheduler Site B Job Scheduler File System Site A Job Scheduler File System Metascheduler Worldwide distributed file system

Gfarm Grid File System [CCGrid 2002] Distributed file system that federates storage of each site It provides scalable I/O performance wrt the number of parallel processes and users It supports fault tolerance and avoids access concentration by automatic replica selection It is an open source project hosted by sourceforge.net Gfarm File System /gfarm ggfjp aistgtrc file1file3 file2 file4 file1file2 File replica creation Global namespace mapping

Scalable I/O Performance Decentralization of disk access putting priority to local disk When a new file is created, Local disk is selected when there is enough space Otherwise, near and the least busy node is selected When a file is accessed, Local disk is selected if it has one of the file replicas Otherwise, near and the least busy node having one of file replicas is selected File affinity scheduling Schedule a process on a node having the specified file Improve the opportunity to access local disk

Scalable I/O performance in distributed environment CPU Gfarm file system Cluster, Grid File A network Job A File A Users viewPhysical execution view in Gfarm (file-affinity scheduling) File B Job A Job B File B File system nodes = compute nodes Shared network file system Do not separate storage and CPU (SAN not necessary) Move and execute program instead of moving large-scale data exploiting local I/O is a key for scalable I/O performance User A submits that accessesis executed on a node that has User B submits that accessesis executed on a node that has

What is CSF4 CSF4 is a WSRF compliant meta-scheduler, its first version was released as an execution management service component of Globus Toolkit 4.(2004) It is an open source project. (sourceforge.net)

CSF4 Services CSF4 consists of –Job Service interface for end users to fully control a job –Reservation Service reserve the resources in advance to guarantee the resource availability –Queuing Service represent a specific scheduling policy Plugin mechanism to easily extend scheduling policy –FCFS, SJF plugins –Workflow plugin, data aware plugin –Array job plugin Resource co-allocation by virtual job management

CSF4 Plugin Mechanism CSF4 Plug-in Architecture

Summary Two open source software that are indispensable for distributed data analysis Gfarm v2 distributed file system CSF4 metascheduler Workflow and data-aware plugins enables integration and efficient use Further integration including automatic file replica creation is considered