Nan DUN Kenjiro Taura Akinori Yonezawa The University of Tokyo Sep 12, http://

 Background  Related Work ◦ SSHFS ◦ FUSE  HandyFS ◦ Designs and Implementation ◦ Evaluation and Problems  Future Work Sep 12,

 User explicit sharing operation ◦ rcp, scp [OpenSSH] ◦ FTP [J. Postel, et al ], GridFTP [Globus]  Conventional Distributed File System ◦ Inner-Cluster Sharing  NFSv3, NFSv4 ◦ Inter-Cluster Sharing  AFS [OpenAFS], Gfarm [Tatebe et al. ‘04] Sep 12, http://

 Complexity of conventional DFSs ◦ Client-Server Configurations ◦ Administrative Privilege ◦ Typical Firewall Setting of the Grid  Simplicity of SSHFS [ ◦ Only need available SSH connection ◦ Client-side mount only ◦ User-level Operations without privileges Sep 12,

 Create local mount point ◦ mkdir mnt_chiba  Mount remote directory to mount point ◦ sshfs chiba000.intrigger.nii.ac.jp:/sharedir mnt_chiba  Access remote files via local mount point ◦ [ls, rm, cp, mkdir] mnt_chiba Sep 12,

Sep 12, fuse hostA_mountpointhostB_mountpoint SSH client hostA SSHFS User Application General File System Call SSH Commands Translation fuse SSH client hostB

Sep 12, Data Transfer Rate (Mbps) SSHFS-FUSE overhead

 Disadvantages of Merely Using SSHFS ◦ Non-scalable  Users have to create many mount points for hosts ◦ Non-unified namespace  Files under different mount points (hosts) should be manipulated separately  Motivation of HandyFS ◦ Inherit simplicity from SSHFS ◦ Create a scalable mount for many hosts ◦ Ad-hoc create a unified namespace for files on many hosts Sep 12,

Sep 12,  Create ONE local mount point ◦ mkdir mnt_all  Mount MULTIPLE remote directories to mount point ◦ handyfs chiba000.intrigger.nii.ac.jp:/sharedir1 chiba001.intrigger.nii.ac.jp:/sharedir mnt_all  Access remote files via local mount point ◦ [ls, rm, cp] mnt_all

Sep 12, FUSE Kernel Module User Application HandyFS SSHFS Host1 Host2 User Kernel SSHFS The trick is using FUSE twice 1. SSHFS mount for remote host 2. Local mount for directory merge /handyfs_mnt/sshfs_mnt1

Source directory Tree at HostB Source directory Tree at HostA  Directories Merge ◦ Source directories and Target (virtual) directory  Source directory: SSHFS mount points for each host  Target directory: Directory in HandyFS namespace Sep 12, /mnt_hostA /A /B /B/C /mnt_hostB /B /C /B/D / / /A /B /B/C Virtual directory Tree /C /B/D

 Directories Merge ◦ Source directories and Target (virtual) directory ◦ Directory Table  To remember virtual directory is merged from which source directory ◦ On-Demand Merge Operation  Merge operation occurs only when request reach non- merged path Sep 12, /cmndir /srcdirA/cmndir Virtual DirectorySource Directories /srcdirB/cmndir Implemented by using BerkeleyDB to achieve good performance

 File Lookup Sep 12, Lookup a file with full pathname Retrieve source directory entries from directory table Lookup file in each retrieved source directory entries If target pathname has not been merged, then merge and store into directory table Return result to user On-demand Merge

 Directory Split ◦ Remove one or several source directories from virtual directory, we have two alternatives  Remove directory table entries  Heavy Operation: Search entire directory table  Re-insert overhead if source become available again  Blind user view of source directories by filtering  Use a in-memory filter to mark current available sources  File lookup results are filtered before returning to user  Efficient and re-usable Sep 12,

 Directory Split Sep 12, Lookup a file with full pathname Retrieve source directory entries from directory table Lookup file in each retrieved source directory entries If target pathname has not been merged, then merge and store into directory table Return result to user Filtering out results that are from removed source directories

 File Creation ◦ File is created at random source directory ◦ For directory file, an entry is created in directory table  Directory Read ◦ Similar as file lookup  Lookup target directory  Read all entries from source directories that composed of target directory Sep 12,

 Some Practical Problems ◦ Same file from multiple sources  Not compatible with UNIX file semantics  Define priority of source directory  Only files from source directory with high priority are visible to user ◦ Stale of Directory Table  Remote directory tree may be modified by its local operations  Periodically update Sep 12,

 Experimental Environments ◦ InTrigger Platform: tokyo.ac.jp/intrigger/ tokyo.ac.jp/intrigger/ Sep 12, Hardware/SoftwareSpecification CPUCore2 Duo 2.13Hz Memory4GB NetworkGigabit Ethernet OSLinux FUSE Kernel Interface7.8 FUSE Library2.7.0 SSHFS1.8

 Experimental Results ◦ Data Transfer Rate Sep 12, SSHFS-FUSE overhead TWICE-FUSE- MOUNT overhead

 Experimental Results ◦ Source Compilation Sep 12, Wide-Area Latency SSHFS Overhead Wide-Area Latency SSHFS Overhead TWICE-FUSE- MOUNT overhead

 Performance ◦ HandyFS itself  To remove MOUNT-TWICE overhead ◦ SSHFS performance  Need be tuned at source level  Scalability ◦ “One-to-All” mount  “All-to-All” mount Sep 12,