Design of File System Directory Services Osamu Tatebe Grid Technology Research Center, AIST GFS-WG, GGF10 March 2004, Berlin GGF10 GFS-WG March 2004, Berlin

GGF10 GFS-WG File System Directory Services Virtual directory tree (Grid namespace for files)  Virtual directories  Virtual files Attributes: ACL, times Status: open, lock Virtual file handle (VFH) Operations  Lookup “Path” name to GSH (GSH is a general pointer)  Creation, removal, rename  Management of attributes and status /grid ggfjp datagfs file1 file3 file2 file4 file1file2

GGF10 GFS-WG Example Set of Operations of file system directory services VFH getrootVFH() VFH lookup(VFH, String) Dirent[] getdents(VFH) Int mkdir(VFH, String, Attr[]) Int rmdir(VFH, String) Int access(VFH, Int) Attr[] getattrs(VFH, String[]) Int setattrs(VFH, Attr[]) Int addfile(VFH, String, Attr[]) Int removefile(VFH, String) Int link(VFH, VFH, String) Int rename(VFH, String, VFH, String) Server state management  open/close  lock/lockt/locku

GGF10 GFS-WG Issues not included in the document Error code/error status/exception Specification of attributes  Mandatory, recommended, and named attributes (NFSv4) Optimization for long latency  COMPOUND procedure (NSFv4) Group management  CAS???

GGF10 GFS-WG Federation of file system directory services Scalable, large-scale, and distributed management of file system directory services Two type of federation  Indirect federation using file system table services Client library mounts several file system directory services  Direct federation by linking each other

GGF10 GFS-WG Indirect federation introducing file system table services (1) File system table services  Mount table such as /etc/fstab, or maps of automount /Directory services 1 /usr/localDirectory services 2 /usr/shareDirectory services 3 /usr/local/srcDirectory services 4 /host+hosts

GGF10 GFS-WG Indirect federation introducing file system table services (2) Client library mounts several virtual file systems  Flexible management of a directory tree  Access control between file system directory services relies on client library  renaming and making a “hard” link across directory services is not possible

GGF10 GFS-WG Indirect federation of file system directory services (3) / ggfjp datagfs file1 file3 file2 file4 file1 Service A / aisttsukuba-u gtrcdata file1 file3 file2 file4 file1file2 Service B tsukuba /Service A /jp/tsukubaService B File system table services

GGF10 GFS-WG Direct federation of file system directory services (1) VFH = GSH of factory service + local VFH / ggfjp datagfs file1 file3 file2 file4 file1 Service A tsukuba aisttsukuba-u gtrcdata file1 file3 file2 file4 file1file2 Service B

GGF10 GFS-WG Direct federation of file system directory services (2) Transaction needed across directory services  mkdir, rmdir, rename, and link Operations required by mkdir  Int mkdir(VFH fh, GSH factoryGSH, String name, Attr[] attrs)  VFH createdir(Attr[] attrs, long timeout)  Int commitdir(VFH fh, VFH parentfh, String name)

GGF10 GFS-WG Transaction procedure of mkdir Service AService B mkdir createdir Create a temporal virtual directory new VFH Add the VFH to directory entry temporarily commitdir After checking the entry in the parent directory, commit the creation of the directory (add the parent directory to directory entry) success Make the temporal Entry formal

GGF10 GFS-WG Problems needed to be settled Replication and backup of file system directory services Consistency issue in access permission between a virtual file and the corresponding file data Removal or modification of a file data without notification to file system directory services Consistency issue between file data replicas Interoperability issue with NFS and CIFS

GGF10 GFS-WG Summary and conclusion Proposes example set of operations for file system directory services Two types of federation of directory services  Indirect federation by File system table services  Direct federation of directory services