1. Legion: The Grid OS Architecture and User View Anand Natrajan ( anand@virginia.edu ) Marty Humphrey ( humphrey@cs.virginia.edu ) The Legion Project, University of Virginia ( http://legion.virginia.edu )

2. Grid Environment  Computers  Networks  People  Data  Devices  Disjoint file systems  Disjoint namespaces  Multiple administration domains  Unpredictable load, availability, failures  Security problems

3. Grid OS Requirements Wide-area High Performance Complexity Management Extensibility Security Site Autonomy Input / Output Heterogeneity Fault-tolerance Scalability Simplicity Single Namespace Resource Management Platform Independence Multi-language Legacy Support

4. Legion - A Grid OS

5. Tools MPI / PVM P-space studies - multi-run Parallel C++ Parallel object-based Fortran CORBA binding Object migration Accounting Remote builds and compilations Fault-tolerant MPI libraries Post-mortem debugger Console objects Parallel 2D file objects Collections Licence support

6. Commercial Support - Avaki Corp. Mentat Legion Avaki Web Venture funded Headquartered in Boston Growing number of employees Multi-tiered support offering

7. Protein Folding with CHARMM Molecular Dynamics Simulations 100-200 structures to sample (r,R gyr ) space R gyr 

8. IBM Blue Horizon SDSC 375MHz Power3 512/1184 IBM Blue Horizon SDSC 375MHz Power3 512/1184 Resources Available HP V-class CalTech 440 MHz PA-8700 128/128 HP V-class CalTech 440 MHz PA-8700 128/128 IBM SP3 UMich 375MHz Power3 24/24 IBM SP3 UMich 375MHz Power3 24/24 IBM Azure UTexas 160MHz Power2 32/64 IBM Azure UTexas 160MHz Power2 32/64 Sun HPC 10000 SDSC 400MHz SMP 32/64 Sun HPC 10000 SDSC 400MHz SMP 32/64 DEC Alpha UVa 533MHz EV56 32/128 DEC Alpha UVa 533MHz EV56 32/128

9. Transparent Remote Execution User initiates “run” User/Legion selects site Legion copies binaries Legion copies input files Legion starts job(s) Legion monitors progress Legion copies output files

10. Mechanics of CHARMM Runs Legion Register binaries Create task directories & specification Dispatch runs Dispatch more runs

11. Types Of Applications Legacy applications Legion-aware applications –I/O library –2D file object Applications Using Stdgrid Parameter Space Studies Parallel Programs –MPI, PVM, MPL, Basic Fortran Support (BFS)

12. Grid Application Requirements Security Fault-tolerance Heterogeneity Collaboration … Legion supports these and other needs

13. Heterogeneous Runs BT-Med Ocean Model

14. Cross-Organisation Collaboration Different companies Proprietary simulations and data Each needs the other Form virtual partnership

15. Platforms Windows NT, 2K, 98, 95 Sun (Solaris) SGI (Irix, Origin) Intel (Linux, Free BSD) DEC (Unix, Linux) Cray (T90, T3E) IBM (AIX, SP-2) HP (HPUX) Codine LoadLeveler Maui PBS NQS LSF

16. Applications Biochemistry and Molecular Science Information Retrieval Materials Science Climate Modelling Neuroscience Aerospace Astronomy Graphics NPACI - SDSC, UCSD, Caltech, UTexas, Umich, UCB, UVa. DoD MSRCs - NAVO & ARL, NASA Ames

17. User View Command-Line Interface

18. Setup Setup shell environment variables. ~legion/setup.sh OR export LEGION=/home/legion/Legion export LEGION_OPR=/home/maya/OPR. $LEGION/bin/legion_env.sh Specifies where binaries and configuration files can be found Sets root context

19. Login Authentication to system legion_login /users/stephen Currently uses password - other mechanisms, e.g., Kerberos ticket possible Login object (a.k.a. Authentication object) - /users/stephen - is user’s proxy to world Login object generates certificate identifying user

20. Context Space / hostsusershome mach1mach2youmemydir progfile1ttysubdir Unix-like legion_ls legion_pwd legion_cd legion_cat...

21. Context Space Network-wide, transparent file system Location-independent read/write of files Convenient transfer of files between context space and local file system I/O libraries for access Unix-like utilities

22. Context Example legion_ls /

23. Another Context legion_ls /hosts

24. Yet Another Context legion_ls /users

25. More Context Fun

26. Other Context Commands Locate a LOID in context space legion_list_names Locate an object on a machine legion_whereis Find status of an object legion_object_info List metadata of an object legion_list_attributes

27. Status Of An Object legion_object_info -c work

28. Physical Location Of Object legion_whereis -c work

29. Context Space vs. Local Space Local space = your machine’s directory structure –OS-specific, Machine-specific –Use cp, copy, etc. –e.g., C:\Program Files\, /usr/bin, /mnt/disk1 Context space = Legion’s directory structure –OS-independent, Machine-independent –Use legion_cp, etc.

30. Context Space and Local Space Transfer one file from local space to context space legion_cp -localsrc Transfer one file from context space to local space legion_cp -localdest

31. Context Space and Local Space Copying local directory to context space legion_cp -r -localsrc OR legion_import_tree Copying context directory to local space legion_cp -r -localdest

32. Context Space and Local Space Map (not copy!) local directory to context space temporarily legion_export_dir Does NOT make copy of local directory Merely provides Legion-like access to local directory –Use legion_cat on local files

33. Making Context Space… Local sub-directory with Legion NFS daemon –Use cat on context files FTP directory with FTP interface Windows directory with Samba interface URL tree with HTTP interface

34. I/O Performance –X-Axis = number of clients simultaneously performing 1MB reads on 10MB files –Y-Axis = total read bandwidth –Each point = average of multiple runs –Clients = 400MHz Intels, NFS Server = 800MHz Intel

35. Making Context Space… Local sub-directory with Legion NFS daemon –Use cat on context files FTP directory with FTP interface Windows directory with Samba interface URL tree with HTTP interface

36. Flexible Context Space Context Directory Disk e ftp legion_export_dir legion_import_tree Samba NFSHTTP FTP

37. Access Control MayI for each object implements access control on a per-function basis Users named by login object Sets of users grouped by contexts legion_change_permissions [+-rwx] [-v] legion_change_permissions +r /users/fred /home/grimshaw/myfile

38. Access Control Example

39. Unified Console Prog. File TTY User creates tty object User starts running program Legion passes tty LOID to program Program produces stdout, stderr User shares tty LOID User shares tty LOID

40. TTY Object Redirect run-time output to central (or multiple) consoles Connect and disconnect dynamically Debug quickly and simply Monitor status, errors, easily Share console with others legion_tty

41. User View Web Interface

42. Logging In

43. Listing Contents Of A Context

44. Control Window

45. Status Window

46. StdOut Window

47. StdErr Window

48. Listing Classes (Contents of /class)

49. Listing Hosts (Contents of /hosts)

50. List Attributes Of An Object

51. Start A Run

52. Check The Status Of A Job

53. Start An Amber (BioGrid) Run

54. Check The Status Of An Amber Run

56. Graphically Check An Amber Run

57. Interact With Amber Run

60. Start A Hawley-Hydro Run

61. Check The Status Of A Hydro Run

63. Graphically Check A Hydro Run

67. Run RenderGrid Jobs (P-Space Jobs)

69. Check The Status of A RenderGrid Job

70. Check Accounting Logs

71. User View Windows Interface

72. Windows Browser

73. Context Space in Windows  Ability to export local directories into Legion’s context space  Easy-to-use interface  Ability of users to control when shared directories are visible to other users

74. Access Control  Ability of users to specify access control policies  Fine-grained nature of policies  Allow/Deny read access to users or groups  Allow/Deny write access to users or groups  Ease with which access rights can be changed  Speed at which access rights are propagated through Legion space

75. Windows Legion FTP Daemon

76. Windows Job Sandbox

77. Windows Process Control

78. National Legion Net

79. Summary Philosophy –Grid as a Single Virtual Machine –Provide mechanisms; let others build policies Architecture –Object-based, integrated –Default policies for scheduling, security, … User Interfaces –Command-line, Web, Windows, FTP, HTTP, …

80. Future Directions Improved user interfaces More robust system Research activities - University of Virginia Commercial activities - Avaki Corporation Legion-G? Continued participation @GGFs Continued support for nationwide grid, grid applications