Grid Computing Environment Shell By Mehmet Nacar Las Vegas, June 2003

Overview GCEShell (Grid Computing Environment Shell) provides a user view of computational Grid technologies. GCEShell (Grid Computing Environment Shell) provides a user view of computational Grid technologies. GCEShell is a shell-like system portal. GCEShell is a shell-like system portal.  It is inspired by Unix shell environments Using Web Services and OGSA infrastructure as a global operating system. Using Web Services and OGSA infrastructure as a global operating system. Shell commands for manipulating remote files, for listing system resources. Shell commands for manipulating remote files, for listing system resources. GCEShell engine is simple, well defined mechanism for adding new shell commands. GCEShell engine is simple, well defined mechanism for adding new shell commands.

GCEShell Engine The shell engine is the core application The shell engine is the core application  runs client commands  communicates with applications and registries  manages application lifecycles The engine spawns a new thread for each shell command The engine spawns a new thread for each shell command GCEShell involves both local and remote commands. GCEShell involves both local and remote commands.  If given argument is a URL, the command known as remote and WSDL interfaces gives description of the service.  Otherwise, it is local like history, kill, exit etc.

GCEShell Engine cont. The shell engine aggregates all the objects that perform functionality to the shell container. The shell engine aggregates all the objects that perform functionality to the shell container.  Objects are commands, tasks and workflows If the server is down, the request may be repeated according to service priority and importance. If the server is down, the request may be repeated according to service priority and importance. if a part of command arguments fails, either entire command fails or partial result given on demand. if a part of command arguments fails, either entire command fails or partial result given on demand.

GCEShell Engine cont. The base shell context is responsible for creating child shell contexts to hold individual commands and for managing the lifecycle of these child contexts. The base shell context is responsible for creating child shell contexts to hold individual commands and for managing the lifecycle of these child contexts. It also manages communications between the child contexts; that is, the pipes and redirects are functions of the base context. It also manages communications between the child contexts; that is, the pipes and redirects are functions of the base context. The child context will need to implement a listener that gets notified when the command completes on the server. The child context will need to implement a listener that gets notified when the command completes on the server.

Components Figure shows the shell engine’s principal components. Figure shows the shell engine’s principal components. Each component in our implementation is a Java interface with an implementing class. The shell interface has the following components. Each component in our implementation is a Java interface with an implementing class. The shell interface has the following components.  Base Shell  Parser  Workflow Manager  Exception Handler

GCE Interface (User Commands) SOAP Parser Base Shell Exception Handler Local ClientsWorkflow Manager WS Clients Remote Services GCE Shell Engine WSOGSA (1) (5) (3) (2) (4a) (4b) Exception Thrown

Execution Steps A command is caught by the shell engine and divided into the tokens by the Parser. A command is caught by the shell engine and divided into the tokens by the Parser. The workflow manager is responsible for managing the execution of the clients and their arguments. The workflow manager is responsible for managing the execution of the clients and their arguments. Arguments may indicate either local applications such as shell history command, or services that must connect to remote applications. In the latter case, the client must then interact with the remote Web Service as next step. Arguments may indicate either local applications such as shell history command, or services that must connect to remote applications. In the latter case, the client must then interact with the remote Web Service as next step.

GCEShell Tasks GCEShell contains command and context interfaces which must be implemented by new commands. GCEShell contains command and context interfaces which must be implemented by new commands. The shell interface has the following methods. The shell interface has the following methods.  For each attribute, write accessor (get and set) methods.  For execution directions, execute() method.  To kill the command or process, exit() method.  Allowing process to sleep, suspend() method.

GCEShell Commands gce-list is based on WSIL gce-list is based on WSIL gce-copy is Grid FTP gce-copy is Grid FTP gce-ls could be local and remote gce-ls could be local and remote gce-ps could be local and remote gce-ps could be local and remote gce-set is a local command gce-set is a local command gce-history is a local command gce-history is a local command gce-man is a local command gce-man is a local command gce-kill is a local command gce-kill is a local command gce-help is a local command gce-help is a local command

Adding New Commands Commands are similar to jakarta-ant tasks and JXTA ShellCommands in terms of adding new functionalities. Commands are similar to jakarta-ant tasks and JXTA ShellCommands in terms of adding new functionalities. Commands will be well defined by interfaces, so a developer might want to add more commands. Commands will be well defined by interfaces, so a developer might want to add more commands. To plug a new command into shell container, updating property file giving command name and package name pairs. To plug a new command into shell container, updating property file giving command name and package name pairs.

Exception Handling GCEShell has modular and integrated design, to prevent conflicts in terms of using services and crashes. GCEShell has modular and integrated design, to prevent conflicts in terms of using services and crashes. GCEShell, which interact with distributed resources that may become unavailable for a number of reasons. GCEShell, which interact with distributed resources that may become unavailable for a number of reasons. Each exception interacted with related module, but most of them are handled by the workflow manager and base shell. Each exception interacted with related module, but most of them are handled by the workflow manager and base shell.

Exceptions ParserException is thrown, when the command stream consist of unknown syntax parameters or characters. ParserException is thrown, when the command stream consist of unknown syntax parameters or characters. CommandNotFound exception is thrown, if the given command name is not specified in the properties file. CommandNotFound exception is thrown, if the given command name is not specified in the properties file. LocalClientException is thrown when a local command has an internal error, perhaps caused by improper input. LocalClientException is thrown when a local command has an internal error, perhaps caused by improper input. RemoteClientExceptions may be thrown either if the remote command was sent improper input or the remote server is unreachable for any number of reasons. RemoteClientExceptions may be thrown either if the remote command was sent improper input or the remote server is unreachable for any number of reasons.

Discovering Information Systems The Shell Commands are responsible for discovering the service that they need and for communicating with that service. The Shell Commands are responsible for discovering the service that they need and for communicating with that service. Workflow manager coordinates all negotiations with services and adjust timeouts according to priority of specified services. Workflow manager coordinates all negotiations with services and adjust timeouts according to priority of specified services. The shell container is eligible to deal with some possible failures. The shell container is eligible to deal with some possible failures.

Inspection of Services The purpose of involving WSIL is that the base shell needs to inspect web services instantly. The purpose of involving WSIL is that the base shell needs to inspect web services instantly. gce-list The gce-list command examines the inspection.wsil at this location and inspects what WS running and gets back the list of WSDL interfaces. The gce-list command examines the inspection.wsil at this location and inspects what WS running and gets back the list of WSDL interfaces.

Inspection of Services cont. >> gcelist $TORO GCE List Display list of WSDL service description references... Service: Version Service: Version [1] [1] Service: AdminService Service: AdminService [1] [1] Service: Submitjob Service: Submitjob [1] [1] Service: ApplicationDescriptor3 Service: ApplicationDescriptor3 [1] [1] Service: Submitjob Service: Submitjob [1] [1] Service: AdminService Service: AdminService [1] [1] Service: Version Service: Version [1] [1] Service: ApplicationDescriptor3 Service: ApplicationDescriptor3 [1] [1]

Future Work Dynamic class loading needed so that grid users could place new features on run time. Dynamic class loading needed so that grid users could place new features on run time. Support for shell command composition and scripting. Support for shell command composition and scripting. XML-based BPEL4WS for workflow XML-based BPEL4WS for workflow Python Python One of the powers of the shell environment is that new, specialized commands may be created as needed from the basic library of shell commands. One of the powers of the shell environment is that new, specialized commands may be created as needed from the basic library of shell commands.