1. Error Scope on a Computational Grid Douglas Thain University of Wisconsin 4 March 2002

2. Overview  We have added a Java Universe to Condor. (More from Todd.)  Adding this code forced us to think about the fundamental problem of coupling systems and representing errors.  A lesson: One must consider the scope of an error as well as its detail.

3. Java for Scientific Computing  Java is emerging as a tool for large scale (Grande) scientific computing. More accessible to domain scientists. Simplified porting. Faster development, debugging.  User communities are forming: ACM Java Grande Conference The Java Grande Forum

4. The Hype:  Java: “Write once, run anywhere!”  Condor: “Submit once, run everywhere!”  The Grid: Uniform, dependable, consistent, pervasive, and inexpensive computing.

5. The Reality:  Coupling systems is not trivial!  The easy part: Putting java in front of the program name.  The tricky parts: Dealing with unexpected events! –Bad java installation. –Unavailable file system. –Temporary resource exhaustion.

6. Architecture  Execution: User just specifies “java” universe. Execution site gives details of JVM.  I/O: Know all of your files? –Condor transfers whole files for you. Need online I/O? –Link program with Chirp I/O Library. –Execution site provides proxy to home site.

7. startershadow Home File System Execution SiteSubmission Site

8. startershadow Home File System Execution SiteSubmission Site JVM Fork

9. startershadow Home File System Execution SiteSubmission Site JVM Fork The Job

10. JVM Fork startershadow Home File System I/O Library The Job I/O ServerI/O Proxy Secure Remote I/O Local System Calls Local I/O (Chirp) Execution SiteSubmission Site

11. Initial Experience  Bad news: Nearly any unexpected failure would cause the job to be returned to the user: Out of memory at execution site. Java misconfigured at execution site. I/O proxy can’t initialize. Home file system offline.

12. What do Users Want?  This was correct in a certain sense: The information was true. But, still frustrating.  Users want to know when their program fails by design (NullPointerException,) but not if it fails due to the environment.

13. What Did We Do Wrong?  We thought that we were very careful to propagate errors: I/O errors: server->proxy->library->job JVM exit code: JVM->starter->home  But, we failed to draw a distinction: Errors that are a natural property of the program. Errors that were an incidental result of the environment.

14. Scope and Detail  The scope of an error is the portion of the system that it invalidates.  The detail of an error describes its philosophical cause.  An error must be delivered according to the handler that manages its scope.

15. Examples DetailScopeHandler Program exited normally.ProgramUser Null pointer exception.ProgramUser Out of memory.Remote Machine Condor Home file system offline.Home Machine Condor Program image corrupt.JobUser

16. An Example  With this understanding, we reconsidered many elements of the Java Universe.  One example: The JVM exit code is not a useful result. It gives results that ignore error scope.  Solution: Trap the program exit at a higher level. Report the result and scope on a separate channel.

17. JVM Exit Code DetailScopeHandlerExit Code Program exited normally.ProgramUser(x) Null pointer exception.ProgramUser1 Out of memory.Remote Machine Condor1 Home file system offline.Home Machine Condor1 Program image corrupt.JobUser1

18. JVM startershadow Home File System Wrapper I/O Library The Job Result File JVM Result Program Result or Error and Scope Starter Result + Program Result

19. JVM starter shadow Home File System Wrapper I/O Library The Job Result File JVM Result Starter Result + Program Result I/O Proxy Local I/O (Chirp) Errors of Larger Scope Errors Inside Program Scope

20. Conclusion  We started building the Java Universe with some naive assumptions about errors.  On encountering practical difficulties, we thought more abstractly about errors and developed the notion of scope and detail.  By routing errors according to their scope, we made the system more robust and usable.  Details in an upcoming paper.

21. Deeper Problems  Systems have deep semantic differences that cross multiple functions.  Consider this self-cleaning program: Open a file. Delete the file. Close the file.  Works on UNIX, fails on WinNT.  Can we really provide a uniform interface?

22. More Info:  Demo on Wednesday Morning Room 3381 CS anytime  The Condor Project: http://www.cs.wisc.edu/condor  These slides: http://www.cs.wisc.edu/~thain  Douglas Thain thain@cs.wisc.edu  Questions now?