UK e-Science All Hands Meeting 2005 Paul Groth, Simon Miles, Luc Moreau

UK e-Science All Hands Meeting 2005 Outline  Process Documentation for Provenance  Power of the P-Structure  P-assertion Recording Protocol  PReServ’s Functionality  Performance  Pitch

UK e-Science All Hands Meeting 2005 Provenance  The Provenance Question –Lots of definitions… –Boil it down to a question. –What is the process that led to a particular result?  How do we answer this question? –Search through documentation.

UK e-Science All Hands Meeting 2005 Documentation  Process Documentation –encompasses all other documentation  SOA based model of process  Actors communicate via message passing  Actors make ASSERTIONS to document process. Termed p-assertions.  Three types of p-assertions

UK e-Science All Hands Meeting 2005 M1 M2 M3 M4 Actor 1 Actor 2 I received M1, M4 I sent M2, M3 I received M3 I sent M4 From these p-assertions, we can derive that M3 was sent by Actor 1 and received by Actor 2 (and likewise for M4) If actors are black boxes, these assertions are not very useful because we do not know dependencies between messages Interaction P-assertions

UK e-Science All Hands Meeting 2005 M1 M2 M3 M4 Actor 1 Actor 2 M2 is in reply to M1 M3 is caused by M1 M2 is caused by M4 M4 is in reply to M3 These assertions help identify order of messages, but not how data were computed Relationship P-assertions

UK e-Science All Hands Meeting 2005 f M1 M2 M3 M4 Actor 1 Actor 2 f1 f2 M3 = f1(M1) M2 = f2(M1,M4)M4 = f(M3) These assertions help identify how data is computed, but provide no information about non-functional characteristics of the computation (time, resources used, etc) Relationship P-assertions

UK e-Science All Hands Meeting 2005 M1 M2 M3 M4 Actor 1 Actor 2 I used 386 cluster Request sat in queue for 6min I used sparc processor I used algorithm x version x.y.z Actor State P-assertions

UK e-Science All Hands Meeting 2005 P-Structure

UK e-Science All Hands Meeting 2005 P-Structure View

UK e-Science All Hands Meeting 2005 Benefits  Domain independent queries  That are provenance specific  P-structure is a shared logical organisation of p-assertions  Does not prescribe how p-assertions are exactly stored in an implementation.

UK e-Science All Hands Meeting 2005 PReP  Introduces the role of a Provenance Store –A separate conceptual entity that is responsible for maintaining process documentation  PReP specifies how an actor can communicate with the Provenance Store to record p-assertions.  PReP has a number of nice properties. – Statelessness – Idempotence – Termination

UK e-Science All Hands Meeting 2005 An Implementation  What is PReServ? –A Web Services implementation of a Provenance Store –Implements PReP for recording XQuery for querying –Provides libraries and wrappers for making applications provenance aware.

UK e-Science All Hands Meeting 2005 Axis Handler Axis Handler Provenance Store Backend Store Interface Database Store In-Memory Store … Backend Stores PS Client Side Library PS Client Side Library Web Service WS Client Query Actor WS PS Client Side Library WS Calls Java Calls PReServ Implementation Diagram

UK e-Science All Hands Meeting 2005 Implementation cont. Backend Store Interface Java Object DatabaseMemory… Store Plug InQuery Plug In … Dispatcher SOAP Msg

UK e-Science All Hands Meeting 2005 Tuning  PReServ can be tuned to fit application characteristics.  When does your application need to query p-assertions?  Cache and then process P-assertion Processed P-Assertions Wait X milliseconds

UK e-Science All Hands Meeting 2005 Requirements  Apache Tomcat 5.0  Apache Ant  Java 1.5 (1.4 supported with some help)  Pure Java, tested on –Windows –Mac OS X –Debian Linux

UK e-Science All Hands Meeting 2005 Evaluation Deployment  Protein Compressibility Experiment –HPDC’05  Workflow runs under VMWare –deployment consistency –ease of development  Workflow is executed on one machine  PReServ runs on another machine

UK e-Science All Hands Meeting 2005 Recording Performance

UK e-Science All Hands Meeting 2005 Query Performance

UK e-Science All Hands Meeting 2005 Applications

UK e-Science All Hands Meeting 2005 What’s new  New version of PReP –Beyond request-response message exchange patterns  P-structure compatible  XQuery interface  Tuning  Refactored code  Performance tests

UK e-Science All Hands Meeting 2005 Conclusion  The p-structure allows for domain independent, provenance specific queries using XQuery.  Both recording and query times are linear with respect to experiment size.  PReServ has a extensible architecture allowing for further functionality to be easily added.

UK e-Science All Hands Meeting 2005 Download!  Try it out!  Download PReServ 0.2: –The AHM release –Released under Open Source MIT License  –Click software  Contact us, we will try to help you make your application provenance-aware.

UK e-Science All Hands Meeting 2005 Configuration  Redhat Linux 9.1 on VMWare on Windows XP  Pentium P4 2.8 GHZ 1.5 GB RAM  PReServ on another machine –Database backend Berkley JDB  100 Mb local ethernet