Toward Multi-Language, Multi-Component Interface Contract Enforcement Workshop on Component-Based High Performance Computing (CBHPC ‘09) November 15-16, 2009 Good morning. This talk presents highlights of the current state of multiple language, multiple component interface contract enforcement using the Babel scientific middleware toolkit. Tammy Dahlgren and Irina Abramova dahlgren1@llnl.gov iabramov@usc.edu Lawrence Livermore National Laboratory, P. O. Box 808, Livermore, CA 94551 This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 LLNL-PRES-420084

Toward Multi-Language, Multi-Component Interface Contract Enforcement Overarching Goal Improve the quality of component-based scientific applications Approach Support specifying interface contracts in SIDL* Integrate automated contract enforcement in the Babel middleware Support enforcement strategies for different software development life-cycle phases Recent Work Hardening and expanding enforcement Demonstrating enforcement with clients and servers in different languages Demonstrating enforcement in multi-component programs The overarching goal of executable interface contracts is help improve the quality of component-based scientific applications. The approach supports the specification and runtime enforcement of SIDL interface contracts thorught theBabel middleware toolkit. A variety of enforcement options are supported for different software lifecycle phases, from testing through deployment. Recent work focused on hardening and expanding the demonstration of enforcement across languages. Preliminary work was also done by Irina this summer on demonstrations of enforcement in multi-component programs. Here is an example of an interface contract written by Irina. I’ll elaborate on contract definitions shortly. *SIDL = Scientific Interface Definition Language

Toward Multi-Language, Multi-Component Interface Contract Enforcement Motivation and Background Contract Enforcement Multi-Component Contract Enforcement This talk is organized as follows: Present an overview of motivation and background for interface contract enforcement; Describe contract enforcement, including snippets from example clients in each language currently demonstrated; and Show highlights from preliminary work on multi-component contract enforcement.

Errors can occur at any of these points! Hidden .…… Scientists need mechanisms to facilitate gaining confidence in software. Focus specifically on helping programs built from components & libraries. = x = y * z sun() Sun Model Algorithm Software Errors can occur at any of these points! <heading> Understanding and predicting complex real-world processes involves developing models that are then translated into algorithm(s) which are codified to produce running software. This is very much a learning process that can involve iterations on any of these phases. Furthermore, errors can occur at any of these points. The focus of executable interface contract support integration is specifically on helping gain confidence in programs built from components (and libraries). Paul Dubois, “Maintaining Correctness in Scientific Programs,” CS&E, May-June 2005.

Different development processes (and rigor)  varying quality .….. Common Component Architecture Forum’s vision of plug-and-play components depends on common interfaces. Each implementation must conform to the same specification. Require same basic inputs Provide same basic services Implementations can vary significantly to include differences in… Algorithms Solution accuracies Underlying data structures Implementation language(s) C, C++ FORTRAN 77, Fortran 90, … Python Java Application Component Repository Component A Component B Component C <heading> Plug-and-play inherently means multiple implementations conforming to the same specification. It also means those implementations must provide the same basic services and require the same basic inputs. However, the implementations could vary significantly to include differences in the algorithms, such as solution accuracies, the underlying data structures, and even the implementation languages. PLUS different organizations and even different projects and individuals within those organizations inherently follow different development processes with varying levels of rigor. Different development processes (and rigor)  varying quality See http://www.scidac.gov/compsci/TASCS.html.

.... Contracts added to common interface specifications can be used to improve the quality of the resulting applications. void getEntArrAdj( in array<opaque> entity_handles, in int entity_handles_size, in EntityType entity_type_requested, inout array<opaque> adj_entity_handles, out int adj_entity_handles_size, inout array<int> offset, out int offset_size) throws TSTTB.Error, sidl.PreViolation, sidl.PostViolation; require /* Precondition clause defines obligations on callers */ passed_entity_handles: entity_handles != null; handles_is_1d: dimen(entity_handles) = = 1; handles_size_okay: irange(entity_handles_size, 0, size(entity_handles)); ensure /* Postcondition clause defines obligations on all implementations */ adj_handles_returned: adj_entity_handles != null; adj_handles_is_1d: dimen(adj_entity_handles) = = 1; irange(adj_entity_handles_size, 0, size(adj_entity_handles)); offset_returned: offset != null; offset_is_1d: dimen(offset) = = 1; offset_size_okay: offset_size = = entity_handles_size + 1; offset_values_okay: irange(offset, 0, offset_size); offset_non_decreasing: nonDecr(offset); claim_no_side_effects: is pure; <heading> Here is an example contract for a single method illustrating various features supported through SIDL contracts. The method is one of the early Terascale Simulation Tools and Technologies (now ITAPS) methods. The contract consists of the precondition and postcondition clauses – specified under require and ensure, respectively. The caller is responsible for satisfying the preconditions, i.e., passing in a 1 dimensional, non-null entity handles array and an appropriate size argument. If the caller satisfies the preconditions, all implementations are obligated to ensure the specified postconditions are satisfied. I’d like to call your attention to four built-in methods for contracts: dimen(), irange(), nonDecr(), and size(). <explain each> In addition, irange() and nonDecr() are linear in the size of their first arguments. dimen(), irange(), nonDecr(), and size() are built-in SIDL functions available to contract clauses. O(|offset|) Source: tstt.sidl v0.7 (http://www.tstt-scidac.org/).

… Concerns over the performance overhead of their enforcement in some software life cycle phases have been addressed. Program Initiation Termination Total Execution Time Component method(s) Preconditions Postconditions Enforcement Overhead ≤ X% ? Invariants <heading> Current features integrated into Babel specifically address the overhead in terms of execution (or wall clock) time. If you could unroll the execution time of a program, from initiation to termination, and consider where time is spent, the issue is how much of it is spent checking the assertions association with precondition, postcondition, and (when present) invariant clauses of the methods actually executed by the program. According to an informal poll*, Computational Scientists are typically willing to incur no more than 10% overhead when adopting new technologies. *Conducted by members of the Common Component Architecture Forum (http://www.cca-forum.org).

Contract Clause Classification* Enforcement Frequency* .. A variety of enforcement policies are supported that include traditional and sampling-based techniques. Enforcement Policy Decision Basis Contract Clause Classification* Enforcement Frequency* Always Baseline AllClasses Never Constant Characteristics Linear Method Calls MethodCalls Postconditions Preconditions Simple Expressions SimpleExpressions Periodic Sampling Random Adaptive Fit Performance AdaptiveFit Adaptive Timing AdaptiveTiming <heading> The intent was to build in a lot of flexibility for addressing the execution time overhead of interface contract enforcement and to provide mechanisms for measuring the characteristics of contracts actually exercised by programs to obtain more data for analysis. Note: The introduction of Adaptive Fit performance-driven sampling. Unlike Adaptive Timing, which continues to make enforcement decisions based on execution time estimates solely on contracts and methods, Adaptive Fit includes the accumulated time spent on program statements in enforcement decisions. *There are actually several more options available, but those listed have been used so far in studies.

. Experiments conducted on multiple single-component programs provided some insights regarding enforcement. Chiefly, sufficient work must be performed within the methods (not, in these cases, in the programs) to offset contract checking. Data Source: Dahlgren, CBSE ’07. *Early experiments conducted using prototype implementation in Babel.

Toward Multi-Language, Multi-Component Interface Contract Enforcement Motivation and Background Contract Enforcement Multi-Component Contract Enforcement

Interface Contract Enforcement Enforcement Execution Traces …….. Enforcement routines are automatically generated and supported by the Babel scientific middleware toolkit. Interface Contract Enforcement Annotated Scientific Interface Definition Language (SIDL) files Stub Code Intermediate Rep Skeleton Code Enforcement Execution Traces Impl Code Component or Library <heading> Annotated SIDL files fed into the Babel compiler results in the generation of language interoperability middleware, including contract check routines in the intermediate representation layer. Timing instrumentation for optional enforcement execution tracing is integrated into the generated contract check routines.

……. The path taken through the generated middleware layers depends on the enforcement policy. Stub layer stub_method() {…} d_epv Application epv cepv Intermediate Object Representation (IOR) layer . . check_method() {…} Skeleton layer skel_method() {…} (External) Component or Library <heading> The path taken through the layers depends on the enforcement policy in affect. Specifically, options disabling contract checking take the standard Babel route bypassing the check methods. All policies enabling enforcement go through the check method. Actually, what’s missing from this diagram is the call from the check method out to the SIDL library to determine whether the given set of checks should or should not be performed, based on the policy in affect and information about the contract [clause]. Implementation layer impl_method() {…} The associated assertion checks are generated in ANSI C in the IOR layer. Enforcement disabled Enforcement enabled

…... Upcoming release of Babel will illustrate clients in each programming language calling implementations in others. package vect version 1.0 { … class Utils { … static bool vuAreEqual(in array<double> u, in array<double> v, in double tol) throws sidl.PreViolation; require not_null_u : u != null; u_is_1d : dimen(u) == 1; not_null_v : v != null; v_is_1d : dimen(v) == 1; same_size : size(u) == size(v); non_negative_tolerance: tol >= 0.0; ensure no_side_effects : is pure; … } } Client C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_size) Note: Java and Python are onlyexercised using dynamic loading; whereas the rest include static loading executions as well. Based on: ${BABEL}/regression/contracts/vect.sidl

….. The C client calls methods implemented in each of the programming languages (during separate executions). #include “sidl_EnfPolicy.h” … int main(int argc, char** argv) { sidl_BaseInterface exception = NULL; struct sidl_double__array *u = NULL; … u = sidl_double__array_create1d(max_size); … sidl_EnfPolicy_setEnforceAll( sidl_ContractClass_ALLCLASSES, TRUE, &exception); SIDL_CHECK(exception); … int x = vec_Utils_vuAreEqual(u, u, (sidl_BaseInterface*)(&exception)); … sidl__array_deleteRef((struct sidl__array*)u); … } Client (C) C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_size) NOTE: Second argument to setEnforceAll() is a boolean for clearing enforcement statistics. Note: Java and Python are onlyexercised using dynamic loading; whereas the rest include static loading executions as well. Based on: ${BABEL}/regression/contracts/runC/vectortest.c

…. The C++ client calls methods implemented in each of the programming languages (during separate executions). #include “sidl_EnfPolicy.hxx” … int main(int argc, char** argv) { … ::sidl::array<double> u = ::sidl::array<double>::create1d(max_size); … ::sidl::EnfPolicy::setEnforceAll( ::sidl::ContractClass_ALLCLASSES, true); … bool x = vect::Utils::vuAreEqual(u, u, tol); … u.deleteRef(); } Client (C++) C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_s Note: Java and Python are onlyexercised using dynamic loading; whereas the rest include static loading executions as well. ize) Based on: ${BABEL}/regression/contracts/runCxx/vectortest.cxx

... The FORTRAN 77 client calls methods implemented in each of the programming languages (during separate executions). … program vectortest implicit none … integer*8 exception, u include ‘sidl_ContractClass.inc’ … call sidl_double__array_createRow_f(1, low, up, u) … call sidl_EnfPolicy_setEnforceAll_f( $ ALLCLASSES, .true., exception); … logical x call vect_Utils_vuAreEqual(u, u, tol, x, exception) … call sidl_double__array_deleteRef_f(u, throwaway) … stop end Client (F77) C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_size) low and up, both integer*4 initialized as low(1)=0, up(1)=MAX_SIZE-1 Note: Java and Python are onlyexercised using dynamic loading; whereas the rest include static loading executions as well. sidl_ContractClass.inc is automatically generated from the ContractClass enumeration in sidl.sidl. Values are defines as: integer*8 ALLCLASSES parameter (ALLCLASSES = 0) for example. Based on: ${BABEL}/regression/contracts/runF77/vectortest.f

.. The Java client calls methods implemented in each of the programming languages (during separate executions). import sidl.EnfPolicy; import sidl.Double.Array1; … public class vectortest { … private sidl.EnfPolicy d_policy = new sidl.EnfPolicy(); private Array1 d_u = new Array1(0, S_MAX_SIZE-1, false); … private void run() throws InterruptedException { … d_policy.setEnforceAll( sidl.ContractClass.ALLCLASSES, true); … boolean x = vect.Utils.vuAreEqual(u, u, tol); … } … } Client (Java) C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_size) Note: Java executions rely on dynamic loading of all other implementations. Based on: ${BABEL}/regression/contracts/runJava/vectortest.java

. The Python client calls methods implemented in each of the programming languages (during separate executions). import sidl.EnfPolicy … def driver(counter): u = zeros((MAX_SIZE, ), float64) sidl.EnfPolicy.setEnforceAll( sidl.ContractClass.ALLCLASSES, TRUE); x = vect.Utils.vuAreEqual(u, u, tol); } Client (Python) C C++ FORTRAN 77 Fortran 90 Java Python u is initialized to 1.0/sqrt(max_size) Note: Python executions rely on dynamic loading of all other implementations. Based on: ${BABEL}/regression/contracts/runPython/vectortest

Toward Multi-Language, Multi-Component Interface Contract Enforcement Motivation and Background Contract Enforcement Multi-Component Contract Enforcement

... Multi-language, multi-component examples based on simple implementations of fundamental graph search algorithms. Breadth-First Depth-First Traversal starts at the root, then explores all neighboring nodes before exploring (unvisited) neighbors of nearest nodes, and so on Output: List of reachable vertices, each with depth (or hop count) O(|V| + |E|) time and space Traversal starts at the root, then recursively expands each child node Output: Vertex ordering (or precedence) O(|V| + |E|) time and space 2 6 8 3 1 7 5 4 10 11 9 2 5 9 4 1 7 3 8 10 11 6 The software was initially implemented in Babel (all in C?), then refactored as CCA components. Note: Vertex ordering of DFS are not necessarily unique.

GraphSource, InitGraphPort, GraphSrchPort .. Main components of the CCA application are: Driver, InitGraph, BreadthFirstSrchJ, and DepthFirstSrch. Component Lang Uses Port(s) Provides Port(s) Driver C++ GraphSource, InitGraphPort, GraphSrchPort GoPort GraphCFactory --- GraphSource GraphC1Factory* GraphC2Factory** InitGraph Java DataSource InitGraphPort BreadthFirstSrchJ GraphSrchPort DepthFirstSrch ListCFactory ListSource IntDataFactory For contracts in Graph classes, refer to $HOME/components/irinas/Abramova2009/graph-cca: SetOps -- ./ports/sidl/ex2.SetOps.sidl (which apparently provided contracts for AdjListC and SetC) GraphC -- ./components/sidl/ex2.GraphC.sidl GraphOps -- ./ports/sidl/ex2.GraphOps.sidl (which apparently provides contracts for GraphC1 & GraphC2) ListOps -- ./ports/sidl/ex2.ListOps.sidl Note: Routines in flawed graphs are implemented to succeed about 70% of the time. The graph implementation has the following interfaces: Interface: Data, ListNode (list element), ListOps (list), SetOps (set), AdjList (adj list), GraphOps (graph). Classes: IntData (integer wrapper), ListNodeC (list element), ListC (list), SetC (set), AdjListC (adj list), GraphC (graph), GraphC1 (flawed graph), GraphC2 (flawed graph). Contract checking defined for four graph classes, all implemented in C. *Creates Graph with flawed insEdge(). **Creates Graph with flawed insVertex().

The CCA GUI was used to test each algorithm. Java C++ Java Max depth (or hop count) for a 258 Vertex (774 edge) graph was 14.

Summary and Future Work Executable SIDL contracts are an aid to improving the quality of multi-language, component-based applications Explicit obligations on callers and callees Enforcement decoupled from implementation Consistent contract checking across languages Recent work focused on Hardening, extending, and demonstrating enforcement for all supported languages Preliminary work demonstrating multi-component enforcement Current Priorities*: Complete F90 client  C, C++, F77, F90, Java, and Python impls Expand multi-component, CCA enforcement examples and conduct study Update the Babel User’s Guide <heading> *Not necessarily in this order

Toward Multi-Language, Multi-Component Interface Contract Enforcement Thank you for your attention. Any Questions?

For more information related to this work, refer to the following web sites. Components Project/Babel http://www.llnl.gov/casc/components Note: Experiments were conducted using experimental prototypes of the Babel toolkit. However, the latest version of Babel supports global enforcement and enforcement tracing. Common Component Architecture Forum http://cca-forum.org Center for Technology for Advanced Scientific Component Software SciDAC’s Plug and Play Supercomputing http://www.scidac.gov/compsci/TASCS.html

Toward Multi-Language, Multi-Component Interface Contract Enforcement Supplemental Material

Results reported earlier were based on thirteen trial sets formed from five programs and up to five input sets. Component Program Description Simplicial Mesh A Retrieve all faces from the mesh then, for each face, retrieve the adjacent vertices. AA Retrieve all faces from the mesh in sets based on size of input array. Sizes 1, 14587 (10%), and 145870 (100%). MA Retrieve faces as in program A plus, for each set of faces, retrieve their corresponding adjacent vertices. Input sizes were the same as those used for program A. Volume Mesh MT Exercise and check consistency of five mesh interfaces: core, single entity query and traversal, entity array query and traversal, single entity mesh modification, and entity array mesh modification. Vector Utilities VT Exercise all supported functions to include successful execution; one or more precondition violations; and one or more postcondition violations. Sizes 6 (original), 10, 100, 1000, and 10000. <heading> In this case, the three programs from the first two studies are retained, but input array sizes are selected to induce the known violation. Two additional specifications and implementations are included: Volume Mesh – which is GRUMMP 0.2.2b – and vector utilities from the Babel regression tests. A single input file was chosen for the mesh “compliance” testing program (MT). Since the vector utilities program was driven by input arrays, 6 input array sizes were chosen of increasing size to vary processing. Volume mesh implementation is GRUMMP 0.2.2b; vector utilities from Babel regression tests.

Of sampling-based policies, performance-driven enforcement traces detected significantly more violations during trials. <heading> * * Data Source: Dahlgren, CBSE ’07. *Based on execution time per entity or element (when appropriate)