1. TerascaleSimulation Tools and Technologies Compliance Testing a TSTTM Implementation Carl Ollivier-Gooch University of British Columbia

2. 2 Overview Why bother? –Provide guarantee that implementations work as advertised –Confirm that a service will work with a partial implementation –Supplement documentation with unambiguous software interpretation of interface Test suite covers all functions In many cases, relies on consistency between different ways to retrieve same data

3. 3 Before You Build the TSTTM Compliance Suite Preparing your implementation –Currently, tests a full interface at a time (Mesh, Entity, Modify, Set, Tag, etc...) –Set up TSTTB-Defs.inc and TSTTM- Defs.inc Install Babel runtime (≥ 0.10.10) Download and unpack the tarball –Wherever we're pointing to for these... Have a mesh file or two handy for testing

4. 4 Building TSTTM_unitTest Run configure –Give paths to TSTT?-Defs.inc using --with-TSTTB and --with-TSTTM –If your Babel runtime isn't in /usr/local/babel, use --with-babel to specify Run make Set your LD_LIBRARY_PATH environment variable to include TSTTB, TSTTM, and Babel libraries

5. 5 Running TSTTM_unitTest The Simplest Case TSTTM_unitTest mesh_file_name Tests every interface the implementation has Outputs fairly verbose messages –Send to file with --output=file_name –Suppress with --silent Finishes with summary of tests passed and failed

6. 6 What to Do When Things Go Wrong Scan the output file for failures –All refer to a specific line in the test code –Many also explain what's being tested accurately enough to be helpful Determine which is the offending function in the implementation Fix it! And/or request clarification about what the function should do from TSTT team

7. 7 More Thorough Testing TSTT functions should throw errors for nonsense input TSTTM_unitTest checks for many (not all) error conditions: TSTTM_unitTest --error... Iterate on fixing implementation

8. 8 Testing for a Particular Service Most TSTTM-based services don't use all functions (example: swapping uses approximately 10 TSTTM functions) Ideally, should test only those functions you actually need Planning for this is underway, but not yet implemented Challenge: some functions are tested using other functions  prerequisite issues