A Test Automation Language for Behavioral Models Nan Li, Medidata Solutions

A Test Automation Language for Behavioral Models Nan Li Medidata Solutions This work is part of Nan’s PhD dissertation at George Mason University and a paper is submitted to ICST 2015

Model-based Testing What is model-based testing Why to use model-based testing ICST 2014: model-based testing is the future Usually applied to complicated software Software for planes, trains, cars, etc. ICST 2014 keynote speech from Tom Wissink (Lockheed Martin) Model-based testing is the future Drawback: complicated to use https://sites.google.com/site/icst2014/program/keynotes/2014%20ICST%20De javu%20IandT.pptx?attredirects=0&d=1

Model-based Testing Model Test Requirements (TRs) Abstract Tests Concrete Tests Test Execution and Reports Criteria Extra Info

An Example - Vending Machine Coverage Criteria TRs e.g. [1, 3] Generate abstract tests test paths (TPs) e.g. [1, 3, 4] how to generate TPs to satisfy TRs with minimum costs? Generate concrete tests solve constraints Coin 1 Coin 2 GetChoc AddChoc AddChoc Coin 3 4 GetChoc AddChoc AddChoc / Coin State name: state invariants State1: credit = 0 & #stock = 0 State2: credit > 0 & #stock = 0 State3: credit = 0 & #stock > 0 State4: credit > 0 & #stock > 0 composite states, joins, etc. node, edge, edge-pair, prime path state invariants, preconditions and postconditions

The Mapping Problem Transform abstract tests to concrete tests [1, 3, 4, 1, 2, 4] [AddChoc, Coin, GetChoc, Coin, AddChoc] Coin 1 Coin 2 GetChoc AddChoc AddChoc Coin 3 4 Existing approaches use too many additional models GetChoc AddChoc AddChoc / Coin State name: state invariants State1: credit = 0 & #stock = 0 State2: credit > 0 & #stock = 0 State3: credit = 0 & #stock > 0 State4: credit > 0 & #stock > 0 When requirements change, models and code have to change Testers often convert abstract tests to concrete tests by hand AddChoc, Coin, and GetChoc appear multiple times

Solution The structured test automation language (TAL) Use only behavioral models (state machine diagrams) Identifiable elements Transitions and constraints Mappings From elements to executable code Element and object mappings

Element Mapping Mapping mappingName TYPEOFELEMENT nameOfElement Requires objectMappingName … [TYPEOFCONSTRAINT nameOfElement …] … { test code } TYPEOFELEMENT: transition or constraint TYPEOFCONSTRAINT: guards, post-conditions, or state-invariants

Object Mapping Mapping mappingName Class nameOfClass Object nameOfObject Requires objectMappingName … { test code }

Mapping Examples Mapping coinOver90 Transition coin { VendingMachine vm = new VendingMachine(); vm.coin (100);} Mapping vmInit Class VendingMachine Object vm { VendingMachine vm = new VendingMachine (); } Mapping coinOver90 Transition coin Requires vmInit { vm.coin (100);}

Mapping Examples (Cont.) Mapping constraintCredit Constraint ConstriantCreditOver0 StateInvariants State2, State4 { vm.getCredit () > 0;}

Concrete Test Generation … State3 (credit = 0 & #stock > 0), Coin, State4 (credit > 0 & #stock > 0), ... To satisfy a constraint, select a mapping for the preceding transition Report to users if all mappings cannot satisfy the constraint Inability to satisfy constraints may suggest that models or programs have errors

Summary and Future Applications at Medidata Automate the mapping problem Original Requirements Use TAL Requirements in TAL Original Data Automatic Validation Transformed Data Inspiration: To automate the test generation and validation, we need to change the requirement formats and use automation languages to connect the requirements to test code At Medidata, we plan to create another TAL to translate requirements for validating data transformation

Thank you Nan Li nli@mdsol.com Medidata Solutions http://cs.gmu.edu/~nli1/stale/ http://www.cs.gmu.edu/~offutt/documents/theses/NanLiDissertation.pdf