Testing Generation at UPenn Model-Based Test Generation Temp. Prop. Translator Controller Model Checker Witness generator  1 Æ …… Æ  n ii i ² ii ² i Concretizer TS={  1,……,  n } Specification Model Implementation Tester demand /Specification Test Suite Test Setting = temporal properties + specification + implementation Coverage Criteria

Testing Generation at UPenn Model-Based Test Generation Overview: Use witness-ready model checker to automate the test generation. Flexible: properties can mimic the traditional coverage criteria, or can be directly specified by tester. Efficient: Efficiency inherent from checker’s search engine. Effective: fine-tune test by specifying extra constrains f=G ( (begin ) F end) Æ (setValue ) F move)) Add: f 1 =F (setValue): Test f Æ f 1 Current research Testing discrete systems. Mimicing traditional coverage criteria Is in general LTL property “testable”? Testing hybrid systems. Randomized simulation approach Reachability checker-assisted approach.

Testing Generation at UPenn Testing discrete systems Given: Test setting = LTL/ 9 LTL + the specification+ Blackbox implementation. Problem: Currently testing properties is limited to 9 LTL with eventuality only. Question: is there a test for “F( G( a ! Xb))”. Require to test all the possible executions May require a test with infinite length.

Testing Generation at UPenn Property-Coverage Testing Synthesizing test suites for 9 LTL property. 1. Is the property “E GF a” testable? 1. No finite trace can be attest to this property. 2. If the number of states in blackbox is bounded by n, 1. A trace for 9 LTL + the specification is rational:  (  ) . 2. A infinite trace  (  )  can be cut to  (  ) n 3. Buchi tree automaton-based model checker can be used to generate rational traces. Synthesizing test suites for LTL property. 1. LTL can be translated to a set of interesting 9 LTL properties. 1. E( GF( a) Æ F(G(a ) X b)) is an interesting property for F(G(a ) X b)) 2. Each interesting 9 LTL property focuses on testing a particular portion of LTL formula.

Testing Generation at UPenn Testing Hybrid System: Phase I Randomized test generator=Randomized Simulator+ Coverage Checker. 1. Local ramdomization, gobal strategy. 1. Stay or jump 2. Where to jump 3. How long to stay 2. Gobal ramdomization 1. Aborting/Continuing on current trace. Mode A df/dt=1 a: True:f=0 b: 1 · f<3:m=1 c: 2 · f<4:m=2 Mode B dw/dt=1 Mode C d: 2 · f<5:m=4

Testing Generation at UPenn Testing Hybrid System: Phase I 1. Heuristic search 1. Uncovered neighbor first 2. Syntax-based distance matrix (Shortest distance to uncovered state/location) 3. Open question: Make local decision based global information/history. 1. deciding the weight for outgoing transitions based on the history (What should we learn from a failed search). 2. Deciding the duration to stay in a mode. 2. Current status: a working version of randomized test generation is written on CHARON simulator.

Testing Generation at UPenn Testing Hybrid System: Phase II System Modeling CHARON (Model) Flatten hybrid model Concretizer Implementation Test Suite Set of predicates Coverage criteria Bad set Reachability Checker Yes w/ Trace Simulation /refinment NO w/ more predicates YES No

Testing Generation at UPenn Intelligent simulator Intelligent simulator=simulator+ property checker (monitor) 1. Verification as the byproduct of simulation 1. LTL Property encoded as the monitor 1. MEDL: A subset of LTL, has been applied to Java running-time monitoring. 2. Monitor advances when the simulation proceeds. 3. Open problem: LTL with eventuality only is easy, but how about other formula requires circularity reasoning. 1. Need to remember the states traversed to sense the loop. 1. Difficult because the domain of continuous variables are dense. 2. The search is tailored by the property. 1. A transition “measure” has the priority higher than others if the property is G(measure => X (home)). 2. Most interesting simulation trace: Covering as many parts of property as possible using less steps.