Testing Transition Systems with Input and Output Testers Alexandre Petrenko Nina Yevtushenko Jia Le Huo TestCom’03, May 27 th, 2003
2 Outline Motivation Basic idea Testing frameworks –Queued-quiescence –Queued-suspension Conclusions
3 Conformance Testing SpecImp TestsTest execution Does Imp conform to Spec? Fault model
4 Input/Output Transition Systems Non-deterministic IOTS: model of Spec and Imp Output of IOTS cannot be blocked: basic assumption Spec and Imp are input-enabled coin coffee, milk ?coin !coffee !milk ?coin Spec
5 ?coin Motivating Example ?coin !coffee !milk ?coin !coffee !milk ?coin !coffee SpecImp Quiescent trace coin coin coffee milk yields the ioco test case: fail coin coffee milk, coffee pass milk, fail ?coin Is the test case still sound if output of Spec cannot be blocked?
6 Contradiction The ioco test case blocks coffee after coin Output of IOTS cannot be blocked: basic assumption How can I block a coffee on its way out?
7 Problem Statement A tester not blocking output of Imp must be input-enabled An input-enabled tester usually –has unbounded test runs –makes choice between input and output Both features are not desirable in practice We have to define a tester that –is input-enabled –reaches verdict in finite steps –never chooses between input and output
8 Basic Idea Tester Imp Output queue finite capacity for practical reasons Input test process executes a given input sequence Output test process produces pass when an expected output sequence followed by quiescence is read from the queue and fail otherwise Input-enabled, and no choice between input and output Input queue
9 Queued-Quiescent Traces For a quiescent trace of Spec, I O Qqtraces(Spec) For input sequence , Qqtraces o (Spec, ) includes all the output sequences in response to : { ' O* | ' Qqtraces(Spec)} Qqtraces(Spec) = Qqtraces(Imp) (qq-trace equivalent) ?coin !coffee !milk ?coin Spec ?coin !coffee !milk ?coin !coffee Imp !coffee !milk Qqtraces o (Spec, ) !coffee !milk Qqtraces o (Imp, )
10 Queued-Quiescence Relations For non-oscillating Spec and Imp, if input sequence exists such that : –Qqtraces o (Imp, ) Qqtraces o (Spec, ) = (no pass) then Imp is qq-separable from Spec –Qqtraces o (Imp, ) Qqtraces o (Spec, ) ( fail, pass) then Imp is qq-distinguishable from Spec –Qqtraces o (Imp, ) Qqtraces o (Spec, ) (no fail) then Imp is qq-weakly-distinguishable from Spec qq-separable qq-distinguishable qq-weakly- distinguishable
11 Test Derivation with Fault Model Given Spec and Imp, check input sequences of length 0, 1, 2, … until a sequence is found such that –qq-separates, or –qq-distinguishes, or –qq-weakly-distinguishes Imp from Spec, or –the input buffer capacity k is reached Repeat for all Imp k = ?
12 A Finer Tester After executing the qq-test case with input coin on Spec and Imp !coin ?coin !coffee !milk ?coin !coffee !milk ?coin !coffee SpecImp fail ?milk pass fail ?milk, ?coffee ?milk, ?coffee, The resulting IOTS can be distinguished by a qq-test case: ?coin
13 Queued-Suspension Testing 11 22 11 qq-test case of Spec-after-( 1, 11 ) qs traces are sequences of qq traces executed by the qq testers: ( 1 1i )( 2 2j )...( n nk ) 22 12 qq-test case of Spec-after-( 1, 12 )... 33 21 22 23 24 25 33 33 33 33 qq-test case of Spec
14 Conclusions Outputs cannot be blocked: basic assumption Testing frameworks: qq and qs Comparison with the ioco framework: –information of Interleaving of inputs and ouputs are lost –so the conformance relations are coarser than ioco –but ioco testing is not applicable here Open questions: oscillating Spec, k = ?, etc
15 Merci beaucoup!
16 An Example ?coin !coffee !milk ?coin Spec coin Output queue (capacity 2) Input queue (capacity 2) ?coin !coffee !milk ?coin !milk ?coin !coffee
17 An example (Cntd.) We have coin coffee milk pass fail When the ioco test case below is applied to Spec with queues fail coin coffee milk, coffee pass milk, fail Is this test case still sound?