Formal Testing with Input-Output Transition Systems Ed Brinksma Course 2004

© Ed Brinksma/Jan Tretmans Formal Testing exec : TESTS  IMPS   (OBS) der : SPECS   (TESTS) T s  TESTS s  SPECS IUT  IMPS imp i IUT  MODS obs : TESTS  MODS   (OBS) t :  (OBS)  {fail,pass} OBS pass fail Proof soundness and exhaustivess:  i  MODS. (  t  der(s). t (obs(t,i)) = pass )  i imp s Test hypothesis :  IUT  IMPS.  i IUT  MODS.  t  TESTS. exec(t, IUT ) = obs(t,i IUT )

© Ed Brinksma/Jan Tretmans Input-Output Transition Systems dub coffee kwart tea S1 S2 S3 S4 S0 L I = { ?dub, ?kwart } L U = { !coffee, !tea } dub, kwartcoffee, tea from user to machinefrom machine to user initiative with userinitiative with machine machine cannot refuseuser cannot refuse inputoutput L I L U L I  L U =  L I  L U = L ! ? ? !

© Ed Brinksma/Jan Tretmans Input-Output Transition Systems L I = { ?dub, ?kwart } L U = { !coffee, !tea } Input-Output Transition Systems IOTS (L I,,L U )  LTS (L I,  L U ) IOTS is LTS with Input-Output and always enabled inputs: for all states s, for all inputs ?a  L I : ?dub ?kwart ?dub ! coffee ?kwart !tea S ?a

© Ed Brinksma/Jan Tretmans ?kwart ?dub ?kwart ?kwart ?dub ?kwart ?kwart ?dub ?dub ?kwart ?kwart ?dub ?kwart ?kwart ?dub ?dub ?kwart Input-Output Transition Systems ! coffee ?dub !tea ! coffee ?dub ! coffee ?dub !tea

© Ed Brinksma/Jan Tretmans Labelled Transition System Testing lSPECS  LTS ( L I  L U )  LTS lMODS  IOTS (L I, L U )  LTS lTESTS  TTS ( L U, L I )  LTS lOBS  traces lobs  t || i lder  der : LTS   ( LTS ) lWhich imp ? n(strong, weak, branching,... ) bisimulation ntrace-, testing-, refusal - preorder / equivalence nconf, conf*, aconf, nioconf, ioco, mioco F ioco

© Ed Brinksma/Jan Tretmans Formal Correctness testing equivalences refusal testing canonical tester Input Output Automata quiescence ioco

© Ed Brinksma/Jan Tretmans Preorders on Transition Systems implementation i specification s environment e   ? ? ?  i  s   e  E. obs ( e, i )  obs (e, s ) i  LTS s  LTS

© Ed Brinksma/Jan Tretmans Preorders on Input-Output Transition Systems implementation i specification s environment e imp i  IOTS(L I,L U ) s  LTS(L I  L U ) imp  IOTS (L I,L U ) x LTS (L I  L U ) Observing IOTS where system inputs interact with environment outputs, and v.v.

© Ed Brinksma/Jan Tretmans Preorders on Input-Output Transition System implementation i specification s environment e  IOTS(L U,L I ) imp i imp s   e  E. obs ( e, i )  obs (e, s ) i  IOTS(L I,L U ) s  LTS(L I  L U )

© Ed Brinksma/Jan Tretmans Input-Output Testing Relation implementation i specification s environment e obs ( e, p ) = ( traces (e||i ), Ctraces (e||i ) )  iot i  iot s   e  IOTS(L U,L I ). obs ( e, i )  obs (e, s ) i  IOTS(L I,L U ) s  LTS(L I  L U )

© Ed Brinksma/Jan Tretmans Input-Output Refusal Relation implementation i specification s environment e obs ( e, p ) = ( traces (e||i ), Ctraces (e||p) )  ior i  ior s   e  IOTS(L U,L I  {  } ). obs ( e, i )  obs (e, s ) i  IOTS(L I,L U ) s  LTS(L I  L U )

© Ed Brinksma/Jan Tretmans Input-Output Testing Relation i,s  LTS : i  te s   e  LTS. obs ( e, i )  obs (e, s ) inputs can never be refused by i outputs can never be refused by e :  FP ( i )  FP ( s ) FP ( p ) = {  , A  | A  L,   traces(p), p afer  refuses A } i  IOTS(L I,L U ) : i  iot s   e  IOTS(L U,L I ). obs ( e, i )  obs (e, s ) i afer  refuses A  A =  or A = L U

© Ed Brinksma/Jan Tretmans Input-Output Testing Relation  FP ( i )  FP ( s ) i  IOTS(L I,L U ) : i  iot s   e  IOTS(L U,L I ). obs ( e, i )  obs (e, s )  {  |   traces(i), i afer  refuses  }  {  |   traces(s), s afer  refuses  } and{  |   traces(i), i afer  refuses L U }  {  |   traces(s), s afer  refuses L U }  traces(i)  traces(s) and Q traces(i)  Qtraces(s) Q traces : Quiescent traces = traces ending in quiescence  i i= i i =  !x  L U  {  } : i !x  LULU

© Ed Brinksma/Jan Tretmans Input-Output Refusal Relation  Ftraces( i )  Ftraces ( s ) i  IOTS(L I,L U ) : i  ior s   e  IOTS(L U,L I  {  } ). obs ( e, i )  obs (e, s )   ( L  ( L ) )* : i  Failure trace  : Failure traces of i : Ftraces ( i ) = {   ( L  ( L ) )* | i }  Failure A :     A  {  } : i A i  where: inputs can never be refused by i outputs can never be refused by e : i afer  refuses A  A =  or A = L U

© Ed Brinksma/Jan Tretmans Input-Output Refusal Relation  Ftraces( i )  Ftraces ( s ) i  IOTS(L I,L U ) : i  ior s   e  IOTS(L U,L I  {  } ). obs ( e, i )  obs (e, s ) Straces : Suspension traces = Failure traces restricted to refusals quiescence L U =   Straces( i )  Straces ( s ) Straces ( i )= Ftraces ( i )   ( L  { L U } )* = {   ( L  {  } )* | i } 

© Ed Brinksma/Jan Tretmans Input-Output Refusal Relation i  IOTS(L I,L U ) : i  ior s   e  IOTS(L U,L I  {  } ). obs ( e, i )  obs (e, s )  Straces( i )  Straces ( s )    ( L  {  } )*: out ( i after  )  out ( s after  ) where: out ( I ) = { !x  L U | i !x, i  I }  {  | i  i, i  S }  !x out ( i after  ) = { !x  L U  {  } | i }

© Ed Brinksma/Jan Tretmans Implementation Relation ioco i  IOTS(L I,L U ) : i  ior s    ( L  {  } )*: out ( i after  )  out ( s after  ) To allow under-specification : i ioco s    Straces( s ) : out ( i after  )  out ( s after  )

© Ed Brinksma/Jan Tretmans i ioco s = def   Straces (s) : out (i after  )  out (s after  ) Implementation Relation ioco Correctness expressed by implementation relation ioco: Intuition: i ioco-conforms to s, iff if i produces output x after trace , then s can produce x after  if i cannot produce any output after trace , then s cannot produce any output after  ( quiescence  )

© Ed Brinksma/Jan Tretmans i ioco s = def   Straces (s) : out (i after  )  out (s after  ) Implementation Relation ioco p p= p p =  !x  L U  {  } : p !x  LULU Straces (s)= Ftraces (s)   ( L  { L U } )* = {   ( L  {  } )* | s }  p after  = { p’ | p p’ }  out ( P )= { !x  L U | p, p  P }  {  | p p, p  P } !x 

© Ed Brinksma/Jan Tretmans   Implementation Relation ioco out ( i after  ) = out ( i after ?dub ) = out ( i after ?dub.?dub ) = out ( i after ?dub.!coffee) = out ( i after ?kwart ) = out ( i after !coffee ) = out ( i after ?dub.!tea ) = out ( i after  ) = ! coffee ?dub ?kwart ?dub ?kwart i { }{ } { !coffee } { }{ } {  }{  }   { }{ } i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans !coffee ?dub i !coffee ?dub s out (i after  )= {  } out (i after ?dub) = { !coffee } out (i after ?dub.!coffee)= {  } out (s after  ) = {   } out (s after ?dub) = { !coffee } out (s after ?dub.!coffee) = {   } ioco Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans !coffee ?dub i !coffee ?dub s !tea out (i after ?dub) = { !coffee }out (s after ?dub) = { !coffee, !tea } ioco Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans !coffee ?dub s !coffee ?dub i !tea ?dub out (i after ?dub) = { !coffee, !tea }out (s after ?dub) = { !coffee}  ioco Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans out (i after ?dub) = { !coffee, !tea }out (s after ?dub) = { !coffee, !tea} ioco ?dub !coffee ?dub i !tea ?dub Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  ) !coffee ?dub s !tea

© Ed Brinksma/Jan Tretmans ioco ?dub ?kwart !coffee ?kwart i !tea !coffee ?dub s out (i after ?dub) = { !coffee } out (i after ?kwart) = { !tea } out (s after ?dub) = { !coffee } out (s after ?kwart) =  But ?kwart  Straces ( s ) Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  ) ?dub ?kwart !coffee ?kwart i !tea ioco s ?dub !coffee ?kwart !tea out (i after ?dub) = { !coffee } out (i after ?kwart) = { !tea } out (s after ?dub) = { !coffee } out (s after ?kwart) = { !tea }

© Ed Brinksma/Jan Tretmans !coffee ?dub i ?kwart ?dub ?kwart ?dub ?kwart out (s after ?kwart) = { !tea } out (i after ?kwart) = {  } ioco Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  ) s ?dub !coffee ?kwart !tea

© Ed Brinksma/Jan Tretmans out (s after ?dub) = { !coffee } out (i after ?dub) = { , !coffee } ioco ?dub !coffee ?dub i !coffee ?dub s Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  )

© Ed Brinksma/Jan Tretmans out (s after ?dub) = { , !coffee } out (i after ?dub) = { , !coffee } ioco !coffee ?dub  s Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  ) ?dub !coffee ?dub i

© Ed Brinksma/Jan Tretmans out (i after ?dub.?dub) = out (s after ?dub.?dub) = { !tea, !coffee } i ioco s Implementation Relation ioco i ioco s = def   Straces (s) : out (i after  )  out (s after  ) i ?dub !tea ?dub !coffee ?dub s !coffee ?dub !tea ?dub !tea s ioco i out (i after ?dub.   ?dub) = { !coffee }  out (s after ?dub. .?dub) = { !tea, !coffee }

© Ed Brinksma/Jan Tretmans ?kwart ! coffee ?dub !tea ! coffee ?dub ?kwart ?dub ?kwart ! coffee ?dub !tea ioco Implementation Relation ioco ioco

© Ed Brinksma/Jan Tretmans i ioco s = def   Straces (s) : out (i after  )  out (s after  ) ? x (x >= 0) !  x ? x (x < 0) ? y implementation i ! -  x ? x (x >= 0) !  x ? x (x < 0) ? y specification s i ioco s s ioco i Implementation Relation ioco equation solver for y 2 =x :

© Ed Brinksma/Jan Tretmans Genealogy of ioco Labelled Transition Systems IOTS (IOA, IOSM, IOLTS) Testing Equivalences (Preorders) Refusal Equivalence (Preorder) Canonical Tester conf Quiescent Trace Preorder Repetitive Quiescent Trace Preorder (Suspension Preorder) ioco ioconf

© Ed Brinksma/Jan Tretmans Formal Testing with Transition Systems t :  (traces)  {fail,pass} exec : TESTS  IMPS   (OBS) traces der : LTS   (TTS) T s  TTS s  LTS IUT  IMPS ioco i IUT  IOTS pass fail obs : TTS  IOTS   (traces) Soundness and exhaustivess proved:  i  IOTS. (  t  der(s). t (obs(t,i)) = pass )  i ioco s Test hypothesis :  IUT  IMPS.  i IUT  IOTS.  t  TTS. exec(t, IUT ) = obs(t,i IUT )