TESTCOM/FATES Test Plan Generation for Concurrent Real-Time Systems based on Zone Coverage Analysis Farn Wang Dept. of Electrical Eng. National Taiwan University joint work w. G.-D. Huang

TESTCOM/FATES Background Testing is still the main verification technique. CRTS (Concurrent Real-Time Systems) imposes challenges to test coverage. –concurrency  event ordering –event time instances –appropriate coverage mertics for CRTS ? states ? regions ? zones ?

TESTCOM/FATES Proposal CTA (communicating timed automata) for the model of CRTS AZC (Active Zone Coverage) –Estimate how big a zone is. –Only active variables are used in determining zones Testplan for efficient AZC gains Experiments

TESTCOM/FATES Related work (1/2) Classic: –statement coverage, visited-state, transition coverage TA (timed automata) reachability –arc coverage, trigger coverage –region coverage, estimation technique of regions in a zone. UPPAAL –time-optimal testplans –edge coverage, location coverage, dataflow coverage

TESTCOM/FATES Related work (2/2) event-recording automata  reachability graph of equivalence classes  test plan –arc coverage, location coverage, domain analysis, dataflow coverage uniformly priced timed automata  test plans with a minimum cost Test plan based on fault-detection rate for regression testing Test case with region coverage annotations for timed automata model

TESTCOM/FATES CTA Communicating Timed Automata wait retry x<=52 ?cd !end x1==808 idle busy collision x<26 ?begin x3<26 x3=0; ?begin x3=0; ?end Sender 1 Sender 2 Bus !begin x1=0; !cd Sender1,Sender2, and Bus are all PTAs. ?cd send x1<=808 !begin x1=0; 4 ?cd 3 5 wait retry x<=52 ?cd !end x2== !begin x2=0; ?cd send x2<=808 !begin x2=0; 11 ?cd 10 12

TESTCOM/FATES CTA LGT (legitimate global transitions) synchronized: for every event e, every !e is received by exactly a ?e. minimal: it cannot be decomposed to more than one legitimate global transitions

TESTCOM/FATES CTA LGT (legitimate global transitions) wait retry x<=52 ?cd !end x1==808 idle busy collision x<26 ?begin x3<26 x3=0; ?begin x3=0; ?end Sender 1 Sender 2 Bus !begin x1=0; !cd ?cd send x1<=808 !begin x1=0; 4 ?cd 3 5 wait retry x<=52 ?cd !end x2== !begin x2=0; ?cd send x2<=808 !begin x2=0; 11 ?cd (1,15) is an LGT.(8,15) is an LGT. (4,13,18) is an LGT. (1,8,15) is not. not sync’d (4) is not. not sync’d (4,18) is not. not sync’d (19,8,15) is not. not minimal 19

TESTCOM/FATES CTA states and transitions state, a valuation from propositions to {true, false} from clocks to non-negative reals +t, a state identical to except that for all clocks x, ( +t)(x) = (x)+t, transitions  g ’ satisfies the trigger of g. ’ is identical to except that clocks are reset according to g.

TESTCOM/FATES Regions & Zones Zones: convex space characterizable with constraints of the form x-y<c, x-y≤c –x,y: clocks or zeros, –c: an integer with |c| ≤ the biggest timing constant used in the model and property. Regions: the smallest non-empty zones Region-equivalence is fine enough for TCTL model-checking. Zone-equivalence can also be fine enough and efficient for the same purpose.

TESTCOM/FATES Symbolic postcondition  ’ = post( ,g): the zone for the timed postcondition  ’ of zone  after LGT g. post( ,g)={ | ’ , t  R, t≥0, ’+t  g }

TESTCOM/FATES Zone Forest construct the reachability graph ZoneForest(A,  ) { Rewrite the initial condition of A in DN  1  …   n R:={  1, …,  n }; V:=R; Φ:=R; K:=  ;  :=  1  …   n ; While  , { Pick a zone  from  ; let  :=  – {  }; For each LGT g { Let  ’ := post( , g); If  ’   is satisfiable, {  :=    ’;  :=   {  ’}; V:=V  {  ’ }; K:=K  {( ,g,  ’)}. } } } return (V,R,K); } The reachability predicate already- reachables are not explored.

TESTCOM/FATES Active zones (1/2) A variable x is active in a state iff there is a run from along which x is read before ever written to.  (Write(x)  Read(x)) U Read(x) Write(x), the disjunction of triggers of all LGT g which writes to x. Read(x), –the disjunction of triggers of all LGT g which reads x in either triggers or the RHS of actions –the invariance condiitons –the risk conditions

TESTCOM/FATES Active zones (2/2) ActiveZone (  ) { for every clock x, if    (Write(x)  Read(x)) U Read(x) is unsatisfiable,  :=  x(  ); return  ; } Reduce the representation sizes of reachable spaces! Improve performance in ZoneForest construction! Gain more regions in coverage!

TESTCOM/FATES Prioritized Test Plan Generation TestPlan(A,  ) { Let  be the set of root-to-leaf paths in ZoneForest(A,  ); Let  be an empty sequence;  := false. While    { Pick a  1 …  n   s.t.   ’ 1 …  ’ m   RCM(ActiveZone(  1  …   n   )) ≥ RCM(ActiveZone(  ’ 1  …   ’ m   ))  :=  - {  1 …  n };  :=  testcase(  1 …  n );  :=    1  …   n ; } return  } The region coverage estimation technique in FORTE 2003, by Wang, Hwang, Yu. Sequence concatenation.

TESTCOM/FATES Experiments RED, version 7.0 –model-checker for TCTL and TAs –simulation-checker for TAs –parametric analyzer for LHAs –In sourceforge project REDLIB. BLUETOOTH, L2CAP Linux, REDHAT 9, 3.2GHz We did not compare with other tools. previous state-based coverage for timed systems ? could have to concoct some SUTs that show our performance.

TESTCOM/FATES AZC estimation for zone forest construction

TESTCOM/FATES AZC gain for prioritized test plan

TESTCOM/FATES Future work The strength of state-based coverage for timed systems ? New state-based coverage metrics for timed systems Cost of testcases ?