Performance Model Checking Scenario-Aware Dataflow Bart Theelen, Marc Geilen, Jeroen Voeten

2 Overview Dataflow Formalisms Timed Probabilistic Systems Performance Model Checking Experimental Results Conclusions & Outlook

Dataflow Formalisms Example digital signal processing areas 3 Streaming Multi-MediaLoop-Control in Mechatronics Dataflow formalisms describe task graphs where potential parallelism is made explicit MP3 Decoder

Dataflow Formalisms: Expressivity vs Analyzability 4 Stuijk, et al. Scenario-Aware Dataflow: Modeling, Analysis and Implementation of Dynamic Applications. SAMOS’11 Synchronous Dataflow (Weighted Marked Graphs) Kahn Process Networks Scenario-Aware Dataflow

Scenario-Aware Dataflow(SADF) Scenario = operation modes with similar resource usage Detectors control processes by sending scenario-valued tokens Detectors contain automata to capture scenario occurrences Real-life: data-dependent control behaviour (normal state machine) Modelling worst/best-case only: non-deterministic state machine Modelling worst/best-case & average-case: Markov chain 5 kernel data channel detector RateIP0P0 PxPx a001 b00x c991x d101 e 0x x = {30, 40, 50,60, 70, 80, 99} MPEG-4 Decoder control channel rate tokens

Processes run in parallel according to ‘extended actor semantics’ 1.Determine scenario depending on 1.Kernels & Detectors: scenario-valued control tokens 2.Detectors: next state of Markov chain 2.Wait until sufficient tokens available 3.Perform the actual task (sample from discrete time distribution) 4.Produce and consume tokens Scenario-Aware Dataflow(SADF) 6 RateIP0P0 PxPx a001 b00x c991x d101 e 0x x = {30, 40, 50,60, 70, 80, 99} MPEG-4 Decoder

Timed Probabilistic Systems(TPS) Compositional semantic model with guarded (interactive) action transitions probabilistic transitions deterministic time transitions Alternates action/time transitions with probabilistic fan-out Pattern for generic discrete execution time distributions time advances exactly t i time units with probability p i for i=1,…,n 7 a p t t1t1 τ p1p1 pnpn 1 1 tntn

Illustrative Example 8 Rateς1ς1 ς2ς2 a20 b10 ProcessScenario Execution Time Probability Aς1ς1 198/11 511/11 572/11 B ς1ς1 55/ /19 472/19 ς2ς2 131/ /15 631/15 D ς1ς1 34/14 131/14 299/14 ς2ς2 73/4 191/4 TPS for Kernel ATPS for Detector D

Semantic Properties Model checking based on (relevant) state-space Exploit semantic properties to limit state-space explosion SADF satisfies various semantic properties Time additivity, action persistency, action urgency, action determinacy Only non-determinism between actions as a result of concurrency Policy for resolving non-determinism does not effect net behaviour 9 policy for resolving non- determinism may however effect performance result

Performance Model Checking Direct computation of quantitative results based on model checking techniques Broad variety of performance metrics Mostly complex reward-based properties Mostly time-related properties 10 MetricType Probabilities Relevant Scope Max Buffer OccupancyWorst CaseNoAll States Min/Max Response DelayBest/Worst CaseNoTransient Min/Max Inter-Firing DelayBest/Worst CaseNoAll States Response Deadline Miss ProbabilityProbabilistic ReachabilityYesTransient Expected Response DelayExpected ReachabilityYesTransient ThroughputEvent RateYesSteady State Periodic Deadline Miss Probability Sample Average / Expected Reachability YesSteady State Average Inter-Firing LatencySample AverageYesSteady State Variance in Inter-Firing LatencySample VarianceYesSteady State Average Buffer OccupancyTime-Weighted AverageYesSteady State Variance in Buffer OccupancyTime-Weighted VarianceYesSteady State Policy for resolving non- determinism only affects Max Buffer Occupancy

Model Checking Strategy - Theory 11 TPS per SADF Process Discrete Markov Chain Move Transition Labels into States |S| Deterministic TPS of Complete SADF Model Resolve Non-Determinism | S’ | TPS of Complete SADF Model Parallel Composition |S||S| p2p2 S2S2 a p1p1 S1S1 S3S3 S 1, - S2, aS2, a S 3, a p1p1 p2p2 Information on occurrence of actions and time available through reward functions on states only ≤| S | >| S’ | Reduced Discrete Markov Reward Model Remove Irrelevant States |S c | <<|S| Performance Number Compute Equilibrium

Throughput MPEG-4 throughput= average number of frames per second = average number of RC firings per time unit Let {X i | i ≥ 1} denote Markov chain with state space S Define reward c to identify firing completion action of RC If c(s) = 1 for state s, it is relevant, otherwise it is irrelevant 12 Δ is temporal reward function denoting amount of time elapsed since previous RC firing

Metric-Dependent State-Space Reduction If relevant positive recurrent state for ergodic Markov chain exists, then Reduction yields ergodic Markov chain Reduction preserves performance properties 13

Transformed TPS per SADF Process Model Checking Strategy - Practice 14 TPS per SADF Process Discrete Markov Chain Move Transition Labels into States |S| Deterministic TPS of Complete SADF Model Resolve Non-Determinism | S’ | TPS of Complete SADF Model Parallel Composition |S||S| Reduced Discrete Markov Reward Model Remove Irrelevant States |S c | Performance Number Compute Equilibrium Transformed TPS of Complete SADF Model | S’’ | Discrete Markov Chain |S| Parallel composition with on-the-fly reduction and resolving non-determinism

Experimental Results -requires more than 1.5GB of memory ¤takes more than 6 hours Reduction after resolving non-determinism 15 Case Study| S’’ ||S| MPEG-4 AVC Channel Equalizer MPEG-4 SP (PD = 1) MPEG-4 SP (PD = 2) MPEG-4 SP (PD = 3) -- MP3 (PD = 1) -- MP3 (PD = 2) -- MP3 (PD = 3) -- MP3 (4 ≤ PD ≤ 9) -- |S c |Factor ¤- Time [s]Memory [MB] ≤ > 6h¤

Statistical Model Checking as Alternative Statistical model checking supported by modelling SADF in POOSL POOSL is much more expressive than SADF but also has TPS semantics Compositional estimation of confidence intervals for long-run averages 16 Case StudyThroughputTime [s] MPEG-4 AVC ≤ Channel Equalizer · MPEG-4 SP (PD = 1) · MPEG-4 SP (PD = 2) · MPEG-4 SP (PD = 3) · MP3 (PD = 1) · MP3 (PD = 2) · MP3 (PD = 3) · MP3 (PD = 9) ¤> 6h 95% Confidence IntervalTime [s] [ , ]0.14 [ ·10 -3, ·10 -3 ]0.75 [ ·10 -3, ·10 -3 ]7.4 [ ·10 -3, ·10 -3 ]6.86 [ ·10 -3, ·10 -3 ]6.85 [ ·10 -7, ·10 -7 ]32.8 [ ·10 -7, ·10 -7 ]31.5 [ ·10 -7, ·10 -7 ]32.1 [ ·10 -7, ·10 -7 ]30.6

Conclusions & Outlook Performance model checking approach for SADF Compositional TPS semantics with discrete time distributions Exploit semantic properties Removal of metric-dependent irrelevant states On-the-fly construction of relevant state-space Broad variety of pre-defined performance metrics All expressible as temporal reward formula Statistical model-checking for long-run averages as alternative Increase flexibility to allow computing user-defined metrics Lift Markov chain reduction to bisimulation reduction on TPS Support temporal rewards as property specification language Could contemporary quantitative model checkers supporting Probabilistic Timed Automata be a suitable alternative? 17