A Structured Solution Approach for Markov Regenerative Processes Elvio G. Amparore 1, Peter Buchholz 2, Susanna Donatelli 1 1 Dipartimento di Informatica, Università degli Studi di Torino, Italy 2 Department of Computer Science, TU Dortmund, Germany UNIVERSITÀ DEGLI STUDI DI TORINO

Research Objective Efficient representation and solution of MRP models (as whose underlying a DSPN) Line of Research Combine: advances in structured (Kronecker) solution of (G)SPN and alike with recent advances in “matrix-free” solution techniques of MRP

Context MRP ( Markov Regenerative Process ) with CTMCs as subordinated processes (at most one general event enabled at a time) MRP generated from GSPN ( Generalized Stochastic Petri Nets ). Kronecker representation of the state space.

Straightforward approach CTMC representation requires Q MRP representation requires: – : rates of exponential transitions that do not affect the enabling of general transitions; – : rates of exponential transitions whose firing disables a general transitions; – : effect of firing a general transitions.

Straightforward approach GSPN = {GSPN (j), }DSPN = {DSPN (j), } Does not work!

Why? Firing of t contributes to Q. Enabling of g depends only on the local state.

Why? Firing of t contributes to. Enabling of g is affected by t. Firing of t contributes to Q, since g is not enabled.

More difficulties… Assume we can express in Kronecker form… Steady-state MRP solution requires the computation of the embedded DTMC P, which: is typically dense; might not be easily expressible as a Kronecker expression.

Talk Outline 1.A successful structuring of the state space and of the generators 2.Matrix-free solution of the “structured” MRP 3.Results 4.Future work

Talk Outline 1.A successful structuring of the state space and of the generators 2.Matrix-free solution of the “structured” MRP 3.Results 4.Future work

State space structuring for GSPN Two components: set of transitions, rate of. Component 1 Component 2

State space structuring for GSPN Each component state space is partitioned into subsets: Component 1 Component 2

State space structuring for GSPN Component 1 Component 2 State space is made by 2 macro-states:

State space structuring for GSPN Component 1 Component 2 For each transition, each component we have a block matrix with the rates from subset k to subset l.

State space structuring for CTMC Basic structuring* is: Two local states s,s’ are in the same subset iff: * [P. Buchholz. Hierarchical structuring of superposed GSPNs. IEEE TSE, 1999.]

State space structuring for MRP With t 14 general, subset is split in two: Component 1 Component 2 State space has 3 macro-states:

State space structuring for MRP Partitioning has to be refined according to enabling (or not) of general transitions: Therefore, the states in each component subset has the same enabling of general transitions.

Structuring of the MRP generators Rates of the exponential transitions that do not affect the enabling of general transitions: and we obtain: Only contributions that do not locally disable g (be sure there is no premption and immediate re-enabling of g )

Structuring of the MRP generators where the diagonal is: with:

Structuring of the MRP generators Rates of the exponential transitions that disable the currently enabled general transition: and we obtain:

Structuring of the MRP generators Transition probability of the general transitions: and we obtain: We have the Kronecker form of the generator matrices of a structured MRP.

Talk Outline 1.A successful structuring of the state space and of the generators 2.Matrix-free solution of the “structured” MRP 3.Results 4.Future work

Problem formulation The matrix of the embedded Markov chain is: Exponential states Firing of general transitions Preemption of general transitions Uniformized SMC. are the k -th α -factors of g. Problem : sum of matrix powers will be dense.

Fill-in of matrix P P is dense due to matrix exponentials. – Space occupation: O ( N 2 ) – Very slow to compute. Solution* : do not construct matrix P. * [R. German, Iterative analysis of Markov regenerative models, 2001]

Matrix-free solution In short: solve a linear equation system without having the system matrix, only its expression. Numerical solution: Power method. Expand the expression for P Expand the expression for P [K. Trivedi, A. Reibman, Transient Analysis of Markov and Markov Reward Models, 1987]

Matrix-free Kronecker Form (1) Matrices in Kronecker form only. Derive a Kronecker form for and. uv w u is simple (with ):

Matrix-free Kronecker Form (2) Terms and are the instantaneous and cumulative Uniformization in Kronecker form: The sequence: has Kronecker form:

Matrix-free Kronecker Form (3) Let: Sequence of iterative y l α-factors so we can write:

Matrix-free Kronecker solution All the terms of the x = xP product are in Kronecker form. Does not require the construction of. Computes one uniformization per xP product. Suitable for Power method and Krylov methods. uv w [E. G. Amparore, S. Donatelli, Revisiting the matrix-free solution of Markov regenerative processes, NSMC11, 2011]

Talk Outline 1.A successful structuring of the state space and of the generators 2.Matrix-free solution of the “structured” MRP 3.Results 4.Future work

Test with two tools Kronecker structure Implements the newly proposed MRP solution. No Kronecker structure Sparse matrix. Matrix-free / explicit solution. nsolve structured Kronecker solution any general transition nsolve structured Kronecker solution any general transition DSPN-tool non-structured MRP deterministic transition DSPN-tool non-structured MRP deterministic transition Parallel tests to check the correctness of the results.

Test 1/3: Flexible Manufacturing System Four parallel machines Maintenance system

FMS: structured vs. non-structured states, 865 macrostates Kronecker (nsolve): BiCG-Stab takes sec. with 74 iterations. Needs ~3.2×10 5 matrix entries. Non-structured (DSPN-tool): BiCG-Stab takes 88.6 sec. with 73 iterations. Needs ~3×10 6 matrix entries.

Test 2/3: Moving Server System Four client stations with m requests. Moving Server

Structured MRP vs. phase-type Compare MRP with general transitions against CTMC with phase-type transitions. MRP with general: – Deterministic – Erlang-3 CTMC with phase-type expansion: – Simple exponential – Erlang-2 – Erlang-3 nsolve

Structured MRP vs. phase-type Compare MRP with general transitions against CTMC with phase-type transitions. MRP with general: – Deterministic – Erlang-3 CTMC with phase-type expansion: – Simple exponential – Erlang-2 – Erlang-3 DSPN-tool

Structured MRP vs. phase-type Kronecker (nsolve): MRP:7.3 M states,1240 nnz. Exp:7.3 M states,1224 nnz. Erlang-2:11 M states, 3484 nnz. Erlang-3:15 M states, 5056 nnz. Non-structured : MRP: 7.3 M states. 35M nnz. State space comparison:

Structured MRP vs. phase-type Kronecker (nsolve): MRP/det:341 seconds. MRP/Erlang-3:369 seconds. Exp:151 seconds. Ph-type/Erlang-2:331 seconds. Ph-type/Erlang-3:613 seconds. Non-structured : 4048 seconds. Steady-state solution performance:

Test 3/3: Multi-threaded program Three parallel threads Sequential part

Large state space Kronecker (nsolve): 12 macrostates, 8969 nnz 3306 second. Non-structured (DSPN-tool): 49M nnz seconds. ~ 7M states

Conclusions General Kronecker structuring of MRP. Efficient matrix-free steady-state solution. Future Works MRP from symmetric nets (SWN) exploiting the symmetric structure ( lumped embedded process ). Model checking with one-clock automata (like CSL TA logic) with Kronecker structure. More efficient computation when subnets are separated by general transitions. Approximate solution based on decomposition. In this case, several general transitions can be enabled simultaneously in separated subnets.