1. Metrics for real time probabilistic processes Radha Jagadeesan, DePaul University Vineet Gupta, Google Inc Prakash Panangaden, McGill University Josee Desharnais, Univ Laval

2. Outline of talk Models for real-time probabilistic processes Approximate reasoning for real-time probabilistic processes

3. Discrete Time Probabilistic processes Labelled Markov Processes For each state s For each label a K(s, a, U) Each state labelled with propositional information 0.5 0.3 0.2

4. Discrete Time Probabilistic processes Markov Decision Processes For each state s For each label a K(s, a, U) Each state labelled with numerical rewards 0.5 0.3 0.2

5. Discrete time probabilistic proceses + nondeterminism : label does not determine probability distribution uniquely.

6. Real-time probabilistic processes Add clocks to Markov processes Each clock runs down at fixed rate r c(t) = c(0) – r t Different clocks can have different rates Generalized SemiMarkov Processes Probabilistic multi-rate timed automata

7. Generalized semi-Markov processes. Each state labelled with propositional Information Each state has a set of clocks associated with it. {c,d} {d,e} {c} s tu

8. Generalized semi-Markov processes. Evolution determined by generalized states Transition enabled when a clock becomes zero {c,d} {d,e} {c} s tu

9. Generalized semi-Markov processes. Transition enabled in 1 time unit Transition enabled in 0.5 time unit {c,d} {d,e} {c} s tu Clock c Clock d

10. Generalized semi-Markov processes. Transition determines: a. Probability distribution on next states b. Probability distribution on clock values for new clocks {c,d} {d,e} {c} s tu Clock c Clock d 0.20.8

11. Generalized semi Markov proceses If distributions are continuous and states are finite: Zeno traces have measure 0 Continuity results. If stochastic processes from converge to the stochastic process at

12. Equational reasoning Establishing equality: Coinduction Distinguishing states: Modal logics Equational and logical views coincide Compositional reasoning: ``bisimulation is a congruence’’

13. Labelled Markov Processes PCTL Bisimulation [Larsen-Skou, Desharnais-Panangaden-Edalat] Markov Decision Processes Bisimulation [Givan-Dean-Grieg] Labelled Concurrent Markov Chains PCTL [ Hansson-Johnsson ] Labelled Concurrent Markov chains (with tau) PCTL Completeness : [ Desharnais- Gupta-Jagadeesan-Panangaden ] Weak bisimulation [Philippou-Lee-Sokolsky, Lynch-Segala]

14. With continuous time Continuous time Markov chains CSL [Aziz-Balarin-Brayton- Sanwal - Singhal-S.Vincentelli] Bisimulation,Lumpability [ Hillston, Baier-Katoen-Hermanns ] Generalized Semi- Markov processes Stochastic hybrid systems CSL Bisimulation:????? Composition:?????

15. Alas!

16. Instability of exact equivalence Vs

17. Problem! Numbers viewed as coming with an error estimate. (eg) Stochastic noise as abstraction Statistical methods for estimating numbers

18. Problem! Numbers viewed as coming with an error estimate. Reasoning in continuous time and continuous space is often via discrete approximations. eg. Monte-Carlo methods to approximate probability distributions by a sample.

19. Idea: Equivalence metrics Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell Replace equality of processes by (pseudo)metric distances between processes Quantitative measurement of the distinction between processes.

20. Criteria on approximate reasoning Soundness Usability Robustness

21. Criteria on metrics for approximate reasoning Soundness Stability of distance under temporal evolution: ``Nearby states stay close '‘ through temporal evolution.

22. ``Usability’’ criteria on metrics Establishing closeness of states: Coinduction. Distinguishing states: Real-valued modal logics. Equational and logical views coincide: Metrics yield same distances as real- valued modal logics.

23. ``Robustness’’ criterion on approximate reasoning The actual numerical values of the metrics should not matter --- ``upto uniformities’’.

24. Uniformities (same) m(x,y) = |x-y| m(x,y) = |2x + sinx -2y – siny|

25. Uniformities (different) m(x,y) = |x-y|

26. Our results

27. For Discrete time models: Labelled Markov processes Labelled Concurrent Markov chains Markov decision processes For continuous time: Generalized semi-Markov processes

28. Results for discrete time models BisimulationMetrics Logic(P)CTL(*)Real-valued modal logic CompositionalityCongruenceNon- expansivity ProofsCoinduction

29. Results for continuous time models BisimulationMetrics LogicCSLReal-valued modal logic Compositionality??? ProofsCoinduction

30. Metrics for discrete time probablistic processes

31. Bisimulation Fix a Markov chain. Define monotone F on equivalence relations:

32. Defining metric: An attempt Define functional F on metrics.

33. Metrics on probability measures Wasserstein-Kantorovich A way to lift distances from states to a distances on distributions of states.

34. Metrics on probability measures

36. Example 1: Metrics on probability measures Unit measure concentrated at x Unit measure concentrated at y xy m(x,y)

37. Example 1: Metrics on probability measures Unit measure concentrated at x Unit measure concentrated at y xy m(x,y)

38. Example 2: Metrics on probability measures

39. THEN:

40. Lattice of (pseudo)metrics

41. Defining metric coinductively Define functional F on metrics Desired metric is maximum fixed point of F

42. Real-valued modal logic

43. Tests:

44. Real-valued modal logic (Boolean) q q

45. Real-valued modal logic

46. Results Modal-logic yields the same distance as the coinductive definition However, not upto uniformities since glbs in lattice of uniformities is not determined by glbs in lattice of pseudometrics.

47. Variant definition that works upto uniformities Fix c<1. Define functional F on metrics Desired metric is maximum fixed point of F

48. Reasoning upto uniformities For all c<1, get same uniformity [see Breugel/Mislove/Ouaknine/Worrell] Variant of earlier real-valued modal logic incorporating discount factor c characterizes the metrics

49. Metrics for real-time probabilistic processes

50. Generalized semi-Markov processes. {c,d} {d,e} {c} s tu Clock c Clock d Evolution determined by generalized states : Set of generalized states

51. Generalized semi-Markov processes. {c,d} {d,e} {c} s tu Clock c Clock d Path: Traces((s,c)): Probability distribution on a set of paths.

52. Accomodating discontinuities: cadlag functions (M,m) a pseudometric space. cadlag if:

53. Countably many jumps, in general

54. Defining metric: An attempt Define functional F on metrics. (c <1) traces((s,c)), traces((t,d)) are distributions on sets of cadlag functions. What is a metric on cadlag functions???

55. Metrics on cadlag functions Not separable! are at distance 1 for unequal x,y xy

56. Skorohod metrics (J2) (M,m) a pseudometric space. f,g cadlag with range M. Graph(f) = { (t,f(t)) | t \in R+}

57. t f g (t,f(t)) Skorohod J2 metric: Hausdorff distance between graphs of f,g f(t) g(t)

58. Skorohod J2 metric (M,m) a pseudometric space. f,g cadlag

59. Examples of convergence to

60. Example of convergence 1/2

61. Example of convergence 1/2

62. Examples of convergence 1/2

63. Examples of convergence 1/2

64. Examples of non-convergence Jumps are detected!

65. Non-convergence

69. Summary of Skorohod J2 A separable metric space on cadlag functions

70. Defining metric coinductively Define functional on 1-bounded pseudometrics (c <1) Desired metric: maximum fixpoint of F a. s, t agree on all propositions b.

71. Real-valued modal logic

73. h: Lipschitz operator on unit interval

74. Real-valued modal logic

75. Base case for path formulas??

76. Base case for path formulas First attempt: Evaluate state formula F on state at time t Problem: Not smooth enough wrt time since paths have discontinuities

77. Base case for path formulas Next attempt: ``Time-smooth’’ evaluation of state formula F at time t on path Upper Lipschitz approximation to evaluated at t

78. Real-valued modal logic

79. Non-convergence

80. Illustrating Non-convergence 1/2

81. Results For each c<1, modal-logic yields the same uniformity as the coinductive definition All c<1 yield the same uniformity. Thus, construction can be carried out in lattice of uniformities.

82. Proof steps Continuity theorems (Whitt) of GSMPs yield separable basis Finite separability arguments yield closure ordinal of functional F is omega. Duality theory of LP for calculating metric distances

83. Results Approximating quantitative observables: Expectations of continuous functions are continuous Continuous mapping theorems for establishing continuity of quantitative observables

84. Summary Approximate reasoning for real-time probabilistic processes

85. Results for discrete time models BisimulationMetrics Logic(P)CTL(*)Real-valued modal logic CompositionalityCongruenceNon- expansivity ProofsCoinduction

86. Results for continuous time models BisimulationMetrics LogicCSLReal-valued modal logic Compositionality??? ProofsCoinduction

87. Questions?