Download presentation
Presentation is loading. Please wait.
1
IENG 362 Markov Chains
2
Steady State Conditions
Proposition: For any irreducible ergodic M.C., lim Pij(n) exists and is independent of i. lim ( ) n ij j P = > p j i ij M P = å p j M = å p 1
3
Steady State Conditions
Heuristic p j n i M ij P X = å { } | ( )
4
Steady State Conditions
Now, P ij n ik k M kj ( ) + = å 1
5
Steady State Conditions
Now, P ij n ik k M kj ( ) + = å 1 Taking the limit as n p j k M kj P = å
6
å Example; Inventory p P = p 080 632 264 08 184 368 = + . P = 080 184
080 184 368 632 264 . p j k M kj P = å p 1 2 3 080 632 264 08 184 368 = + .
7
Example; Inventory p 1 2 3 080 632 264 08 184 368 = + . 1 2 3 = + p
8
Example; Inventory p 368 1 - = . ( ) p 582 = . p 080 632 264 08 184
1 2 3 080 632 264 08 184 368 = + . Choose p0 = 1 p 3 368 1 - = . ( ) p 3 582 = .
9
Example; Inventory p 080 632 264 08 184 368 = + . p 368 1 582 - = + .
1 2 3 080 632 264 08 184 368 = + . p0 = 1, p3 = 0.582 p 2 368 1 582 - = + . ( ) p 2 9212 = .
10
Example; Inventory p 080 632 264 08 184 368 = + . p 368 184 582 - = +
1 2 3 080 632 264 08 184 368 = + . p0 = 1, p3 = 0.582, p2 = p 1 368 184 582 - = + . ( ) (.9212) ) p 1 1.0 = .
11
Example; Inventory p 1 2 3 000 921 582 503 = å .
12
å Example; Inventory p 000 921 582 503 = . p 503 285 263 166 / . Þ = 1
1 2 3 000 921 582 503 = å . p 1 2 3 503 285 263 166 / . Þ =
13
Example; Inventory p 1 2 3 285 263 166 . = j i ij M P = å p 1
14
Example; Inventory å p 285 263 166 . = P p 1 lim P = > p P . 286
1 2 3 285 263 166 . = j i ij M P = å p 1 P ( ) . 16 286 263 166 = lim ( ) n ij j P = > p
15
Special Cases Irreducible positive recurrent periodic Markov Chain, period = d lim ( ) n jj nd j P d = m p
16
Special Cases Ex: 1 = P If we start in state 0, P does not exist , .
00 1 2 3 4 ( ) , . lim = Ex: 1 P = 1 1 1
17
Special Cases Ex: lim P = 1 p = 1 2 p 1 = P However, 1 1 1 ( ) n jj j
= 2 1 p j = 1 2 p Ex: 1 P = 1 1 1
Similar presentations
