1. Chapter 5 Probability Models Introduction –Modeling situations that involve an element of chance –Either independent or state variables is probability or random variable –Markov chain –Random variables –Statistics, system reliability, –Game theory –Casino, ……

2. An example The Problem --- Consider a car rental company with branches in Orlando and Tampa. Each branch rents car to Florida tourists. The company specializes in catering to travel agents who want to arrange tourist activities in both Orlando and Tampa. Consequently, a traveler will rent a car in one city and drop off the car in either city. Travelers begin their itinerary in either city. Cars can be returned to either location, which can cause an imbalance in available cars to rent. Historical data on the percentages of cars rented and returned to these companies is collected from the previous few years as shown here.

3. An example – transition matrix

4. The recorded data form a transition matrix and show that the probability for –Returning a car to Orlando that was rented in Orlando is 0.6; whereas the probability it will be returned in Tampa is 0.4. –Returning a car to Orlando that was rented in Tampa is 0.3; whereas the probability it will be returned in Tampa is 0.7. There are two states – Orlando and Tampa –The sum of the probabilities for transitioning from a present state to next state, which is the sum of the probabilities in each row, ==1--all possible outcomes are taken into account.

5. An example -- model Variables –p n --- the percentage of cars available to rent in Orlando at the end of period n –q n --- the percentage of cars available to rent in Tampa at the end of period n Probabilistic model

6. An example – model solution

8. An example – model interpretation If the two branches begin the year with a total of n cars, after 14 time periods or days approximately 57% of cars will be in Tampa and 43% will be in Orlando. Starting with 100 cars in each location, about 114 cars will be based out of Tampa and 86 will be based out of Orlando in the steady state (and it would take about 5 days to reach this state)

9. Markov chain A Markov chain is a process in which there are the same finite number of states or outcomes that can be occupied at any given time. The states do not overlap and cover all possible outcomes. In a Markov process, the system may move from one state to another one at each time step, and there is a probability associated with this transition for each possible outcome. The sum of the probabilities for transitioning from the present state to the next state is equal to 1 for each state at each time step.

10. A Markov process with two states

11. Markov chain A sequence of events with the following properties: –An event has a finite number of outcomes, called states. The process is always in one of these states –At each stage or period of the process, a particular outcome can transition from its present state to any other state or remain in the same state. –The probability of going from one state to another in a single state is represented by a transition matrix for which the entries in each row lie between 0 and 1; each row sums to 1. These probabilities depend only on the present state and not on past states.

12. Application of Markov chain In biology –Color of a plant root --- yellow or green –Color of pig hair --- black or white These can be explained by Markov chain The outside character of a species is determined by its genome which can be divided into two kinds – dominate (d) and reminder (r). For each outside character, there is two genomes and each can be either d or r.

13. Application of Markov chain Three combinations –d d --- good --- denoted by D –d r --- mixture -- denoted by H –r r --- worse --- denoted by R From the biology theory –With either d d or d r genome combination, the outside character shows dominate (or d), e.g. green in plant. –With r r genome combination, the outside character shows reminder (or r), e.g. yellow in plant.

14. Application of Markov chain In biology theory –A child randomly picks up one genome from the two genomes of his father and one genome from the two genomes of his mother to form its own genome. –Three states: D, H and R Case 1: Parents are both D, child is D Case 2: Parents are D+H, child is either D or H with probability ½ and ½ Case 3: Parents are H+H, child is D or H or R with probability ¼, ½ and ¼

15. Application of Markov chain Problem 1 – Marriage with mixture state –Random pick up one, marry it with a mixture H, for their children, marry them again with mixture, and continue. What is probability of children shows outside characters d and r? Solution: There are three states – D, H & R For each marry, the transition matrix is

16. Application of Markov chain Model Limiting behavior Conclusion – after many generations –The probability of children shows outside character d is 0.25+0.5 =0.75, and outside character r is 0.25. –This theoretical results agree well with experiments!!

17. Application of Markov chain Problem 2 – Marriage with relatives –The parents at the beginning can be either good, mixture or worse, they have many children. The marriage is only taken between children and continue. What happens after many generations? States of parents --- D, H or R States of children – DD, RR, DH, DR, HH, HR

18. Transition matrix

19. Application Model Limiting behavior, when n is very large Conclusion – After several generations, all children will be either only good or only worse and will keep it forever!! No biodiversity!!

20. System reliability Recent Toyota call home problem Computer, automobile, etc. are complicated system! If they perform well for a reasonably long period of time, we say these system are reliable!! The reliability of a component or system is the probability that it will not fail over a specific time period. Define f(t) to be the failure rate of an item, component, or system over time t, thus f(t) is a probability distribution.

21. System reliability Let F(t) be the cumulative distribution function corresponding to f(t). Define the reliability of the item, component, or system by For a complicated system, if we know the reliability of each component, then we can build simple models to examine the reliability of complex systems.

22. System reliability A complicated system consists of many small parts via – series, parallel or combinations Connected by series – A series system is consisted of several independent components and it becomes failure due to the failure of any one of the independent component –An example: A NASA space shuttle’s rocket propulsion system

23. System reliability –System reliability Connected by parallel – it is a system that performs as long as a single one of its components remains operational. –An example:

24. System reliability –System reliability Connected by combination – combining series and parallel relationships

25. System reliability Exercise – find the system reliability