Markov Processes Aim Higher

What Are They Used For? Markov Processes are used to make predictions and decisions where results are partly random but may also be influenced by known factors Applications include weather forecasting, economic forecasting, manufacturing and robotics

What Are They? Markov Processes use a series of matrices to predict the outcome of a chain of random events which may be influenced by known factors These matrices predict the probability of a system changing between states in one time step based on probabilities observed in the past

When predicting the weather it may be sensible, based on past observations, to assume that it is more likely to rain tomorrow given that it is raining today. Probabilities can be indicated for a given time step P(R 2 |R 1 ) > P(R 2 |S 1 )orP RR > P RS This is not an accurate forecasting method but it can give some indication of the likely probability of the weather changing from one state – rain, sun, cloud, snow, etc – to another. Examples of Application

Creating A Markov System An initial transition matrix is required to show the probability of state changes in one time step: One time step in this case could be decided as 24 hours RainCloudSun Rain Cloud Sun

Weather Forecasting We can now predict tomorrow’s weather using these probabilities and applying them to today’s weather. If it is raining today, there is a 60% chance of rain tomorrow and only a 20% chance of sun RainCloudSun SunCloudRain

Distribution Vectors The number of units in each state depends on both the transition probability and the number in each state initially. For example, on the stock market the number of shares an investor owns in four different companies may change with time However, the total number he owns in each one will depend how many of each he begins with.

Distribution Vectors: Shares The Distribution after n time steps can be obtained as: vP n =