Programming for Geographical Information Analysis: Advanced Skills Online mini-lecture: Introduction to Bayesian statistics and networks Dr Andy Evans

Bayesian Networks Of course, it may be that we see people in one state, and their actions, but have no way of replicating the rulesets in human language. In this case, we can generate a Bayesian Network. These gives probabilities that states will occur together. They allow you to update the probabilities on new evidence. They allow you to chain these “rules” together to make inferences. A slightly more complex way of looking at sequences of probabilistic events is with Bayesian Statistics, particularly where two events may be associated. Bayesian statistics are used in a number of fields, not just for modelling proposed causal relationships. Chiefly they’re used for looking at what the probability of something happening is after something else has occurred. They’re often used for exploring solution spaces – for example, if one solution is found not to be viable, Bayesian statistics may tell you what the probabilities therefore are of other solutions being right. Bayesian statistics are named after their discoverer, the Reverend Thomas Bayes, an eighteenth century English mathematician. http://www-groups.dcs.st-andrews.ac.uk/~history/Mathematicians/Bayes.html

Bayesian Networks The probability of an event A is equal to the probability of A given some other event B, plus the probability of A given B doesn’t happen. P(A) = P(AB) + P(ABc) P(A) = P(A|B)P(B) + P(A|Bc)P(Bc) Where A|B is the probability of A happening, given B That is, if we know the chance of an event given another event and the chance of an event given the other not happening, we can calculate the chance of the event happening. 1 – P(B)

Bayesian Networks For example, say we want to predict the likelihood we have a republican (“GOP”) president starting a war. We can calculate the likelihood of a war… P(War) = P(WarGvnGOP)P(GOP) + P(WarGvnDemocrat)P(Democrat) 0.70 = 0.66 * 0.64 + 0.76 * 0.36

Bayesian Networks Equally, though the probability of an event happening, given some other event, is the probability of both events happening, divided by the probability of the other event happening. P (B|A) = P(BA) / P(A) P (B|A) = P(B)P(A|B)/P(A) If we have “A” we can use this to calculate B given A.

Bayesian Networks For example, if we have a war, we can use this to calculate the probability we also have a republican president. P(GOPGvnWar) = P(GOP)P(WarGvnGOP)/P(War) 0.6 = (0.64 * 0.66) / 0.7 If a war occurred, we know the probability of that happening, which means we can plug it into the equation to get the probability we have a GOP president. If we’re having a war, and we can also use this as the P(GOP) to calculate the probability of another war: P(War) = P(WarGvnGOP)P(GOP) + P(WarGvnDemocrat)P(Democrat) P (B|A) = P(B)P(A|B)/P(A)

Bayesian statistics Or, more generally… Pr(Bx|A) = Pr(Bx)Pr(A|Bx) Pr(GOPGvnWAR) =Pr(GOP)Pr(WARGvnGOP) ΣPr(Bi)Pr(A|Bi) ΣPr(CAUSES)Pr(WARGvnCAUSES) Where Bi is all the potential causes (or things that must happen in order for A to occur) and Bx is the specific thing we are testing for. ie. Pr(B|A) = Probability of possible Cause B given A. Pr(A|B) = Probability of A given possible Cause B. Pr(A) In the above, B|A means “the probability of B conditional on A”. You can view it as saying that if the event A has happened, what is the probability of B also having happened. Hence, in our example, B|A equals “if the war has happened, what is the probability of the president is a republican” A|B equals “if the president is a republican, what is the probability of war also having happened” Note that these are not necessarily the same thing. It’s sometimes easier to think about things using the inverse probabilities. In our example above this would be Given there’s a war, what is the chances a democrat isn’t in power, vs. what are the chances there’s not peace given a gop is in power. i.e. these are asking two very different things.

Uses Bayesian stats used to control: Help systems. Amazon recommendations. Given a series of situations, and previous responses, what is the likelihood of a response being needed given this situation. Also, other types of systems that learn solutions and predict answers: i.e. AI. Possible uses: Flood gate control. Land use planning. Policy making. For example, see MS Office Helper Not Dead Yet, Wired News, 19 Apr 2001: http://www.wired.com/news/technology/0,1282,43065,00.html

Bayesian Networks In a Bayesian Network the states are linked by probabilities, so: If A then B; if B then C; if C then D Not only this, but this can be updated when an event A happens, propagating the new probabilities by using the new final probability of B to recalculate the probability of C, etc.

Expert Systems All these elements may be brought together in an “Expert System”. These are decision trees, in which rules and probabilities link states. Forward chaining: you input states and the system runs through the rules to suggest a most scenario of action. Backward chaining: you input goals, and the system tells you the states you need to achieve to get there. Don’t have to use Fuzzy Sets or Bayesian probabilities, but often do. Software: Free demo and tutorials: http://www.expertise2go.com/ OpenExpert: http://openexpert.org/ Lists of commercial and freeware tools: http://www.kbsc.com/rulebase.html http://www.pcai.com/web/ai_info/expert_systems.html