Concepts of Computation Session 10b Probability and Statistics Dr Oded Lachish Email: oded@dcs.bbk.ac.uk (Slides prepared with the support of Dr Paul Newman and Eva Szatmari)

Probability and Statistics Number of heads Binomial coefficient Pascal’s triangle Distributions Histograms

Fair coin Recall, the random process of throwing a dart at a rectangle Event: Dart landing on blue, with probability ½ Event: Dart landing on red, with probability ½ and we know for sure that one of them happens. This is like the random process of tossing a fair coin in the air and waiting to see if the side facing up is head or tails Event: getting heads Event: getting tails The probability of the event of getting heads is 1/2 , and the probability of the event of getting tails is 1/2

What if the coin is not fair With a coin that is not fair the events Event: getting heads Event: getting tails Have probability different that ½. This is like the random process of throwing a dart at a rectangle, when both colours don’t occupy the same area in the rectangle. So, the events Event: Dart landing on blue Event: Dart landing on red have probability different than ½. Can be 1/3 for the first event and 2/3 for the second. The importance in both case is that the sum of these two probabilities is 1.

Generalising The random process is tossing a coin, that is not necessarily fair. The events are Event: getting heads Event: getting tails We say that the event of getting heads has probability p. p – must be a number that is at least 0 and at most 1. Now, we know that sum of probabilities of the event of getting heads and the event of getting tails is 1. So, The probability of the event of getting tails is 1-p. Clearly, p + 1 – p = 1

Generalising The random process is tossing a coin, that is not necessarily fair. The events are Event: getting heads, happens with probability p Event: getting tails, happens with probability 1-p Now, we have a new random process: tossing the same coin as before twice independently. We also have new events: Event: getting heads on the first toss and getting heads on the second toss Event: getting heads on the first toss and getting tails on the second toss Event: getting tails on the first toss and getting heads on the second toss Event: getting tails on the first toss and getting tails on the second toss We notice that all these events are disjoint! We want to know the probability of each one of them

Two coin tosses The random process is tossing a coin, that is not necessarily fair. The events are Event: getting heads, happens with probability p Event: getting tails, happens with probability 1-p Random process: tossing the same coin as before twice independently. We also have new events: Event: getting heads on the first toss and getting heads on the second toss Event: getting heads on the first toss and getting tails on the second toss Event: getting tails on the first toss and getting heads on the second toss Event: getting tails on the first toss and getting tails on the second toss If we toss the coin once we know the event of getting heads, happens with probability p. So, the first time we toss the coin, with probability p we get heads, and also the second time, with probability p we get heads. We also know that the events are independent: So, the event of getting heads on the first toss and getting heads on the second toss, happens with probability p * p. If p = 1/2, then p * p = ¼. If p = 1/3, then p * p = 1/9.

Two coin tosses The random process is tossing a coin, that is not necessarily fair. The events are Event: getting heads, happens with probability p Event: getting tails, happens with probability 1-p Random process: tossing the same coin as before twice independently. We also have new events: Event: getting heads on the first toss and getting heads on the second toss, happens with probability p * p Event: getting heads on the first toss and getting tails on the second toss Event: getting tails on the first toss and getting heads on the second toss Event: getting tails on the first toss and getting tails on the second toss If we toss the coin once we know the event of getting heads, happens with probability p the event of getting tails, happens with probability 1-p So the first time we toss the coin, with probability p we get heads, and also the second time, with probability p we get heads. We also know that the events are independent: So, the event of getting heads on the first toss and getting tailss on the second toss, happens with probability p * (1-p). If p = 1/2, then p * (1-p) = ¼. If p = 1/3, then p * (1-p) = 1/3 * 2/3=2/9.

Two coin tosses The random process is tossing a coin, that is not necessarily fair. The events are Event: getting heads, happens with probability p Event: getting tails, happens with probability 1-p Random process: tossing the same coin as before twice independently. We also have new events: Event: getting heads on the first toss and getting heads on the second toss, happens with probability p * p Event: getting heads on the first toss and getting tails on the second toss, happens with probability p * (1-p) Event: getting tails on the first toss and getting heads on the second toss, happens with probability (1-p) *p Event: getting tails on the first toss and getting tails on the second toss, happens with probability (1-p) * (1-p) We used the same explanation to find out the probability of the other events.

8 coin tosses What if the random process is: tossing the same coin 8 times independently. The event is: Event: getting heads five times If the event was Event: getting heads for the first 5 times and tails for the last 3 times The probability is p5*(1-p)3. How did we get this? The event of getting heads on a single coin toss happens with probability p. Since we toss the coins five times in a row independently, the probability of all of them resulting in heads is p times itself 5 times which is p5. But we tossed the coin 8 times. So the last three times we got heads, the probability of that happening is (1-p)3. (same explanation like that for the p5) So we have an event that the first 5 tosses gave us heads and event that the last 3 tosses gave us tails. They rely on different tosses so they are independent. Thus, of the event of getting heads for the first 5 times and tails for the last 3 times , is p5*(1-p)3

8 coin tosses What if the random process is: tossing the same coin 8 times independently. The event is: Event: getting heads five times Event: getting heads for the first 5 times and tails for the last 3 times, happens with probability is p5*(1-p)3. Great how does the second event help. Observe the event Event: getting heads for the first 4 times and then tails for 3 times in a row and then heads, Is disjoint to the event getting heads for the first 5 times and tails for the last 3 times (they cant happen simultaneously, it is one or the other), and happens with the same probability p5*(1-p)3, because the order in which we got the heads actually doesn’t matter

8 coin tosses What if the random process is: tossing the same coin 8 times independently. The event is: Event: getting heads five times Conclusion: The event of getting heads five times, one of the event that events of getting five heads in a specific order, like in the first 5 times, happened. The probability of each one of these event is p5*(1-p)3. The events a disjoint since the heads can appear in only one order. So the probability of one of them happening is the sum of their probabilities. Which is the number of these orders times p5*(1-p)3. This number is denoted by 8 5 which means the number of different ways we can arrange 8 coins in a row so that which exactly 5 having heads on top.

Generalising If the random process is: tossing the same coin n times independently. The event is: Event: getting heads m times The probability of this event is n m pm*(1-p)n-m n m is the number of orders in which we can get exactly m heads pm is the probability of getting m times heads (1-p)n-m is the probability of getting m times tails

n m - what? Lets do 4 2 . This is 6 as you can see below. n! = 1*2*3*…*n So 4 2 = 4! 2!∗ 4−2 ! = 1∗2∗3∗4 1∗2∗(1∗2) =12 . n m - is called a binomial coefficient The above is done systematically, can you see how

Distributions

Distributions We used the example, of throwing a dart at a rectangle with three colours, red, blue and purple. The events were Event: Dart landing on blue, happens with probability 1/6 Event: Dart landing on red, happens with probability ½ Event: Dart landing on purple, happens with probability 1/3 We actually have a distribution here! In hand waving: a distribution for a random process is the probabilities of all possible outcomes of the random process.

Distributions We used the example, of throwing a dart at a rectangle with three colours, red, blue and purple. The events were Event: Dart landing on blue, happens with probability 1/6 Event: Dart landing on red, happens with probability ½ Event: Dart landing on purple, happens with probability 1/3 We actually have a distribution here! In hand waving: a distribution for a random process is the probabilities of all possible outcomes of the random process.

Distribution, formally A distribution of a random process is a function from the set of outcomes of the process to the real numbers from 0 to 1. If f is a distribution of a random process and x is an outcome of the random process, then f(x) is the probability of this outcome. For example: If f is the distribution of the random process from the previous slide, then f( Dart landing on blue ) = 1/6 f( Dart landing on red ) = ½ f( Dart landing on purple ) = 1/3

Distribution, formally The sum If f is a distribution of a random process. Then the sum of f(x), where x ranges over all the direct outcomes of the random process is 1. For example: if the random process is tossing a coin, the direct outcomes are getting heads and getting tails. Throwing a dart the rectangle on the right, the direct outcomes are Dart landing on blue, Dart landing on red, and Dart landing on purple.

Distribution, formally If f is a distribution of a random process. Then the sum of f(x)s, over all x that are direct outcomes of the random process is 1. For example: if the random process is tossing a coin, the direct outcomes are getting heads and getting tails. Throwing a dart the rectangle on the right, the direct outcomes are Dart landing on blue, Dart landing on red, and Dart landing on purple.

Distribution, formally If f is a distribution of a random process. Then the sum of f(x)s, over all x that are direct outcomes of the random process is 1. Usually, the distribution of a random process is given by providing the probability for every direct outcome of the random process. (an indirect outcome is an outcome that includes more than one other outcome, like dart landing on red or purple.

Distributions are important tools but how do we know them in practice? In theory there is such a thing as a fair coin. However, in practice this is not clear. How do we know if a coin is fair. In general, we can toss the coin a hundred times and store. We actually don’t expect to get exactly fifty, fifty. However, we expect to be close. The figure on the right, is a histogram of the results. It depicts how many times each event occurred. In general, if we repeat the experiment enough times we should get close to the truth (assuming we didn’t damage the coin on the way)

Histograms In general a histogram is simply a way of presenting the results of a repeated random process (doesn’t have to be). For example, The following may be the results of an exam. Like running the sum of running the process of examining a student 77 time in parallel. One can select different ranges, though they cannot overlap (if this is a standard histogram) and they should cover all the range.

Cumulative Histograms The top Histogram is a regular one The bottom is a cumulative one, check out the ranges, Here, the ranges can overlap