1. Concepts of Computation
Session 10b
Probability and Statistics
Dr Oded Lachish
(Slides prepared with the support of Dr Paul Newman and Eva Szatmari)

2. Probability and Statistics
Number of heads
Binomial coefficient
Pascal’s triangle
Distributions
Histograms

3. Fair coin Recall, the random process of throwing a dart at a rectangle
Event: Dart landing on blue, with probability ½
Event: Dart landing on red, with probability ½
and we know for sure that one of them happens.
This is like the random process of tossing a fair coin in the air and waiting to see if the side facing up is head or tails
Event: getting heads
Event: getting tails
The probability of the event of getting heads is 1/2 , and the probability of the event of getting tails is 1/2

4. What if the coin is not fair
With a coin that is not fair the events
Event: getting heads
Event: getting tails
Have probability different that ½.
This is like the random process of throwing a dart at a rectangle, when both colours don’t occupy the same area in the rectangle. So, the events
Event: Dart landing on blue
Event: Dart landing on red
have probability different than ½.
Can be 1/3 for the first event and 2/3 for the second.
The importance in both case is that the sum of these two probabilities is 1.

5. Generalising
The random process is tossing a coin, that is not necessarily fair. The events are
Event: getting heads
Event: getting tails
We say that the event of getting heads has probability p.
p – must be a number that is at least 0 and at most 1.
Now, we know that sum of probabilities of the event of getting heads and the event of getting tails is 1.
So,
The probability of the event of getting tails is 1-p.
Clearly, p + 1 – p = 1

6. Generalising
The random process is tossing a coin, that is not necessarily fair. The events are
Event: getting heads, happens with probability p
Event: getting tails, happens with probability 1-p
Now, we have a new random process: tossing the same coin as before twice independently. We also have new events:
Event: getting heads on the first toss and getting heads on the second toss
Event: getting heads on the first toss and getting tails on the second toss
Event: getting tails on the first toss and getting heads on the second toss
Event: getting tails on the first toss and getting tails on the second toss
We notice that all these events are disjoint!
We want to know the probability of each one of them

7. Two coin tosses
The random process is tossing a coin, that is not necessarily fair. The events are
Event: getting heads, happens with probability p
Event: getting tails, happens with probability 1-p
Random process: tossing the same coin as before twice independently. We also have new events:
Event: getting heads on the first toss and getting heads on the second toss
Event: getting heads on the first toss and getting tails on the second toss
Event: getting tails on the first toss and getting heads on the second toss
Event: getting tails on the first toss and getting tails on the second toss
If we toss the coin once we know the event of getting heads, happens with probability p.
So, the first time we toss the coin, with probability p we get heads, and also the second time, with probability p we get heads.
We also know that the events are independent:
So, the event of getting heads on the first toss and getting heads on the second toss, happens with probability p * p.
If p = 1/2, then p * p = ¼. If p = 1/3, then p * p = 1/9.

8. Two coin tosses
The random process is tossing a coin, that is not necessarily fair. The events are
Event: getting heads, happens with probability p
Event: getting tails, happens with probability 1-p
Random process: tossing the same coin as before twice independently. We also have new events:
Event: getting heads on the first toss and getting heads on the second toss, happens with probability p * p
Event: getting heads on the first toss and getting tails on the second toss
Event: getting tails on the first toss and getting heads on the second toss
Event: getting tails on the first toss and getting tails on the second toss
If we toss the coin once we know the event of getting heads, happens with probability p the event of getting tails, happens with probability 1-p
So the first time we toss the coin, with probability p we get heads, and also the second time, with probability p we get heads.
We also know that the events are independent:
So, the event of getting heads on the first toss and getting tailss on the second toss, happens with probability p * (1-p).
If p = 1/2, then p * (1-p) = ¼. If p = 1/3, then p * (1-p) = 1/3 * 2/3=2/9.

9. Two coin tosses
The random process is tossing a coin, that is not necessarily fair. The events are
Event: getting heads, happens with probability p
Event: getting tails, happens with probability 1-p
Random process: tossing the same coin as before twice independently. We also have new events:
Event: getting heads on the first toss and getting heads on the second toss, happens with probability p * p
Event: getting heads on the first toss and getting tails on the second toss, happens with probability p * (1-p)
Event: getting tails on the first toss and getting heads on the second toss, happens with probability (1-p) *p
Event: getting tails on the first toss and getting tails on the second toss, happens with probability (1-p) * (1-p)
We used the same explanation to find out the probability of the other events.

10. 8 coin tosses
What if the random process is: tossing the same coin 8 times independently. The event is:
Event: getting heads five times
If the event was
Event: getting heads for the first 5 times and tails for the last 3 times
The probability is p5*(1-p)3.
How did we get this?
The event of getting heads on a single coin toss happens with probability p.
Since we toss the coins five times in a row independently, the probability of all of them resulting in heads is p times itself 5 times which is p5.
But we tossed the coin 8 times.
So the last three times we got heads, the probability of that happening is (1-p)3. (same explanation like that for the p5)
So we have an event that the first 5 tosses gave us heads and event that the last 3 tosses gave us tails. They rely on different tosses so they are independent.
Thus, of the event of getting heads for the first 5 times and tails for the last 3 times , is p5*(1-p)3

11. 8 coin tosses
What if the random process is: tossing the same coin 8 times independently. The event is:
Event: getting heads five times
Event: getting heads for the first 5 times and tails for the last 3 times, happens with probability is p5*(1-p)3.
Great how does the second event help.
Observe the event
Event: getting heads for the first 4 times and then tails for 3 times in a row and then heads,
Is disjoint to the event getting heads for the first 5 times and tails for the last 3 times (they cant happen simultaneously, it is one or the other), and happens with the same probability p5*(1-p)3, because the order in which we got the heads actually doesn’t matter

12. 8 coin tosses
What if the random process is: tossing the same coin 8 times independently. The event is:
Event: getting heads five times
Conclusion:
The event of getting heads five times, one of the event that events of getting five heads in a specific order, like in the first 5 times, happened.
The probability of each one of these event is p5*(1-p)3.
The events a disjoint since the heads can appear in only one order.
So the probability of one of them happening is the sum of their probabilities.
Which is the number of these orders times p5*(1-p)3.
This number is denoted by which means the number of different ways we can arrange 8 coins in a row so that which exactly 5 having heads on top.

13. Generalising
If the random process is: tossing the same coin n times independently. The event is:
Event: getting heads m times
The probability of this event is n m pm*(1-p)n-m
n m is the number of orders in which we can get exactly m heads
pm is the probability of getting m times heads
(1-p)n-m is the probability of getting m times tails

14. n m - what? Lets do 4 2 . This is 6 as you can see below.
n! = 1*2*3*…*n
So = 4! 2!∗ 4−2 ! = 1∗2∗3∗4 1∗2∗(1∗2) =12 .
n m - is called a binomial coefficient
The above is done systematically, can you see how

15. Distributions

16. Distributions
We used the example, of throwing a dart at a rectangle with three colours, red, blue and purple.
The events were
Event: Dart landing on blue, happens with probability 1/6
Event: Dart landing on red, happens with probability ½
Event: Dart landing on purple, happens with probability 1/3
We actually have a distribution here!
In hand waving: a distribution for a random process is the probabilities of all possible outcomes of the random process.

17. Distributions
We used the example, of throwing a dart at a rectangle with three colours, red, blue and purple.
The events were
Event: Dart landing on blue, happens with probability 1/6
Event: Dart landing on red, happens with probability ½
Event: Dart landing on purple, happens with probability 1/3
We actually have a distribution here!
In hand waving: a distribution for a random process is the probabilities of all possible outcomes of the random process.

18. Distribution, formally
A distribution of a random process is a function from the set of outcomes of the process to the real numbers from 0 to 1.
If f is a distribution of a random process and x is an outcome of the random process, then f(x) is the probability of this outcome.
For example:
If f is the distribution of the random process from the previous slide, then
f( Dart landing on blue ) = 1/6
f( Dart landing on red ) = ½
f( Dart landing on purple ) = 1/3

19. Distribution, formally
The sum
If f is a distribution of a random process. Then the sum of f(x), where x ranges over all the direct outcomes of the random process is 1.
For example: if the random process is
tossing a coin, the direct outcomes are getting heads and getting tails.
Throwing a dart the rectangle on the right,
the direct outcomes are
Dart landing on blue,
Dart landing on red, and
Dart landing on purple.

20. Distribution, formally
If f is a distribution of a random process. Then the sum of f(x)s, over all x that are direct outcomes of the random process is 1.
For example: if the random process is
tossing a coin, the direct outcomes are getting heads and getting tails.
Throwing a dart the rectangle on the right,
the direct outcomes are
Dart landing on blue,
Dart landing on red, and
Dart landing on purple.

21. Distribution, formally
If f is a distribution of a random process. Then the sum of f(x)s, over all x that are direct outcomes of the random process is 1.
Usually, the distribution of a random process is given by providing the probability for every direct outcome of the random process.
(an indirect outcome is an outcome that includes more than one other outcome, like dart landing on red or purple.

22. Distributions are important tools but how do we know them in practice?
In theory there is such a thing as a fair coin.
However, in practice this is not clear.
How do we know if a coin is fair.
In general, we can toss the coin a hundred times and store. We actually don’t expect to get exactly fifty, fifty. However, we expect to be close.
The figure on the right, is a histogram of the results. It depicts how many times each event occurred.
In general, if we repeat the experiment enough times we should get close to the truth (assuming we didn’t damage the coin on the way)

23. Histograms
In general a histogram is simply a way of presenting the results of a repeated random process (doesn’t have to be).
For example,
The following may be the results of an exam.
Like running the sum of running the process of examining a student 77 time in parallel.
One can select different ranges, though they cannot overlap (if this is a standard histogram) and they should cover all the range.

24. Cumulative Histograms
The top Histogram is a regular one
The bottom is a cumulative one, check out the ranges,
Here, the ranges can overlap