1. Please elaborate with your own sketches
Probability Intro
Professor Jim Ritcey
EE 416
Revised Fall 2013
Please elaborate with your own sketches

2. Disclaimer
These notes are not complete, but they should help in organizing the class flow.
Please augment these notes with your own sketches and math. You need to actively participate.
It is virtually impossible to learn this from a verbal description or these ppt bullet points. You must create your own illustrations and actively solve problems.

3. Random Experiments An experiment is prescribed with N outcomes
finite {1 2 3} countable {1 2 3 …} uncountable {z>0}
A chance mechanism selects outcome – uncertainty
We have some knowledge of the likelihood of occurrence – often measured empirically through relative frequency in repeated independent trials
Kolmogorov developed consistent axiomatic framework for probability

4. Relationship to Set Theory
Set Theory Probability
Universe Sample space
Elements Outcomes
Algebra of subsets Events
To each event the theory allows/assigns a number
Probability (event) indicates its likelihood of occurrence 0<= p <= 1

5. Set Operations – PPT Notation
Complement A^c = {c: c in S but c not in A}
Union (cup) A+B = {c: c in A or c in B }
Intersection (cap) AB = {c: c in A and c in B }
Often we use cap/cup/overbar (complement) in traditional math notation
MATLAB provides set operations as functions - very useful when working with large sets

6. Algebra 0f Events Each experiment has outcomes in sample space S
Subsets of outcomes are called events.
The set of events must be an algebra –closure
For events A,B,C, … then
All complements are events A^c, B^c, …
All finite unions & intersections are events
All countable unions & intersections are events
For finite sample spaces the events consist of the set of all subsets. But too large for infinite S

7. Probability Triple (S, E, P) (Sample Space, Events, Probability)
Sample space – set of all possible outcomes
Events – family/collection of all subsets to which we assign probabilities. Subject to closure under set operations
Probability – maps events to unit interval [0,1] subject to 3 Axioms. For events A,B,C,…
(1) P(A) >= 0 non-negative probability
(2) P(S) = 1 something happens w probability 1
(3) A, B disjoint P(A+B) = P(A) + P(B)
(3’) Can extend (3) to countable unions

8. Examples S = {T,H} = {0,1} 1 coin toss
E = { S^c = {}, {0}, {1}, S = {0,1} }
Given P(1) can compute probability of any e in E
In the model, p=P(1), is a parameter
S = {TT,HT,TH,HH} = {00,10,01,11} 2 coin toss
Each event can be given an intuitive name

9. S = {00,10,01,11} {H=1,T=0} e 1 2 3
outcomes
Empty set = nothing happens
Exactly {11} “2 heads”
Exactly {01} “tail then head” 4
{01} or {11} “head on second toss} 5
Exactly {10} 6 7 10
Exactly “2 of the same” 14 15
“Anything but 2 heads” complement of #2 16
S = something happens

10. Probability of an Event
Event is a subset of outcomes
An event is realized (occurs) if any of its members occurs
The outcomes of an experiment are by definition disjoint
P (event) is the sum of the probabilities of outcomes that define it 
P(event) = P(outcome_1) + …
But events can contain many outcomes

11. How to we know P P( any event ) is known in the theory. But how?
Relative frequency in independent repeated trials
Uniformity considerations – insufficient reason to suspect otherwise “at random”
Subjective – expert opinion, must be consistent
What is the probability that the sun will rise tomorrow p=1? p<1?
What is the probability that you will obtain above 3.4 in EE416?
Engineering usage focuses on relative frequency
Based on past ( but relevant) history

12. Theorems are derived from Axioms
P(A^c) = 1 - P(A)
P(S) = 1  P(empty set) = 0
P(A+B) = P(A) + P(B) - P(AB)
P(A+B) < = P(A) + P(B)
A subset of B then P(A) <= P(B)
Proofs are instructive usually they relate to disjoint events

13. Partitions If A us an event, and A in B in S (sample space)
Let B_1, …B_N partition B
Then
P(A) = P(AB_1) + … +P(AB_N)
Simplifies more complicated events

14. General Problem of Computing Prob
Given a combination of events, with knowing prob of each separately, determine prob of combination
Typical combinations “at least one” “k or more of”
Always can be written as set operations
Always can be enumerated, but can be a long list

15. 1 Coin Toss Experiment – toss a coin S –{T,H} |S| =2
Here let the events be the power set Pwr(S)
Set of all subsets with 2^2 -4 elements
S^c = Empty set, H, T, {H or T} =S closed family
Assign P(H) = p, p = 1/2 only for a fair coin
Consistent with axioms if p in [0,1] & P(T) = 1-p
Often insufficient reason to take p neq 1/2
Often p should be measured empirically, the fraction of heads in many trials (how many?)
The outcomes have prob {pT= 1-p, pH = p}

16. Geometric Probability
In many board games spinners are used. You cut a circular region into n congruent pizza slices, and flip a spinner. It is obvious that the probability of any outcome is p = 1/n.
This can be argued to be the ratio of the arc length of the wedge to arc length of the circle
Geometric probabilities reduce to calculation of relative length/area/volume and are based on an “at random” assumption. This implies uniformity.

17. 2 Coin Toss S = {TT, TH, HT, HH} ordered (T1, T2)
Pwr(S) has 2^4 = 16 possible events
It is not possible to combine outcomes in any other way. Next assign a consistent probability to
(p1, p2,p3,p4) p_i in [0,1] sum p_i = 1
Insufficient reason takes p_i = ¼
But it depends on the random experiment (toss)
Allowable is any consistent set of probabilities.
Then eg, P(T1=TorT2=T) = P(HT + TH + TT) = =P(HT) + P(TH) + P(TT)

18. Independence Two events A,B are independent when
P(AB) = P(A)P(B) multiply probabilities
This is a property of the probability assignment
Disjoint events are
Dependent events – statistical relationship/linking
This allows for prediction of one given the other
Not a one-way causal relationship

19. 2 Coin Toss Label the outcomes (00, 01, 10, 11) where T=0,H=1
Suppose we measured
P(00) = P(01) = 0.30
P(10) = P(11) = 0.38
Are the tosses independent?
P(t1=0)=P(00)+P(01) = =0.40 P(t1=1)= 1-.4
P(t2=0)=P(00)+P(10) = =0.32 P(t2=1 =
Independence says that
P(t1=0)P(t2=0) = P(00) but this does not hold!
Under this P, Events are not independent!

20. Are disjoint events independent? NO
Let A,B mutually exclusive AB = empty set
P(AB) = P(empty set) = 1-P(S) = 0
They are not independent
Disjoint events are highly dependent – if one event occurs, the other cannot have occurred.
This must be distinguished from independence
P(AB)=P(A)P(B) which might be zero if P(A)=0
A property of the Prob Assnment, not the events

21. Communications Networks
A network is made up of nodes and links (edges)
The link either work 1 or fail 0 with p=P(1) q=P(0)
Links can be connected in various ways
Find Probability P_N that a network works
Series Net: all links work P_N = p^N
Parallel Net: at least 1 link works P_N = 1 - q^N
Need to work through the derivation
Networks can combine series and parallel

22. Target Detection
A radar takes N looks at a target, target is detected (when present) with p=0.9
How does detection prob improve as we increase N
P_N := 1 –(1-p)^N but we need to explore numerically
For small N,
P = [ p, 2p-p^2, 3p -3p^2 +p^3, … ]
Is this a series or parallel problem?

23. Birthday Problem Birthdays occur at random on [1:365] no leap yr
Sample r people’s birthdays, n=365
find P(hit) =P( 2or more birthdays in sample)
P(no hit) = # ways to select wo rep/# ways w rep
1-P(hit)= (n)_r / n^r where (n)_r = n!/(n-r)!
Numerical implications are interesting

24. Birthday Problem Numerics
Place r balls at random in n bins. What is the probability that 2 or more land in any bin
N=365, r=23 for a 50:50 chance of hits
Randomness causes clumping!
Table shows P(hit) vs r, number of balls placed
Balls can be calls, bins can be channels/frequencies 20
0.4 23
0.507 30
0.706 35
0.814 40
0.89

25. Birthday Problem Analysis
Give a simple expression for
P(r,n) = 1 –(n)_r/n^r, when n is large
Write Q = 1-P as a product
Q=1-P(r,n) (n)_r/n^r = Prod_1^(r-1) [1-j/n]
Go to a log form exp( sum of logs)
Approximate the sum & simplify
This is computational probability
Or just compute use Gamma functions

26. Birthday Prob (n=365)

27. Matlab Code function birthdayprob( N, K);
%function birthdaypob( N, K);
%compute all birthday probabilities.
kall =[0:K-1];
%Pnohit =(N)_k/N^k;
Pnohit = exp( cumsum( log(1 -kall/N) ) );
Pnoapp = exp( -(kall+1).*(kall)/(2*N) );
% a very good approximation!
X = [1:K]'*[ 1 1];
P1 = [ Pnohit;1-Pnohit]';
P2 = [ Pnoapp; 1-Pnoapp]';
% note that we are plotting both tails
figure(1); semilogy( X, P1,'-*');
hold on; semilogy( X, P2,'-o'); hold off
title('Exact vs Approx hit Probabilities');
xlabel('Number of Balls Thrown');grid;

28. Ranking Problems
You meet Alice, she has two brothers Bob and Chris. You don’t know anything more.
What is the probability that Alice is the oldest?
Use Insufficient Reason to determine
P(A oldest) = 1/3
Now she tells you that Alice is older than Bob.
Determine P(A is oldest).
Hint: Consider the simplest sample space
Hint: Consider the sample space uses outcomes are all possible rankings of age. (A>B>C, etc)