1. Probability Theory Part 1: Basic Concepts

2. Sample Space - Events  Sample Point The outcome of a random experiment  Sample Space S The set of all possible outcomes Discrete and Continuous  Events A set of outcomes, thus a subset of S Certain, Impossible and Elementary

3. Set Operations  Union  Intersection  Complement  Properties Commutation Associativity Distribution De Morgan’s Rule S

4. Axioms and Corollaries  Axioms   If  If A 1, A 2, … are pairwise exclusive  Corollaries 

5. Computing Probabilities Using Counting Methods  Sampling With Replacement and Ordering  Sampling Without Replacement and With Ordering  Permutations of n Distinct Objects  Sampling Without Replacement and Ordering  Sampling With Replacement and Without Ordering

6. Conditional Probability  Conditional Probability of event A given that event B has occurred  If B 1, B 2,…,B n a partition of S, then (Law of Total Probability) S B1B1 B3B3 B2B2 A

7. Bayes’ Rule  If B 1, …, B n a partition of S then 01 1-pp 1010 1-εε ε input output Example Which input is more probable if the output is 1? A priori, both input symbols are equally likely.

8. Event Independence  Events A and B are independent if  If two events have non- zero probability and are mutually exclusive, then they cannot be independent C AB ½ ½ ½ ½ ½ 11 1 1 11

9. Sequential Experiments  Sequences of Independent Experiments E 1, E 2, …, E j experiments A 1, A 2, …, A j respective events Independent if  Bernoulli Trials Test whether an event A occurs (success – failure) What is the probability of k successes in n independent repetitions of a Bernoulli trial? Transmission over a channel with ε = 10 -3 and with 3-bit majority vote