Review of Probability Theory

© Tallal Elshabrawy 2 Review of Probability Theory Experiments, Sample Spaces and Events Axioms of Probability Conditional Probability Bayes’s Rule Independence Discrete & Continuous Random Variables

© Tallal Elshabrawy 3 Random Experiment It is an experiment whose outcome cannot be predicted with certainty Examples: Tossing a Coin Rolling a Die

© Tallal Elshabrawy 4 Random Experiment in Communications Why is this a random experiment? We do not know The amount of noise that will affect the transmitted bit Whether the bit will be received in error or not Transmission of Bits across a Communication Channel Waveform Generator Waveform Detection Channel v r x y 0 T 0 T +A V. -A V. vivi v i =1 v i =0 xixi 0 y i >0 y i <0 r i =1 r i =0 riri + zizi ]-∞, ∞[ yiyi

© Tallal Elshabrawy 5 Random Experiment in Networks Why is this a random experiment? We do not know Whether the packet will reach the destination or not If the packet reaches the destination, how long would it take to get there? Transferring a Packet across a Communication Network Packet

© Tallal Elshabrawy 6 Sample Space The set of all possible outcomes Tossing a coin S = {H,T} Rolling a die S = {1,2,3,4,5,6} The AWGN in a Communication Channel S = ] -∞, ∞ [ Heads Tails xixi + zizi yiyi

© Tallal Elshabrawy 7 Event An event is a subset of the sample space S Examples Let A be the event of observing one head in a coin flipped two times A = {HT,TH} Let B be the event of observing two heads in a coin flipped twice B = {HH}

© Tallal Elshabrawy 8 Axioms of Probability Probability of an event is a measure of how often an event might occur

© Tallal Elshabrawy 9 Example Let Event A characterize that the outcome of rolling the die once is smaller than 3 A = {1,2} P(A) = 2/6 = 1/3 Let Event B characterize that the outcome of rolling the die once is an even number B = {2,4,6} P(B) = 3/6 = 1/ AB S 3 5

© Tallal Elshabrawy 10 Conditional Probability Probability of event B given A has occurred Probability of event A given B has occurred

© Tallal Elshabrawy 11 Example Two cards are drawn in succession without replacement from an ordinary (52 cards) deck. Find the probability that both cards are aces Let A be the event that the first card is an ace Let B be the event that the second card is an ace

© Tallal Elshabrawy 12 Conditional Probability in Communications Conditioned on v=1, what is the probability of making an error? r=0 Decision Zone 0 T 0 T +A V. -A V. vivi v i =1 v i =0 xixi 0 y i >0 y i <0 r i =1 r i =0 riri + zizi ]-∞, ∞[ yiyi

© Tallal Elshabrawy 13 Bayes’s Rule

© Tallal Elshabrawy 14 Theorem of Total Probability Let B 1, B 2, …, B n be a set of mutually exclusive and exhaustive events.

© Tallal Elshabrawy 15 Bayes’s Theorem Let B 1, B 2, …, B n be a set of mutually exclusive and exhaustive events.

© Tallal Elshabrawy 16 Independent Events A and B are independent if P(B|A) = P(B) P(A|B) = P(A) P(A,B) = P(A)P(B)

© Tallal Elshabrawy 17 Example Let A be the event that the grades will be out on Thursday  P(A) Let B be the even that I will get A+ in Random Signals and Noise  P(B) So What is the probability that I get A+ if the grades are out on Thursday  P(B|A) = P(B)

© Tallal Elshabrawy 18 Random Variable Characterizes the experiment in terms of real numbers Example X is the variable for the number of heads for a coin tossed three times X = 0,1,2,3 Discrete Random Variables The random variable can only take a finite number of values Continuous Random Variables The random variable can take a continuum of values

© Tallal Elshabrawy 19 Bernoulli Discrete Random Variable Represents experiments that have two possible outcomes. These experiments are called Bernoulli Trials Associates values {0, 1} with the two outcomes such that P[X = 0] = 1-p P[X = 1] = p Examples Coin tossing experiment maps a ‘Heads’ to X = 1 and a ‘Tails’ to X = 0 (or vice versa) such that p=0.5 for a fair coin Digital communication system where X = 1 represents a bit received in error and X = 0 corresponds to a bit received correctly. In such system p represents the channel bit error probability

© Tallal Elshabrawy 20 Binomial Discrete Random Variable A random variable that represents the number of occurrences of ‘1’ or ‘0’ in n Bernoulli trials The corresponding random variable X may take and values from {0, 1, 2, …, n} The probability mass function PMF for having k ‘1’ in n Bernoulli trials is P[X = k] = n C k p k (1-p) n-k Examples In a digital communication system, the number of bits in error in a packet depicts a Binomial discrete random variable

© Tallal Elshabrawy 21 Geometric Discrete Random Variable Geometric distribution describes the number of Bernoulli trials in succession are conducted until some particular outcome is observed (lets say ‘1’) The corresponding random variable X may take and values from {1, 2, 3, …, ∞} The probability mass function PMF for having k Bernoulli trials in succession until an outcome of ‘1’ is observed P[X = k] = (1-p) k-1 p Examples: In a communication network, the number of transmissions until a packet is received correctly follows a Geometric distribution