EEE Probability and Random Variables Huseyin Bilgekul EEE 461 Communication Systems II Department of Electrical and Electronic Engineering Eastern Mediterranean University  Why Probability in Communications  Probability  Random Variables  Probability Density Functions  Cumulative Distribution Functions

EEE Why probability in Communications? Modeling effects of noise –quantization –Channel –Thermal What happens when noise and signal are filtered, mixed, etc? Making the “best” decision at the receiver

EEE 461 3Signals Two types of signals –Deterministic – know everything with complete certainty –Random – highly uncertain, perturbed with noise Which contains the most information? Information content is determined from the amount of uncertainty and unpredictability. There is no information in deterministic signals Information = Uncertainty x(t)x(t)y(t)y(t) (t)(t) Let x(t) be a radio broadcast. How useful is it if x(t) is known? Noise is ubiquitous.

EEE Need for Probabilistic Analysis Consider a server process –e.g. internet packet switcher, HDTV frame decoder, bank teller line, instant messenger video display, IP phone, multitasking operating system, hard disk drive controller, etc., etc. Rejected customer, Queue full Customers arrive at random times Queue, Length L Server: 1 customer per  seconds Satisfied customer

EEE Probability Definitions Random Experiment – outcome cannot be precisely predicted due to complexity Outcomes – results of random experiment Events – sets of outcomes that meet a criteria, roll of a die greater than 4 ESample Space – set of all possible outcomes, E (sometimes called the Universal Set)

EEE Example B={  ,  ,   } Complement –B C ={  ,  ,   } Union Intersection Null Set (  ), empty set E AoAo B AeAe

EEE Relative Frequency n A – number of elements in a set, e.g. the number of times an event occurs in N trials Probability is related to the relative frequency For N small, fraction varies a lot; usually gets better as N increases

EEE Joint Probability Some events occur together –Sum of two dice is 6 –Chance of drawing a pair of jacks Events can be –mutually exclusive (no intersection) – tossing a coin –Intersect and have common elements The probability of a JOINT EVENT, AB, is

EEE Bayes Theorem and Independent Events

EEE Axioms of Probability Probability theory is based on 3axioms –P(A) >0 –P(E) = 1 –P(A+B) = P(A) + P(B) If P(AB) = 

EEE Random Variables Definition: A real-valued random variable (RV) is a real- valued function defined on the events of the probability system EventRV Value P(x) A30.2 B-20.5 C00.1 D0.2 E A D C B P(x) x

EEE Cumulative Density Function The cumulative density function (CDF) of the RV, x, is given by F x (a)=P x (x<a) F x (a) a P(x) x

EEE Probability Density Function The probability density function(PDF) of the RV x is given by f(x) Shows how probability is distributed across the axis fx(x)fx(x) x

EEE Types of Distributions Discrete-M discrete values at x 1, x 2, x 3,..., x m Continuous- Can take on any value in an defined interval fx(x)fx(x) x Fx(a)Fx(a) x F x (a) a f x (x) x DISCRETE Continuous

EEE Properties of CDF’s F x (a) is a non decreasing function 0 < F x (a) < 1 F x (-infinity) = 0 F x (infinity) = 1 F(a) is right-hand continuous

EEE PDF Properties f x (x) is nonnegative, f x (x) > 0 The total probability adds up to one f x (x) PDF F x (a) CDF 1 1

EEE Calculating Probability To calculate the probability for a range of values fx(x)fx(x) b a b a F(b) F(a) AREA= F(b)- F(a)

EEE Discrete Random Variables Summations are used instead of integrals for discrete RV. Discrete events are represented by using DELTA functions.

EEE PDF and CDF of a Triangular Wave -A A Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2. Sweep a narrow window across the waveform and measure the relative frequency of occurrence of different voltages. s(t) fx(x)fx(x) A -A fx(x)fx(x) 0 1/2A A -A

EEE PDF and CDF of a Triangular Wave fV(v)fV(v) 0 Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2. 1/ FV(v)FV(v) 0 3/

EEE fV(v)fV(v) 0 Calculate Probability that the amplitude of a triangle wave is in the range [0.5,1] v, if A=2. 1/ FV(v)FV(v) 0 3/ /8 PDF and CDF of a Triangular Wave

EEE PDF and CDF of a Square Wave -A A Calculate Probability that the amplitude of a square wave is at +A. Sketch PDF and CDF s(t) fx(x)fx(x) 0 A-A

EEE PDF and CDF of a Square Wave Calculate Probability that the amplitude of a square wave is at +A. 1/4 Sketch PDF and CDF -A A s(t) fx(x)fx(x) 0 A-A Fx(x)Fx(x) 0 A

EEE Ensemble Averages The expected value (or ensemble average) of y=h(x) is:

EEE Moments The r th moment of RV x about x=x o is