Probability & Random Variables Prof. Jae Young Choi, Ph.D. Pattern Recognition Lab., Jungwon University Department of Biomedical Engineering Email: jyoung.choi@jwu.ac.kr URL: http://bprlab.tistory.com/notice/4 http://www.slideshare.net/MuhannadSaleh/2-random-variables-notes-2p3

RANDOM VARIABLES ▣ RANDOM VARIABLES D/ Random Variable X(s) : (랜덤 변수, 불규칙 변수, rv) 정의 구역이 표본 공간 S 인 실변수 함수로 실험에서 정의된 S 의 원소를 s라면 s∈S, rv는 X(s)로 정의 ~ X(s) rv는 대문자 X, Y, Z로 표시하며, 그 특정값은 소문자 x, y, z로 표시 Ex) Tossing a coin {H, T} and a die {1, 2, 3, 4, 5, 6} A/ 동전 던지기에서 ⑴ H가 나타나면: 주사위의 숫자를 그대로 사용 ~ X ⑵ T가 나타나면: 주사위의 숫자에 2배하여 음수(-)로 만듦 ~ -2X

Random Variables A random variable, x, is a real-valued function defined on the events of the sample space, S. In words, for each event in S, there is a real number that is the corresponding value of the random variable. Viewed yet another way, a random variable maps each event in S onto the real line. That is it. A simple, straightforward definition. Event s X R.V X(s) Mapping

Random Variables

Random Variables (Con’t) Example: Consider again the experiment of drawing a single card from a standard deck of 52 cards. Suppose that we define the following events. A: a heart; B: a spade; C: a club; and D: a diamond, so that S = {A, B, C, D}. A random variable is easily defined by letting x = 1 represent event A, x = 2 represent event B, and so on. Sample space D R.V space 1 2 A B 4 C 3

All a random variable does is map events onto the real line. Random Variables (Con’t) Note the important fact in the examples just given that the probability of the events have not changed All a random variable does is map events onto the real line.

RANDOM VARIABLES

RANDOM VARIABLES ▣ RANDOM VARIABLES Ex) Tossing a coin {H, T} and a die {1, 2, 3, 4, 5, 6} Sample Space S S= {(H,1), (H,2), (H,3), (H,4), (H,5), (H,6), (T,1), (T,2), (T,3), (T,4), (T,5), (T,6)} ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ X(s)= 1 2 3 4 5 6 -2 -4 -6 -8 -10 -12 Mapping (Function)

RANDOM VARIABLES

RANDOM VARIABLES Ex) Continuous rv. ~ Clock Face ⑴ rv X = X(s) = s2 w/ 0 ＜ s ≤ 12 ⑵ Sample Space S Mapping SX = {0 ＜ x ≤ 144} ▶ Random Variable X = X(s) s∈S → X(s) = Numerical Assignment to X based on s in S

RANDOM VARIABLES Conditions for a Function to Be a Random Variable

RANDOM VARIABLES ▣ RANDOM VARIABLES (rv) ▶ Conditions for a Function to be a rv Not a multivalued function, i.e. ∃ a unique value X(s), ∀s ∈ S X가 rv가 되기 위한 조건 2가지 ① 집합 {X≤x}가 임의의 실수 x에 대한 사건이어야 함 P{X≤x} = sum of probabilities, ∀s ∈ S satisfying X(s)≤x ② P{X=∞} = 0 and P{X=-∞}=0 ▶Discrete rv and Continuous rv Discrete rv : Discrete Sample Space에서 정의된 rv ex) 주사위 던지기에서 X(s) = s, w/ s∈{1≤s≤6, s는 정수} Continuous rv : Continuous Sample Space에서 정의된 rv ex) 시계판을 임의로 돌려서 나타나는 실수 X(s) = s2, {0 ＜ x ≤144} Mixed rv : 위의 2가지 rv들을 혼용하여 사용할 경우

RANDOM VARIABLES

RANDOM VARIABLES

RANDOM VARIABLES Example of Discrete Random Variable

RANDOM VARIABLES Example of Discrete Random Variable (cont’d)

RANDOM VARIABLES Thus far we have been concerned with random variables whose values are discrete. To handle continuous random variables we need some additional tools. In the discrete case, the probabilities of events are numbers between 0 and 1. When dealing with continuous quantities (which are not denumerable) we can no longer talk about the "probability of an event" because that probability is zero. This is not as unfamiliar as it may seem. For example, given a continuous function we know that the area of the function between two limits a and b is the integral from a to b of the function. However, the area at a point is zero because the integral from,say, a to a is zero. We are dealing with the same concept in the case of continuous random variables.

RANDOM VARIABLES Probability of Continuous Random Variable

RANDOM VARIABLES D/ Unit-Step Function : u(x) D/ Unit-Impulse Function : d(x) Relationship : u(x) vs. d(x) 1.0 1.0

RANDOM VARIABLES ♣ DISTRIBUTION FUNCTION

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTION

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTION, FX(x) D/ Distribution Function (Cumulative Distribution Function, CDF) : Discrete CDF : Properties of CDF, FX(x) : (1) 경계값 : FX(-∞) = 0, FX(+∞) = 1, 0 ≤ FX(x) ≤ 1 (2) 단조비감소 : FX(x1) ≤ FX(x2), if x1 ＜ x2 (3) 구간확률 : P{x1 ＜ x ≤ x2} = FX(x2) - FX(x1) (4) 우방극한값만 존재하고 연속이다 : Continuous from the right side FX(x+) = FX(x) D/ x+ = lime0 (x + e) w/ e ＞ 0

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTIONS D/ Distribution Function : Axiomatic Skeletons A1/ 0 ≤ FX(x) ≤ 1 (-∞ ＜ x ＜ ∞) A2/ FX(-∞) = 0, FX(∞) = 1 A3/ FX(x) is non-decreasing as x increases. A4/ P(x1 ＜ X ≤ x2) = FX(x2) - FX(x1) T/ P(X ＞ x) = 1 - FX(x) Proof) From A2/ FX(∞) = 1 and from A4/, let x1 = x, x2 = ∞ → P(x ＜ X ＜ ∞) = FX(∞) - FX(x) ∴ P(X ＞ x) = 1 - FX(x)

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTIONS

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTIONS

RANDOM VARIABLES ♣ CUMULATIVE DISTRIBUTION FUNCTIONS

RANDOM VARIABLES ♣ DENSITY FUNCTION, fX(x) D/ Density Function (Probability Density Function, PDF) : Discrete PDF : Properties of PDF, fX(x) : (1) 경계값 : fX(x) > 0, ∀x (2) 전체적분은 1이다 : (3) CDF와의 관계 : (4) 구간확률 :

RANDOM VARIABLES ♣ DENSITY FUNCTION D/ Density Function :

RANDOM VARIABLES ♣ DENSITY FUNCTION D/ Density Function : Axiomatic Skeletons A1/ fX(x) ≥ 0 (-∞ ＜ x ＜ ∞) A2/ A3/ A4/

RANDOM VARIABLES ♣ DENSITY FUNCTION

RANDOM VARIABLES ♣ GAUSSIAN RV D/ Gaussian rv : X is called a“Gaussian rv”if its PDF is represented by a normal density function (정규밀도함수), CDF of a“Gaussian rv”is represented by a normal distribution (정규분포)”

RANDOM VARIABLES ♣ GAUSSIAN RV

RANDOM VARIABLES ♣ GAUSSIAN RV D/ F (x), Normalized Gaussian CDF ~ 평균 0, 분산 1 Properties F (x)에서 x  (x-mX)/sX 를 대입하면 일반 정규밀도함수를 구할 수 있다 x 의 음수 영역에 대해서는 또는

RANDOM VARIABLES ♣ GAUSSIAN RV D/ F (x), Normalized Gaussian CDF ~ 평균 0, 분산 1

RANDOM VARIABLES ♣ GAUSSIAN RV-Example

RANDOM VARIABLES ♣ GAUSSIAN RV-Example

RANDOM VARIABLES ♣ GAUSSIAN RV-Example

RANDOM VARIABLES ♣ GAUSSIAN RV-Q Approximation

RANDOM VARIABLES ♣ GAUSSIAN RV D/ Q-Function : T/ D/ Error Function : Built-in function in MATLAB

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples

RANDOM VARIABLES ♣ Other Distribution and Density Examples Rayleigh Density Rayleigh Distribution

CONDITIONAL DISTRIBUTION & DENSITY Recall, Conditional Probability of A given B: D/ Conditional Distribution : Let an event A = {x| X ≤ x} for rv. X, then P(A | B ) = P{X ≤ x | B } = FX(x | B ) is defined as“conditional distribution function”of X. w/ is the joint event of all outcomes s.t. X(s) ≤ x and s ∈ B, and P(B) ≠ 0

CONDITIONAL DISTRIBUTION & DENSITY Properties of FX(x | B ) : (1) 경계조건 : (2) 단조비감소성 : (3) 구간확률 : (4) 우방극한값만 존재 :

CONDITIONAL DISTRIBUTION & DENSITY ♠ CONDITIONAL DENSITY D/ Conditional Density : “conditional density function”of rv. X is the derivative of the conditional distribution function, s.t. NOTE : If rv. X is discrete or mixed, FX(x|B ) contains step discontinuities, then fX(x|B ) then has impulse functions at discontinuities.

CONDITIONAL DISTRIBUTION & DENSITY ♠ CONDITIONAL DENSITY Properties of fX(x | B ) : (1) 경계조건 : (2) 총밀도적분 : (3) CDF vs. PDF : (4) 구간확률 :

CONDITIONAL DISTRIBUTION & DENSITY ♠ EXAMPLE

CONDITIONAL DISTRIBUTION & DENSITY ♠ EXAMPLE

CONDITIONAL DISTRIBUTION & DENSITY ♠ EXAMPLE

CONDITIONAL DISTRIBUTION & DENSITY ♠ METHODS OF DEFINING CONDITIONING EVENTS 조건 event B를 정하는 방법은 여러 가지가 있다. 예를 들어, B = {X ≤ b} 라면  ☞ Conditional Distribution Function : (i) x ＜ b 일 때 : (ii) x ≥ b 일 때 : Therefore,  ☞ Conditional Density Function :

ONAL DISTRIBUTION & DENSITY ♠ METHODS OF DEFINING CONDITIONING EVENTS From our assumptions that the conditioning event has nonzero probability, we have so that the conditional distribution function is never smaller than the ordinary distribution function

RANDOM VARIABLES ◆ DISCRETE vs. CONTINUOUS DENSITY & DISTR. Discrete Density & Distr. Continuous Density & Distr.

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES ▶ Binomial rv. ~ X ⑴ Binomial Density Function : for n = 1, 2, …, and 0＜p＜1 ⑵ Binomial Distribution Function : NOTE : Bernoulli (independent & repeated) trial experiment ~ S = {success, failure} ▶ Poisson rv. ~ X ⑴ Poisson Density Function : for a real constant b＞0, ⑵ Poisson Distribution Function : b = λT (λ~avg. rate, T~주기) Ex) Counting-type 프로세스 응용: 제조 산업의 불량율, 전화오는 횟수, 음극의 전자 방출율 cf. If n→∞ and p→0, then np ≒ b (constant) & Binomial → Poisson

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES Poisson Binomial

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES ▶ Uniform rv. ~ X ⑴ Uniform Density Fn. : for -∞＜a＜∞, b＞a ⑵ Uniform Distr. Fn. : Ex) PC의 Random Number Generator ▶ Exponential rv. ~ X ⑴ Exponential Density Fn. : for -∞＜a＜∞, b＞0 ⑵ Exponential Distr. Fn. : Ex) 어떤 aircraft로부터 반사된 radar신호 세기의 변화

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES Exponential Uniform

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES ▶ Rayleigh rv. ~ X ⑴ Rayleigh Density Fn. : for -∞＜a＜∞, b＞0 ⑵ Rayleigh Distr. Fn. : Ex) Band Pass Filter (BPF) 통과한 신호 잡음의 진폭, 여러 가지 계측 시스템의 잡음 ☞ NOTE: Refer to MATLAB Examples of Densities & Distributions Rayleigh

RANDOM VARIABLES ♣ DISTRIBUTION & DENSITY EXAMPLES Histograms Uniform Rayleigh Uniform

RANDOM VARIABLES ♡ MATLAB EXAMPLES Histograms % gaushist.m histogram of gausian r.v. n=1000; x=2*randn(1,n)+5*ones(1,n); % generate vector of samples plot(x); % draw 1st figure xlabel('Index'); ylabel('Amplitude'); grid; pause; [m,z]=hist(x); % calculate counts in bins and bin coordinates w=max(z)/10; % calculate bin width mm=m/(1000*w); % find probability in each bin v=linspace(min(x),max(x)); % generate 100 values over range of rv X y=(1/(2*sqrt(2*pi)))*exp(-((v-5*ones(size(v))).^2)/8); % gaussian PDF bar(z,mm,'w'); % plot histogram hold on; % retain histogram plot plot(v,y); % superimpose plot of gaussian PDF xlabel('Random Variable Value'); ylabel('Probability Density'); hold off; % release hold of plot