1. ELEN 5346/4304 DSP and Filter Design Fall 2008 1 Lecture 15: Stochastic processes Instructor: Dr. Gleb V. Tcheslavski Contact: gleb@ee.lamar.edu gleb@ee.lamar.edu Office Hours: Room 2030 Class web site: http://ee.lamar.edu/gleb/ dsp/index.htm http://ee.lamar.edu/gleb/ dsp/index.htm

2. ELEN 5346/4304 DSP and Filter Design Fall 2008 2 Why stochastic DSP is important? Most of the real life signals are contaminated by noise that is usually random. Noise contamination may lead to unpredictable changes in the amplitude and phase of the signal. Strong noise contamination may change parameters of the signal considerably. Signals (processes) 1. Deterministic: One possible value for any time instance. Therefore, we can predict the exact value of the signal for a desired time instance. Don’t really exist… 2. Random (stochastic): Many (infinitely) possible values for any time instance. Therefore, we can predict only the expected value of the signal for a desired time instance. Stock market, speech, medical data, communication signals,…

3. ELEN 5346/4304 DSP and Filter Design Fall 2008 3 Definitions and preliminary considerations x n – discrete random process: a sequence of random values. X – ensemble (family) of discrete random processes. realizations The probability theory calls every action or occurrence an event (me standing in front of you, raining outside, student “A” getting an “F” in this class). Every event may or may not happen. A likelihood that the particular event happens is called a probability of an event P{x}. Probability takes on values between zero and one: 0  P{x}  1. Different events have different probabilities: the probability of rain in June in Beaumont is much higher that the probability of hail, which, in turn, is much higher than the probability of snow. Probability indicates how frequently the event may occur. Events, whose probability is one, are called true events. We can consider a discrete signal as a set of statistical events with particular probabilities: random values. (15.3.1) (15.3.2)

4. ELEN 5346/4304 DSP and Filter Design Fall 2008 4 Cumulative distribution function Important measures of probability are a cumulative distribution function (cdf) and a probability density function (pdf). cdf (continuous): cdf (discrete): a step function (15.4.1) (15.4.2) Properties of cdf: (15.4.3) (15.4.4) (15.4.5) (15.4.6)

5. ELEN 5346/4304 DSP and Filter Design Fall 2008 5 Probability density function A pdf is any function f x (x’) that describes the probability density in terms of the input variable x’ in a particular manner. pdf: (15.5.1) Properties of pdf: (15.5.2) (15.5.3) (15.5.4) (15.5.5)

6. ELEN 5346/4304 DSP and Filter Design Fall 2008 6 Probability and pdf (15.6.1) (15.6.2) (15.6.3) (15.6.4) (15.6.5) (15.6.6) (15.6.7)

7. ELEN 5346/4304 DSP and Filter Design Fall 2008 7 Distribution examples Probability distribution functions Discrete distributions Continuous distributions Binomial Poisson Geometric … Uniform Normal (Gaussian) Rayleigh Cauchy Laplace Exponential Gamma Weibull … (15.7.1) (15.7.2) (15.7.3)

8. ELEN 5346/4304 DSP and Filter Design Fall 2008 8 Statistical moments Most frequently are used: 1 st moment: mean (average) Estimated (sample mean): (15.8.5) (15.8.6) k th statistical moment: Expectation operator (15.8.1) Remark: expectation of a true event equals to the event itself: i.e. k th absolute moment: (15.8.3) (15.8.2) k th central moment: (15.8.4)

9. ELEN 5346/4304 DSP and Filter Design Fall 2008 9 Second statistical moments 2 nd moments: Variance: Estimated (sample) variance: Correlation (coeff): Estimated (sample): Covariance(coefficient): Estimated (sample): (15.9.1) (15.9.2) (15.9.3) (15.9.4) (15.9.5) (15.9.6)

10. ELEN 5346/4304 DSP and Filter Design Fall 2008 10 On correlation r x (n 1, n 0 ) is large – x n1 and x n0 are strongly related r x (n 1, n 0 ) is small – x n1 and x n0 are weakly related If correlation is zero: than random variables x n1 and x n0 are orthogonal. If covariance is zero: than random variables x n1 and x n0 are uncorrelated. (15.10.1) (15.10.2) Zero mean  correlation = covariance.

11. ELEN 5346/4304 DSP and Filter Design Fall 2008 11 Stationarity 1. Strict-sense stationarity: A random process x n (i) is strict-sense stationary iff x n (i) and x n+  (i) have the same statistics of all orders (statistical moments of all orders are constant). 2. Weak-sense n th -order stationarity: Statistics up to n th order are constant. 3. Wide-sense stationarity (wss): Mean is constant and autocorrelation is a function of lag but not the location itself (autocorrelation does not change over time: r xx (t 1,t 2 ) = r xx (  )).

12. ELEN 5346/4304 DSP and Filter Design Fall 2008 12 Properties of autocorrelation 1.r xx (  ) it is real 2.It is an even function of lag: r xx (  ) = r xx (-  ) 3.r xx (  ) can be positive or negative 4.r xx (0)  r xx (  ),   0. For wss processes: 1.r xx (0) = E{|x n | 2 } – average power of the SP (mean square value) 2.c xx (  ) = r xx (  ) - |m x | 2  c xx (0) = r xx (0) - |m x | 2 =  x 2 Power Spectral Density (PSD) of a wss process: Which is known as the Wiener–Khintchine or Khinchin–Kolmogorov theorem Note: in a discrete case, the integral is replaced by the infinite sum. (15.12.1) (15.12.2) (15.12.3)