Assoc. Prof. Dr. Peerapol Yuvapoositanon

Slides:

Advertisements

Similar presentations

Tracking Unknown Dynamics - Combined State and Parameter Estimation Tracking Unknown Dynamics - Combined State and Parameter Estimation Presenters: Hongwei.

Advertisements

Adaptive Filters S.B.Rabet In the Name of GOD Class Presentation For The Course : Custom Implementation of DSP Systems University of Tehran 2010 Pages.

CY3A2 System identification Modelling Elvis Impersonators Fresh evidence that pop stars are more popular dead than alive. The University of Missouri’s.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Newton’s Method Application to LMS Recursive Least Squares Exponentially-Weighted.

Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.

Performance Optimization

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Discrete random variables Probability mass function Distribution function (Secs )

Goals of Adaptive Signal Processing Design algorithms that learn from training data Algorithms must have good properties: attain good solutions, simple.

Comparative survey on non linear filtering methods : the quantization and the particle filtering approaches Afef SELLAMI Chang Young Kim.

EE491D Special Topics in Communications Adaptive Signal Processing Spring 2005 Prof. Anthony Kuh POST 205E Dept. of Elec. Eng. University of Hawaii Phone:

Review of Probability and Random Processes

Adaptive Signal Processing

RLSELE Adaptive Signal Processing 1 Recursive Least-Squares (RLS) Adaptive Filters.

Review of Probability.

Prof. SankarReview of Random Process1 Probability Sample Space (S) –Collection of all possible outcomes of a random experiment Sample Point –Each outcome.

Introduction to Adaptive Digital Filters Algorithms

Statistics for Engineer Week II and Week III: Random Variables and Probability Distribution.

By Asst.Prof.Dr.Thamer M.Jamel Department of Electrical Engineering University of Technology Baghdad – Iraq.

1 Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering EMIS 7370/5370 STAT 5340 : PROBABILITY AND STATISTICS FOR SCIENTISTS AND ENGINEERS Systems.

Kalman Filter (Thu) Joon Shik Kim Computational Models of Intelligence.

CS433: Modeling and Simulation Dr. Anis Koubâa Al-Imam Mohammad bin Saud University 15 October 2010 Lecture 05: Statistical Analysis Tools.

MTH 161: Introduction To Statistics

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.

Least SquaresELE Adaptive Signal Processing 1 Method of Least Squares.

Method of Least Squares. Least Squares Method of Least Squares:  Deterministic approach The inputs u(1), u(2),..., u(N) are applied to the system The.

1 Lecture 7: Discrete Random Variables and their Distributions Devore, Ch

Computer Science, Software Engineering & Robotics Workshop, FGCU, April 27-28, 2012 Fault Prediction with Particle Filters by David Hatfield mentors: Dr.

CHAPTER 4 Adaptive Tapped-delay-line Filters Using the Least Squares Adaptive Filtering.

Unit-V DSP APPLICATIONS. UNIT V -SYLLABUS DSP APPLICATIONS Multirate signal processing: Decimation Interpolation Sampling rate conversion by a rational.

Elements of Stochastic Processes Lecture II

Linear Predictive Analysis 主講人：虞台文. Contents Introduction Basic Principles of Linear Predictive Analysis The Autocorrelation Method The Covariance Method.

LEAST MEAN-SQUARE (LMS) ADAPTIVE FILTERING. Steepest Descent The update rule for SD is where or SD is a deterministic algorithm, in the sense that p and.

Robotics Research Laboratory 1 Chapter 7 Multivariable and Optimal Control.

Random Variable The outcome of an experiment need not be a number, for example, the outcome when a coin is tossed can be 'heads' or 'tails'. However, we.

Computer simulation Sep. 9, QUIZ 2 Determine whether the following experiments have discrete or continuous out comes A fair die is tossed and the.

CHAPTER 10 Widrow-Hoff Learning Ming-Feng Yeh.

Professors: Eng. Diego Barral Eng. Mariano Llamedo Soria Julian Bruno

Overview of Adaptive Filters Quote of the Day When you look at yourself from a universal standpoint, something inside always reminds or informs you that.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Normal Equations The Orthogonality Principle Solution of the Normal Equations.

Discrete-time Random Signals

1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern.

HCI/ComS 575X: Computational Perception Instructor: Alexander Stoytchev

Regression Analysis1. 2 INTRODUCTION TO EMPIRICAL MODELS LEAST SQUARES ESTIMATION OF THE PARAMETERS PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND ESTIMATION.

1 Review of Probability and Random Processes. 2 Importance of Random Processes Random variables and processes talk about quantities and signals which.

Digital Signal Processing

EEEB0765 Digital Signal Processing for Embedded Systems 3 DSP Transforms: FFT and DCT Assoc. Prof. Dr. Peerapol Yuvapoositanon Dept. of Electronic Engineering.

Random Variables By: 1.

Chapter 6 Random Processes

ELG5377 Adaptive Signal Processing Lecture 13: Method of Least Squares.

Thomas F. Edgar (UT-Austin) RLS – Linear Models Virtual Control Book 12/06 Recursive Least Squares Parameter Estimation for Linear Steady State and Dynamic.

Stats 242.3(02) Statistical Theory and Methodology.

ECE3340 Introduction to Stochastic Processes and Numerical Methods

Expectations of Random Variables, Functions of Random Variables

STAT 311 Chapter 1 - Overview and Descriptive Statistics

Optimum Passive Beamforming in Relation to Active-Passive Data Fusion

Outline Introduction Signal, random variable, random process and spectra Analog modulation Analog to digital conversion Digital transmission through baseband.

Department of Civil and Environmental Engineering

Pipelined Adaptive Filters

Equalization in a wideband TDMA system

Instructor :Dr. Aamer Iqbal Bhatti

Random Noise in Seismic Data: Types, Origins, Estimation, and Removal

لجنة الهندسة الكهربائية

Equalization in a wideband TDMA system

Ch2: Adaline and Madaline

Assoc. Prof. Dr. Peerapol Yuvapoositanon

STOCHASTIC HYDROLOGY Random Processes

Learning Theory Reza Shadmehr

Adaptive Filter A digital filter that automatically adjusts its coefficients to adapt input signal via an adaptive algorithm. Applications: Signal enhancement.

Recursively Adapted Radial Basis Function Networks and its Relationship to Resource Allocating Networks and Online Kernel Learning Weifeng Liu, Puskal.

Chapter 6 Random Processes

Presentation transcript:

EECS0712 Adaptive Signal Processing 1 Introduction to Adaptive Signal Processing Assoc. Prof. Dr. Peerapol Yuvapoositanon Dept. of Electronic Engineering CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Course Outline Introduction to Adaptive Signal Processing Adaptive Algorithms Families: Newton’s Method and Steepest Descent Least Mean Squared (LMS) Recursive Least Squares (RLS) Kalman Filtering Applications of Adaptive Signal Processing in Communications and Blind Equalization CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Evaluation Assignment= 20 % Midterm = 30 % Final = 50 % CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Textbooks CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

http://embedsigproc. wordpress http://embedsigproc.wordpress.com /eecs0712-adaptive-signal-processing/ CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

QR code CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Adaptive Signal Processing Definition: Adaptive signal processing is the design of adaptive systems for signal-processing applications. [http://encyclopedia2.thefreedictionary.com/adaptive+signal+processing] CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

System Identification Let’s consider a system called “plant” We need to know its characteristics, i.e., The impulse response of the system CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Plant Comparison CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Error of Plant Outputs CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Error of Estimation Error of estimation is represented by the signal energy of error CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Adaptive System We can do it adaptively CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

One-weight Adjust the weight for minimum error e CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Error Curve Parabola equation CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Partial diff. and set to zero Partial differentiation Set to zero Result: CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Multiple Weight Plants We calculate the weight adaptively Questions: What is the type of signal “x” to be used, e.g. Sine, Cosine or Random signals ? If there is more than one weight w0 , i.e., w0….wN-1, how do we calculate the solution? CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Plants with Multiple Weight If we have multiple weights CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Two-weight In the case of two-weight CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Input From We construct the x as vector with first element is the most recent CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Plants with Multiple Weight (aka “Transversal Filter”) If we have multiple weights CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Regression input signal vector If the current time is n, we have “Regression input signal vector” CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Convolution Output of plant is a convolution Ex For N=2 CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

We can use a vector-matrix multiplication For example, for n=3 we construct y(3) as For example, for n=1 we construct y(1) as CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Let us stop there to consider Random signal theory first. The error squared is Let us stop there to consider Random signal theory first. CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Review of Random Signals CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Wireless Transmissions Ideal signal transmission 1 1 1 1 1 1 1 Information Information is Random CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Random variable CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Random Variable Random variable is a function For a single time Coin Tossing CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Our signal x(n) is a Random Variable For a series of Coin Tossing CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Coin tossing and Random Variable If random We have random variable X CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Random Digital Signal If the random variable is a function of time, it is called a stochastic process CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Probability Mass Function We need also to define the probability of each random variable CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Probability Mass Function PMF is for Discrete distribution function CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Time and Emsemble CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Probability of X(2) CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Probability Density Function PDF is for Continuous Distribution Function CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Probability Density Function PDF values can be > 1 as long as its area under curve is 1 2 1 1/2 1 CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Cumulative Distribution Function CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Expectation Operator CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Expected Value Expected value is known as the “Mean” CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Example of Expected Value (Discrete) We toss a die N times and get a set of outcomes Suppose we roll a die with N=6, we might get CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Example of Expected Value (Discrete) But, empirically we have Empirical (Monte Carlo) estimate as Expected Value CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Theoretical Expected Value But in theory, for a die CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Ensemble Average 1 ensemble i ensembles CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Ensemble Average CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

I) Linearity CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

II) CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

III) CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Autocorrelation CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Autocorrelation n=m CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Autocorrelation Matrix CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Covariance CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Stationarity (I) I) n1 n2 CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Stationarity (II) II) CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Expected Value of Error Energy Let’s take the expected value of error energy CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Vector-Matrix Differentiation CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Partial diff. and set to zero Differentiation Result: CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

2-D Error surface CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Four Basic Classes of Adaptive Signal Processing I) Identification II) Inverse Modelling III) Prediction IV) Interference Cancelling CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

The Four Classes of Adaptive Filtering CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

System Identification CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Inverse Modelling CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Prediction CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

Interference Canceller CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon

What are we looking for in Adaptive Systems? Rate of Convergence Misadjustment Tracking Robustness Computational Complexity Numerical Properties CESdSP EECS0712 Adaptive Signal Processing http://embedsigproc.wordpress.com/eecs0712 Assoc. Prof. Dr. P.Yuvapoositanon