Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ch3. Power-spectrum estimation for sensing the environment (1/2) in Cognitive Dynamic Systems, S. Haykin Course: Autonomous Machine Learning Soojeong.

Published byAngela Turner Modified over 6 years ago

Similar presentations

Presentation on theme: "Ch3. Power-spectrum estimation for sensing the environment (1/2) in Cognitive Dynamic Systems, S. Haykin Course: Autonomous Machine Learning Soojeong."β€” Presentation transcript:

1 Ch3. Power-spectrum estimation for sensing the environment (1/2) in Cognitive Dynamic Systems, S. Haykin Course: Autonomous Machine Learning Soojeong Kim Cloud and Mobile Systems Laboratory School of Computer Science and Engineering Seoul National University

2 CONTENTS Part A Part B Part C Part D Part E
Review: Sensing the environment Power spectrum Part B Power spectrum estimation Part C C1. Parametric methods C2. Nonparametric methods what is the meaning of sensing the Part D Multitaper method Part E Multitaper method in space

3 Part A Review : Sensing the environment

4 Ill-posed inverse problem
Part A Ill-posed inverse problem Two views of perception : ill-posed inverse problem, Bayesian inference Ill-posed problem : have lack of well-posed conditions (existence, uniqueness, continuity) Inverse problem : uncover underlying physical laws from the stimuli = find a mapping from stimuli to the state There are two views of perception, one is ill-posed problem and the other is Bayesian inference. Ill-posed problem has lack of well-posed conditions (existence, uniqueness, continuity) Inverse problem is trying to uncover underlying physical laws responsible for generation of the stimuli, it’s kind of finding mapping from stimuli to the state of the environment Perception in a cognitive dynamic system is viewed as an ill-posed inverse problem.

5 Sensing the environment
Part A Sensing the environment Solving an ill-posed inverse problem So, sensing the environment is same as solving an ill-posed inverse problem. For example, in this image it looks like a rabbit and also a duck. Sensing the environment or perception of the environment means finding what is true meaning, whether it is duck or rabbit from your sensory information. Power spectrum, we discussed from now is an example of sensing the environment. Power spectrum estimation is an example of sensing the environment

6 Part B Power spectrum

7 Part B What is power spectrum?
The average power of the incoming signal expressed as a function of frequency Signal is measured by sensors as a time series(time domain) Power spectrum focus on signal power on frequency(frequency domain)

8 Why power spectrum estimation is useful?
Part B Why power spectrum estimation is useful? Impossible to predict future signal Estimate the shape of the signal by only finite set of data Ex : Radar, Radio, Speech Recognition

9 Power spectrum in a theory
Part B Power spectrum in a theory Fourier transform Incoming signals are picked up as a stochastic(random) process Only finite set of data is used(applying theory is impossible) Power spectrum is an ill-posed inverse problem

10 Part C Power spectrum estimation

11 Power spectrum Estimation
Part C Power spectrum Estimation Parametric methods(model-based) Non-parametric methods(model-free)

12 Parametric method example
Part C Parametric method example Build a model 𝐻 𝑒𝑗2πœ‹π‘“ - H is a function to convert time domain signals to frequency domain signals - Apply prior knowledge of Fourier transform(assume 𝑒𝑗2πœ‹π‘“ relationship) - H contains some parameters/coefficients

13 Parametric method example
Part C Parametric method example Controlled input signal - Controlled signal is given -> input signal power on frequency is known value - Model output is only depends on the model, H Controlled signal Model output

14 Parametric method example
Part C Parametric method example Problem : Finding parameters/coefficient of 𝐻 - Finding parameters of H to produce acceptable output - Finding parameters is a typical machine learning problem

15 Parametric method example
Part C Parametric method example After finding good parameters of H - Model is built! , problem is solved - Use the model as an estimator

16 Power spectrum Estimation
Part C Power spectrum Estimation Parametric methods(model-based) If model is good : produce good quality estimates with less sample data If model is bad : mislead conclusion Non-parametric methods(model-free)

17 Nonparametric method(periodogram)
Part C Nonparametric method(periodogram) π‘₯ 𝑛 →𝑋𝑁(𝑓) →𝑆 𝑓 (Fourier transform) 𝑆 𝑓 = lim π‘›β†’βˆž 1 𝑁 𝐸[|𝑋𝑁(𝑓)|] 2 Only finite set of signal data is used in practice Use estimates of 𝑆 𝑓 , 𝑆 𝑓 = 1 𝑁 |𝑋𝑁(𝑓)|2

18 Nonparametric method(periodogram)
Part C Nonparametric method(periodogram) 𝑆 𝑓 = 1 𝑁 |𝑋𝑁(𝑓)|2 Same effect as 𝐷𝑒𝑓𝑖𝑛𝑒. π‘Ž 𝑛 =1 𝑖𝑓 𝑛 𝑖𝑠 𝑖𝑛 π‘π‘œπ‘’π‘›π‘‘ π‘œπ‘“ 𝑁, 0 π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’ π‘₯ 𝑛 Γ—π‘Ž 𝑛 β†’π‘₯βˆ— 𝑛 β†’π‘‹π‘βˆ—(𝑓) β†’ 𝑆 𝑓 (Fourier transform) π‘Ž(𝑛) is taper

19 Nonparametric method(periodogram)
Part C Nonparametric method(periodogram) Not require any model to develop Directly calculate the value using Fourier transform with taper

20 Bias-variance dilemma with taper
Part C Bias-variance dilemma with taper 2 Conditions for good estimator 1. Low bias 2. Low variance(not noisy)

21 Bias-variance dilemma with taper
Part C Bias-variance dilemma with taper Bias from spectral leakage Cause of spectral leakage

22 Bias-variance dilemma with taper
Part C Bias-variance dilemma with taper Spectral leakage↓ by taper(not rectangle) -> bias↓

23 Bias-variance dilemma with taper
Part C Bias-variance dilemma with taper bias↓ -> variance↑ Taper-> reduce effective sample size -> variance ↑ Bias-variance trade-off!

24 Part D Multitaper method

25 Part D Multitaper method
Address the trade-off using multiple tapers instead of one K tapers are used on the same sample data K set of estimates are created Average K set of estimates

26 Tapers are Slepian sequences
Part D Tapers are Slepian sequences Tapers following Slepian sequences π‘˜th taper : π‘˜β†‘β†’π‘π‘–π‘Žπ‘ β†‘, π‘£π‘Žπ‘Ÿπ‘–π‘Žπ‘›π‘π‘’β†“ 𝑆1 𝑓 𝑆7 𝑓

27 Part D Multitaper method All K estimates are averaged
Average of 𝑆1 𝑓 , 𝑆2 𝑓 , 𝑆3 𝑓 ,

28 Part D Multitaper method All K estimates are averaged
Find appropriate K to compromise bias and variance

29 Why multitaper method is good?
Part D Why multitaper method is good? Reduce bias by tapers Reduce variance by increasing effective sample size(N X K) Easy to solve bias-variance trade-off by choosing K

30 Part E Multitaper method in space

31 Multitaper method in space
Part E Multitaper method in space Previous methods consider power of signal on frequency Extended multitaper method to get power of signal on frequency and space

32 Multitaper method in space
Part E Multitaper method in space π‘‹π‘šπ‘˜(𝑓) : Fourier transform of input signal by m-th sensor π‘Žπ‘šπ‘˜: coefficients accounting for different localized area around the gridpoints

33 Multitaper method in space
Part E A Multitaper method in space Apply singular value decomposition to matrix A Power spectrum estimates(Diagonal) 𝐴(𝑓) π‘ˆ Ξ£ π‘‰βˆ— Sensor related(M X M) Taper related(K X K)

34 Thank You!

Download ppt "Ch3. Power-spectrum estimation for sensing the environment (1/2) in Cognitive Dynamic Systems, S. Haykin Course: Autonomous Machine Learning Soojeong."

Similar presentations

Feature Selection as Relevant Information Encoding Naftali Tishby School of Computer Science and Engineering The Hebrew University, Jerusalem, Israel NIPS.

Feature Selection as Relevant Information Encoding Naftali Tishby School of Computer Science and Engineering The Hebrew University, Jerusalem, Israel NIPS.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: The Linear Prediction Model The Autocorrelation Method Levinson and Durbin.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: The Linear Prediction Model The Autocorrelation Method Levinson and Durbin.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Periodograms Bartlett Windows Data Windowing Blackman-Tukey Resources:

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Periodograms Bartlett Windows Data Windowing Blackman-Tukey Resources:
Bayesian models for fMRI data

Bayesian models for fMRI data
Nonlinear rational model identification and control Professor Quan M. Zhu Bristol Institute of Technology University of the West of England Frenchay Campus.

Nonlinear rational model identification and control Professor Quan M. Zhu Bristol Institute of Technology University of the West of England Frenchay Campus.
Hilbert Space Embeddings of Hidden Markov Models Le Song, Byron Boots, Sajid Siddiqi, Geoff Gordon and Alex Smola 1.

Hilbert Space Embeddings of Hidden Markov Models Le Song, Byron Boots, Sajid Siddiqi, Geoff Gordon and Alex Smola 1.
Spectral analysis Kenneth D. Harris 18/2/15. Continuous processes A continuous process defines a probability distribution over the space of possible signals.

Spectral analysis Kenneth D. Harris 18/2/15. Continuous processes A continuous process defines a probability distribution over the space of possible signals.
Chapter 1 Introduction Β§ 1.1 Problem and Analysis Β§ 1.2 Data Engineering Β§ 1.3 Scope Β§ 1.4 Limitations of Course R. J. Chang Department of Mechanical Engineering.

Chapter 1 Introduction Β§ 1.1 Problem and Analysis Β§ 1.2 Data Engineering Β§ 1.3 Scope Β§ 1.4 Limitations of Course R. J. Chang Department of Mechanical Engineering.
Baysian Approaches Kun Guo, PhD Reader in Cognitive Neuroscience School of Psychology University of Lincoln Quantitative Methods 2011.

Baysian Approaches Kun Guo, PhD Reader in Cognitive Neuroscience School of Psychology University of Lincoln Quantitative Methods 2011.
Introduction to Spectral Estimation

Introduction to Spectral Estimation
Goals For This Class Quickly review of the main results from last class Convolution and Cross-correlation Discrete Fourier Analysis: Important Considerations.

Goals For This Class Quickly review of the main results from last class Convolution and Cross-correlation Discrete Fourier Analysis: Important Considerations.
Lecture 1. References In no particular order Modern Digital and Analog Communication Systems, B. P. Lathi, 3 rd edition, 1998 Communication Systems Engineering,

Lecture 1. References In no particular order Modern Digital and Analog Communication Systems, B. P. Lathi, 3 rd edition, 1998 Communication Systems Engineering,
Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.

Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.
1 Part 5 Response of Linear Systems 6.Linear Filtering of a Random Signals 7.Power Spectrum Analysis 8.Linear Estimation and Prediction Filters 9.Mean-Square.

1 Part 5 Response of Linear Systems 6.Linear Filtering of a Random Signals 7.Power Spectrum Analysis 8.Linear Estimation and Prediction Filters 9.Mean-Square.
Time Series Spectral Representation Z(t) = {Z 1, Z 2, Z 3, … Z n } Any mathematical function has a representation in terms of sin and cos functions.

Time Series Spectral Representation Z(t) = {Z 1, Z 2, Z 3, … Z n } Any mathematical function has a representation in terms of sin and cos functions.
Spectral analysis of multiple timeseries Kenneth D. Harris 18/2/15.

Spectral analysis of multiple timeseries Kenneth D. Harris 18/2/15.
Digital Image Processing DIGITIZATION. Summery of previous lecture Digital image processing techniques Application areas of the digital image processing.

Digital Image Processing DIGITIZATION. Summery of previous lecture Digital image processing techniques Application areas of the digital image processing.
CHEE825 Fall 2005J. McLellan1 Spectral Analysis and Input Signal Design.

CHEE825 Fall 2005J. McLellan1 Spectral Analysis and Input Signal Design.
Chapter 6 Spectrum Estimation Β§ 6.1 Time and Frequency Domain Analysis Β§ 6.2 Fourier Transform in Discrete Form Β§ 6.3 Spectrum Estimator Β§ 6.4 Practical.

Chapter 6 Spectrum Estimation Β§ 6.1 Time and Frequency Domain Analysis Β§ 6.2 Fourier Transform in Discrete Form Β§ 6.3 Spectrum Estimator Β§ 6.4 Practical.
Autoregressive (AR) Spectral Estimation

Autoregressive (AR) Spectral Estimation

Similar presentations

Ads by Google