Presentation is loading. Please wait.

Presentation is loading. Please wait.

Friday 23 rd February 2007 Alex 1/70 Alexandre Renaux Washington University in St. Louis Minimal bounds on the Mean Square Error: A Tutorial.

Published byWilfrid Wright Modified over 8 years ago

Similar presentations

Presentation on theme: "Friday 23 rd February 2007 Alex 1/70 Alexandre Renaux Washington University in St. Louis Minimal bounds on the Mean Square Error: A Tutorial."— Presentation transcript:

1 Friday 23 rd February 2007 Alex 1/70 Alexandre Renaux Washington University in St. Louis Minimal bounds on the Mean Square Error: A Tutorial

2 Friday 23 rd February 2007 Alex 2/70 Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and perspectives 4 3 2 1 Minimal bounds on the Mean Square Error

3 Friday 23 rd February 2007 Alex 3/70 Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and perspectives 4 3 2 1

4 Friday 23 rd February 2007 Alex 4/70 Statistical signal processing Extract informations (estimation) Applications: Radar/Sonar Digital communications Medical imaging Astrophysic … Framework and motivations 1

5 Friday 23 rd February 2007 Alex 5/70 Observations space Parameters space Physical process Observations model Estimation rule performances Statistical framework Framework and motivations 1

6 Friday 23 rd February 2007 Alex 6/70 Performances r.v. ’’Distance’’ between and Mean Square Error Variance Bias Estimates distibution Framework and motivations 1

7 Friday 23 rd February 2007 Alex 7/70 Performances: Cramér-Rao inequality For unbiased with Fisher Information Matrix Estimates distribution Cramér-Rao If equality, then efficient estimator Framework and motivations 1

8 Friday 23 rd February 2007 Alex 8/70 Maximum Likelihood estimators The parameters support is finite Direction of Arrivals estimation Frequency estimation Context Framework and motivations 1

9 Friday 23 rd February 2007 Alex 9/70 Asymptotic Non-Information SNR T Signal to Noise Ratio (dB) Mean Square Error (dB) MSE behavior of ML estimator: 3 areas Rife and Boorstyn 1974 Van Trees 1968 Framework and motivations 1

10 Friday 23 rd February 2007 Alex 10/70 Single frequency estimation (100 observations) Normalized frequency Run 1 SNR = 10 dB SNR MSE Framework and motivations 1

11 Friday 23 rd February 2007 Alex 11/70 Single frequency estimation (100 observations) Run 2 SNR = 10 dB Framework and motivations 1 SNR MSE Normalized frequency

12 Friday 23 rd February 2007 Alex 12/70 SNR MSE Run 20 SNR = 10 dB Single frequency estimation (100 observations) Framework and motivations 1 Normalized frequency

13 Friday 23 rd February 2007 Alex 13/70 Run 1 SNR = -6 dB SNR MSE Single frequency estimation (100 observations) Framework and motivations 1 Normalized frequency

14 Friday 23 rd February 2007 Alex 14/70 Normalized frequency SNR MSE Single frequency estimation (100 observations) Framework and motivations 1 Run 2 SNR = -6 dB

15 Friday 23 rd February 2007 Alex 15/70 Outlier Normalized frequency SNR MSE Run 7 SNR = -6 dB Single frequency estimation (100 observations) Framework and motivations 1

16 Friday 23 rd February 2007 Alex 16/70 Outlier Run 1 SNR = -20 dB Normalized frequency SNR MSE Single frequency estimation (100 observations) Framework and motivations 1

17 Friday 23 rd February 2007 Alex 17/70 Outlier SNR MSE Normalized frequency Run 2 SNR = -20 dB Single frequency estimation (100 observations) Framework and motivations 1

18 Friday 23 rd February 2007 Alex 18/70 Outlier Normalized frequency SNR MSE Run 20 SNR = -20 dB Single frequency estimation (100 observations) Framework and motivations 1

19 Friday 23 rd February 2007 Alex 19/70 Asymptotic Non-Information SNR T Signal to Noise Ratio (dB) - Asymptotic MSE - Asymptotic efficiency - Threshold prediction - Global MSE - Ultimate performances Framework and motivations 1 Mean Square Error (dB)

20 Friday 23 rd February 2007 Alex 20/70 Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and prospect 4 3 2 1

21 Friday 23 rd February 2007 Alex 21/70 Minimal bounds on the MSE: unification Asymptotic Non-Information SNR T Signal to Noise Ratio (dB) Mean Square Error (dB) Cramér-Rao bound Insuffisancy of the Cramér- Rao bound - Optimistic - Bias - Threshold Other minimal Bounds (tightest) 2

22 Friday 23 rd February 2007 Alex 22/70 Two categories Determininstic BoundsBayesian Bounds Deterministic parameters Random parameters Bound the local MSEBound the global MSE Minimal bounds on the MSE: unification 2

23 Friday 23 rd February 2007 Alex 23/70 Two categories Determininstic BoundsBayesian Bounds Deterministic parameters Random parameters Bound the local MSEBound the global MSE Minimal bounds on the MSE: unification 2

24 Friday 23 rd February 2007 Alex 24/70 Deterministic bounds unification In a class of unbiased estimator, we want to find the particular estimator for which the variance is minimal at the true value of the parameter Class of unbiased estimator ???? Constrained optimization problem Minimal bounds on the MSE: unification 2 Barankin Approach Glave IEEE IT 1973

25 Friday 23 rd February 2007 Alex 25/70 Bias Barankin (1949) Deterministic bounds unification Minimal bounds on the MSE: unification 2

26 Friday 23 rd February 2007 Alex 26/70 Deterministic bounds unification Minimal bounds on the MSE: unification 2 Needs the resolution of an integral equation Sometimes, doesn’t exist Barankin

27 Friday 23 rd February 2007 Alex 27/70 Bias Cramér-Rao Deterministic bounds unification Minimal bounds on the MSE: unification 2

28 Friday 23 rd February 2007 Alex 28/70 Deterministic bounds unification Minimal bounds on the MSE: unification 2 CramerRaoFisherFrechetDarmois

29 Friday 23 rd February 2007 Alex 29/70 Bias Bhattacharyya (1946) Deterministic bounds unification Minimal bounds on the MSE: unification 2

30 Friday 23 rd February 2007 Alex 30/70 Bias BhattacharyyaBarankin Deterministic bounds unification Minimal bounds on the MSE: unification 2

31 Friday 23 rd February 2007 Alex 31/70 Deterministic bounds unification Minimal bounds on the MSE: unification 2 ? BhattacharyyaGuttmanFraser

32 Friday 23 rd February 2007 Alex 32/70 Bias McAulay-Seidman (1969) Test points (Barankin) Deterministic bounds unification Minimal bounds on the MSE: unification 2

33 Friday 23 rd February 2007 Alex 33/70 Bias McAulay-SeidmanBarankin Test points Deterministic bounds unification Minimal bounds on the MSE: unification 2

34 Friday 23 rd February 2007 Alex 34/70 Deterministic bounds unification Minimal bounds on the MSE: unification How to choose test points ? 2

35 Friday 23 rd February 2007 Alex 35/70 Bias Chapman-Robbins (1951) 1 test point Deterministic bounds unification Minimal bounds on the MSE: unification 2

36 Friday 23 rd February 2007 Alex 36/70 Deterministic bounds unification Minimal bounds on the MSE: unification 2 Chapman Robbins HammersleyKiefer

37 Friday 23 rd February 2007 Alex 37/70 Bias Abel (1993) Deterministic bounds unification Minimal bounds on the MSE: unification Test points 2

38 Friday 23 rd February 2007 Alex 38/70 Deterministic bounds unification Minimal bounds on the MSE: unification 2

39 Friday 23 rd February 2007 Alex 39/70 Bias Quinlan-Chaumette-Larzabal (2006) Test points Deterministic bounds unification Minimal bounds on the MSE: unification 2

40 Friday 23 rd February 2007 Alex 40/70 Deterministic bounds f 0 =0, K=32 observtions Don’t take into accout the support of the parameter Minimal bounds on the MSE: unification 2

41 Friday 23 rd February 2007 Alex 41/70 Already used in Signal Processing Deterministic bounds Minimal bounds on the MSE: unification 2 CRB for wide range of topics ChRB and Barankin (McAulay-Seidman version) Time delay estimation DOA estimation Digital communications (synchronization parameters) Abel bound Digital communications (synchronization parameters)

42 Friday 23 rd February 2007 Alex 42/70 Two categories Determininstic BoundsBayesian Bounds Deterministic parameters Random parameters Bound the local MSEBound the global MSE Minimal bounds on the MSE: unification 2

43 Friday 23 rd February 2007 Alex 43/70 Best Bayesian bound: MSE of the conditional mean estimator (MMSEE) is the solution of Bayesian bounds unification Minimal bounds on the MSE: unification 2

44 Friday 23 rd February 2007 Alex 44/70 For your information Bayesian bounds unification Minimal bounds on the MSE: unification 2

45 Friday 23 rd February 2007 Alex 45/70 Best Bayesian bound Minimal bound Bayesian bounds unification Minimal bounds on the MSE: unification 2

46 Friday 23 rd February 2007 Alex 46/70 Degres of freedom Constrained optimization problem Bayesian bounds unification Minimal bounds on the MSE: unification 2

47 Friday 23 rd February 2007 Alex 47/70 s h 1 Best Bayesian bound Bayesian bounds unification Minimal bounds on the MSE: unification 2

48 Friday 23 rd February 2007 Alex 48/70 s h 1 Bayesian Cramér-Rao bound (Van Trees 1968) Bayesian bounds unification Minimal bounds on the MSE: unification 2

49 Friday 23 rd February 2007 Alex 49/70 Bayesian bounds unification Minimal bounds on the MSE: unification 2 Van Trees

50 Friday 23 rd February 2007 Alex 50/70 s h 1 Reuven-Messer bound (1997) (Bayesian Barankin bound) Bobrovsky-Zakaï bound (1976) (1 test point) Test points … Bayesian bounds unification Minimal bounds on the MSE: unification 2

51 Friday 23 rd February 2007 Alex 51/70 Bayesian bounds unification Minimal bounds on the MSE: unification 2 ReuvenMesserBobrovskyZakai

52 Friday 23 rd February 2007 Alex 52/70 h 1 Bayesian Bhattacharyya bound (Van Trees 1968) Bayesian bounds unification Minimal bounds on the MSE: unification 2

53 Friday 23 rd February 2007 Alex 53/70 Bayesian bounds unification Minimal bounds on the MSE: unification 2 Van Trees

54 Friday 23 rd February 2007 Alex 54/70 s h 1 Weiss-Weinstein bound (1985) Bayesian bounds unification Minimal bounds on the MSE: unification 2

55 Friday 23 rd February 2007 Alex 55/70 Bayesian bounds unification Minimal bounds on the MSE: unification 2 WeissWeinstein

56 Friday 23 rd February 2007 Alex 56/70 Relationship between deterministic and Bayesian bounds Deterministic boundsBayesian bounds Constrained optimization problem Minimal bounds on the MSE: unification 2

57 Friday 23 rd February 2007 Alex 57/70 Deterministic bounds Cramér-Rao Bhattacharyya Chapman-Robbins McAulay-Seidman Abel ??? Bayesian bounds Bayesian Cramér-Rao Bayesian Bhattacharyya Bobrovsky-Zakaï Reuven-Messer ??? Weiss-Weinstein Relationship between deterministic and Bayesian bounds Minimal bounds on the MSE: unification 2

58 Friday 23 rd February 2007 Alex 58/70 Deterministic bounds Cramér-Rao Bhattacharyya Chapman-Robbins McAulay-Seidman Abel ??? Bayesian bounds Bayesian Cramér-Rao Bayesian Bhattacharyya Bobrovsky-Zakaï Reuven-Messer Bayesian Abel Weiss-Weinstein Relationship between deterministic and Bayesian bounds Minimal bounds on the MSE: unification 2

59 Friday 23 rd February 2007 Alex 59/70 Deterministic bounds Cramér-Rao Bhattacharyya Chapman-Robbins McAulay-Seidman Abel Deterministic Weiss-Weinstein Bayesian bounds Bayesian Cramér-Rao Bayesian Bhattacharyya Bobrovsky-Zakaï Reuven-Messer Bayesian Abel Weiss-Weinstein Relationship between deterministic and Bayesian bounds Minimal bounds on the MSE: unification 2

60 Friday 23 rd February 2007 Alex 60/70 Already used in Signal Processing Minimal bounds on the MSE: unification 2 Bayesian bounds BCRB and Bayesian Barankin bound ??? Bobrovsky-Zakai and Bayesian Abel bounds Digital communications (synchronization parameters) Weiss-Weinstein bounds Spectral Analysis Underwater acoustic Array Processing Digital communications Physics (gravitational waves)

61 Friday 23 rd February 2007 Alex 61/70 Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and perspectives 4 3 2 1

62 Friday 23 rd February 2007 Alex 62/70 Synchronization problem (spectral analysis) Observations (complex) Pilot symbols (known) Additive noise, Gaussian, circular, iid Parameter of interest deterministic or random 3 Minimal bounds on the MSE: application

63 Friday 23 rd February 2007 Alex 63/70 QPSK Modulation 20 observations Maximum Likelihood Local MSE Synchronization problem (spectral analysis) 3 Minimal bounds on the MSE: application

64 Friday 23 rd February 2007 Alex 64/70 Bayesian bounds for deterministic parameters estimators The support of the parameter is taken into account Synchronization problem (spectral analysis) 3 Minimal bounds on the MSE: application

65 Friday 23 rd February 2007 Alex 65/70 QPSK Modulation 20 observations Synchronization problem (spectral analysis) 3 Minimal bounds on the MSE: application Maximum Likelihood Global MSE

66 Friday 23 rd February 2007 Alex 66/70 Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE : application Conclusion and perspectives 4 3 2 1

67 Friday 23 rd February 2007 Alex 67/70 Conclusion and perspectives 4 Contributions Deterministic bounds unification Bayesian bounds unification Bayesian Abel bound and deterministic Weiss-Weinstein bound Closed-form expressions of the minimal bounds in a synchronization framework (+ Gaussian observation model with parameterized mean)

68 Friday 23 rd February 2007 Alex 68/70 BTW -Understanding the CRB - CRB for Singular FIM - Ziv-Zakai Family (Bayesian) - Minimal bounds for discret time filtering (Bayesian) - Some global class of Cramér-Rao bound (Bayesian) - Hybrid bounds (mixing non-random and random parameters) Conclusion and perspectives 4

69 Friday 23 rd February 2007 Alex 69/70 Perspectives Estimators set for Bayesian bounds Interpretation in terms of bias of the Weiss-Weinstein bound Closed form expressions for the multiple parameters case Conclusion and perspectives 4

70 Friday 23 rd February 2007 Alex 70/70 Thank you Minimal bounds on the Mean Square Error

Download ppt "Friday 23 rd February 2007 Alex 1/70 Alexandre Renaux Washington University in St. Louis Minimal bounds on the Mean Square Error: A Tutorial."

Similar presentations

Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.

Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
Ordinary Least-Squares

Ordinary Least-Squares
CHAPTER 8 More About Estimation. 8.1 Bayesian Estimation In this chapter we introduce the concepts related to estimation and begin this by considering.

CHAPTER 8 More About Estimation. 8.1 Bayesian Estimation In this chapter we introduce the concepts related to estimation and begin this by considering.
CHAPTER 13 M ODELING C ONSIDERATIONS AND S TATISTICAL I NFORMATION “All models are wrong; some are useful.”  George E. P. Box Organization of chapter.

CHAPTER 13 M ODELING C ONSIDERATIONS AND S TATISTICAL I NFORMATION “All models are wrong; some are useful.”  George E. P. Box Organization of chapter.
Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning
General Classes of Lower Bounds on Outage Error Probability and MSE in Bayesian Parameter Estimation Tirza Routtenberg Dept. of ECE, Ben-Gurion University.

General Classes of Lower Bounds on Outage Error Probability and MSE in Bayesian Parameter Estimation Tirza Routtenberg Dept. of ECE, Ben-Gurion University.
1 SAMPLE COVARIANCE BASED PARAMETER ESTIMATION FOR DIGITAL COMMUNICATIONS Javier Villares Piera Advisor: Gregori Vázquez Grau Signal Processing for Communications.

1 SAMPLE COVARIANCE BASED PARAMETER ESTIMATION FOR DIGITAL COMMUNICATIONS Javier Villares Piera Advisor: Gregori Vázquez Grau Signal Processing for Communications.
1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.

1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.
Estimation  Samples are collected to estimate characteristics of the population of particular interest. Parameter – numerical characteristic of the population.

Estimation  Samples are collected to estimate characteristics of the population of particular interest. Parameter – numerical characteristic of the population.
Fundamental Performance Limits in Image Registration By Dirk Robinson and Peyman Milanfar IEEE Transactions on Image Processing Vol. 13, No. 9, 9/2004.

Fundamental Performance Limits in Image Registration By Dirk Robinson and Peyman Milanfar IEEE Transactions on Image Processing Vol. 13, No. 9, 9/2004.
NEWCOM – SWP2 MEETING 1 Impact of the CSI on the Design of a Multi-Antenna Transmitter with ML Detection Antonio Pascual Iserte Dpt.

NEWCOM – SWP2 MEETING 1 Impact of the CSI on the Design of a Multi-Antenna Transmitter with ML Detection Antonio Pascual Iserte Dpt.
Visual Recognition Tutorial

Visual Recognition Tutorial
Maximum likelihood (ML) and likelihood ratio (LR) test

Maximum likelihood (ML) and likelihood ratio (LR) test
The Mean Square Error (MSE):. Now, Examples: 1) 2)

The Mean Square Error (MSE):. Now, Examples: 1) 2)
Point estimation, interval estimation

Point estimation, interval estimation
Maximum likelihood (ML)

Maximum likelihood (ML)
AGC DSP AGC DSP Professor A G Constantinides© Estimation Theory We seek to determine from a set of data, a set of parameters such that their values would.

AGC DSP AGC DSP Professor A G Constantinides© Estimation Theory We seek to determine from a set of data, a set of parameters such that their values would.
Maximum likelihood (ML) and likelihood ratio (LR) test

Maximum likelihood (ML) and likelihood ratio (LR) test
Presenting: Assaf Tzabari

Presenting: Assaf Tzabari
Parametric Inference.

Parametric Inference.

Similar presentations

Ads by Google