1. General Classes of Lower Bounds on Outage Error Probability and MSE in Bayesian Parameter Estimation
Tirza Routtenberg
Dept. of ECE, Ben-Gurion University of the Negev
Supervisor: Dr. Joseph Tabrikian

2. Outline
Introduction
Derivation of a new class of lower bounds on the probability of outage error
Derivation of a new class of lower bounds on the MSE
Bounds properties: tightness conditions, relation to the ZZLB
Examples
Conclusion

3. Introduction Bayesian parameter estimation
Goal: to estimate the unknown parameter θ
based on the observation vector x.
Assumptions:
θ and x are random variables
The observation cdf and
posterior pdf are known
Applications:
Radar/Sonar, Communication, Biomedical, Audio/speech,…

4. Introduction Parameter estimation criteria
Mean-square error (MSE)
Probability of outage error

5. Probability of outage error
Introduction Parameter estimation criteria
Advantages of the probability of outage error criterion:
Provides meaningful information in the presence of large errors case.
Dominated by the all error distribution.
Prediction of the operation region.
SNR
Large-errors
Threshold
Small errors
MSE
Convolutive blind source separation (BSS) aims at separating point sources from mixtures picked up by several sensors.
Blind source separation (BSS) refers to the problem of recovering signals from several observed linear mixtures.
EEG-nonstationary signals
we apply independent component analysis (ICA) to electrocardiographic (ECG) signals for improved detection
of abnormal conditions in the heart. Unsupervised ICA neural networks can demix the components of measured ECG
signals. Such components may correspond to individual heart functions, either normal or abnormal. ICA neural networks
have the potential to make abnormal components more apparent, even when they are masked by normal components in the
original measured signals. This is particularly important for diagnosis well in advance of the actual onset of heart attack, in
which abnormalities in the original measured ECG signals may be difficult to detect.
בעלי קורולציה
SNR
Large-errors
Threshold
Small errors
Probability of outage error 5

6. Introduction MMSE estimation
The minimum MSE is attained by MMSE:

7. Introduction h-MAP estimation
The h-MAP estimator is
The corresponding minimum probability of h-outage error is

8. Performance lower bounds
Motivation
Performance analysis
Threshold prediction
System design
Feasibility study
Threshold
bound
PERFORMANCE MEASURE
SNR or number of samples

9. Performance lower bounds
Bounds desired features
Computational simplicity
Tightness
Asymptotically coincides with the optimal performance
Validity: independent of the estimator.
Threshold
bound
PERFORMANCE MEASURE
SNR or number of samples

10. Previous work: probability of outage error bounds
Most of the existing bounds on the probability of outage error are based on the relation to the probability of error in decision procedure (binary/multiple).
Kotelnikov inequality - lower bound for uniformly distributed unknown parameter.

11. Previous work: Bayesian MSE bounds
Bayesian Cramér–Rao (Van Trees, 1968)
Bayesian Bhattacharyya bound
(Van Trees 1968)
Weiss–Weinstein (1985)
Reuven-Messer (1997)
Bobrovski–Zakai (1976)
Bayesian MSE bounds
Weiss–Weinstein class
The covariance inequality
Ziv-Zakai class
Relation to probability of error
in decision problem
Ziv–Zakai (ZZLB) (1969)
Bellini–Tartara (1974)
Chazan–Zakai–Ziv (1975)
Extended ZZLB (Bell, Steinberg,
Ephraim,Van Trees,1997)

12. General class of outage error probability lower bounds
The probability of outage error ?
(Reverse) Hölder inequality for
Taking

13. Objective: obtain valid bounds, independent of .
General class of outage error probability lower bounds
Objective: obtain valid bounds, independent of .

14. General class of outage error probability lower bounds
Theorem:
A necessary and sufficient condition to obtain a valid bound which is independent of the estimator, is that the function
is periodic in θ with period h, almost everywhere.

15. General class of outage error probability lower bounds
Using Fourier series representation
the general class of bounds is

16. Example: Linear Gaussian model
The model
The minimum h-outage error probability:
The single coefficient bound:

17. The tightest subclass of lower bounds
The bound is maximized w.r.t for given p
Convergence condition:
There exists l0h(θ,x), α>0 such that for all │l│≥│l0h(θ,x)│
This mild condition guaranties that
converges for every p≥1.

18. The tightest subclass of lower bounds
Under the convergence condition, the tightest bounds are
Repeat for all x
and
h – sampling period

19. The tightest subclass of lower bounds
Under the convergence condition, the tightest bounds are
Properties:
The bound exists
The bound becomes tighter by decreasing p.
For p→1+, the tightest bound is
h – sampling period

20. General class of MSE lower bounds
The probability of outage error and MSE are related via:
Chebyshev's inequality
Known probability identity

21. General class of MSE lower bounds
New MSE lower bounds can be obtained by using
and lower bounding the probability of outage error
For example:
General class of MSE bounds:
The tightest MSE bound:

22. General class of lower bounds on different cost functions
Arbitrary cost function C(·) that is non-decreasing and differentiable satisfies
Thus, it can be bounded using lower bounds on the probability of outage error
Examples: the absolute error, higher moments of the error.

23. Properties: Relation to the ZZLB
Theorem
The proposed tightest MSE bound is always tighter than the extended ZZLB.
The extended ZZLB is
The tightest proposed MSE bound can be rewritten as

24. For any converging sequence of non-negative numbers
Properties: Relation to the ZZLB
ZZLB
The proposed bound 4 2 8 1 7 4 2 8 1 7 2 2 1 1
max out 4 2 1 7 6 14
For any converging sequence of non-negative numbers
Therefore,

25. Properties: unimodal symmetric pdf
Theorem:
A. If the posterior pdf f θ| x(θ| x) is unimodal, then the proposed tightest outage error probability bound coincides with the minimum probability of outage error for every h>0.
B. If the posterior pdf f θ| x(θ| x) is unimodal and symmetric, then the proposed tightest MSE bound coincides with the minimum MSE.

26. Example 1
Statistics

27. Example 2
The model
Statistics

28. Conclusion
The concept of probability of outage error criterion is proposed.
New classes of lower bounds on the probability of outage error and on the MSE in Bayesian parameter estimation were derived.
It is shown that the proposed tightest MSE bound is always tighter than the Ziv-Zakai lower bound.
Tightness of the bounds:
Probability of outage error- condition: Unimodal posterior pdf.
MSE – condition: Unimodal and symmetric posterior pdf.