General Classes of Lower Bounds on Outage Error Probability and MSE in Bayesian Parameter Estimation Tirza Routtenberg Dept. of ECE, Ben-Gurion University of the Negev Supervisor: Dr. Joseph Tabrikian

Outline Introduction Derivation of a new class of lower bounds on the probability of outage error Derivation of a new class of lower bounds on the MSE Bounds properties: tightness conditions, relation to the ZZLB Examples Conclusion

Introduction Bayesian parameter estimation Goal: to estimate the unknown parameter θ based on the observation vector x. Assumptions: θ and x are random variables The observation cdf and posterior pdf are known Applications: Radar/Sonar, Communication, Biomedical, Audio/speech,…

Introduction Parameter estimation criteria Mean-square error (MSE) Probability of outage error

Probability of outage error Introduction Parameter estimation criteria Advantages of the probability of outage error criterion: Provides meaningful information in the presence of large errors case. Dominated by the all error distribution. Prediction of the operation region. SNR Large-errors Threshold Small errors MSE Convolutive blind source separation (BSS) aims at separating point sources from mixtures picked up by several sensors. Blind source separation (BSS) refers to the problem of recovering signals from several observed linear mixtures. EEG-nonstationary signals we apply independent component analysis (ICA) to electrocardiographic (ECG) signals for improved detection of abnormal conditions in the heart. Unsupervised ICA neural networks can demix the components of measured ECG signals. Such components may correspond to individual heart functions, either normal or abnormal. ICA neural networks have the potential to make abnormal components more apparent, even when they are masked by normal components in the original measured signals. This is particularly important for diagnosis well in advance of the actual onset of heart attack, in which abnormalities in the original measured ECG signals may be difficult to detect. בעלי קורולציה SNR Large-errors Threshold Small errors Probability of outage error 5

Introduction MMSE estimation The minimum MSE is attained by MMSE:

Introduction h-MAP estimation The h-MAP estimator is The corresponding minimum probability of h-outage error is

Performance lower bounds Motivation Performance analysis Threshold prediction System design Feasibility study Threshold bound PERFORMANCE MEASURE SNR or number of samples

Performance lower bounds Bounds desired features Computational simplicity Tightness Asymptotically coincides with the optimal performance Validity: independent of the estimator. Threshold bound PERFORMANCE MEASURE SNR or number of samples

Previous work: probability of outage error bounds Most of the existing bounds on the probability of outage error are based on the relation to the probability of error in decision procedure (binary/multiple). Kotelnikov inequality - lower bound for uniformly distributed unknown parameter.

Previous work: Bayesian MSE bounds Bayesian Cramér–Rao (Van Trees, 1968) Bayesian Bhattacharyya bound (Van Trees 1968) Weiss–Weinstein (1985) Reuven-Messer (1997) Bobrovski–Zakai (1976) Bayesian MSE bounds Weiss–Weinstein class The covariance inequality Ziv-Zakai class Relation to probability of error in decision problem Ziv–Zakai (ZZLB) (1969) Bellini–Tartara (1974) Chazan–Zakai–Ziv (1975) Extended ZZLB (Bell, Steinberg, Ephraim,Van Trees,1997)

General class of outage error probability lower bounds The probability of outage error ? (Reverse) Hölder inequality for Taking

Objective: obtain valid bounds, independent of . General class of outage error probability lower bounds Objective: obtain valid bounds, independent of .

General class of outage error probability lower bounds Theorem: A necessary and sufficient condition to obtain a valid bound which is independent of the estimator, is that the function is periodic in θ with period h, almost everywhere.

General class of outage error probability lower bounds Using Fourier series representation the general class of bounds is

Example: Linear Gaussian model The model The minimum h-outage error probability: The single coefficient bound:

The tightest subclass of lower bounds The bound is maximized w.r.t. for given p Convergence condition: There exists l0h(θ,x), α>0 such that for all │l│≥│l0h(θ,x)│ This mild condition guaranties that converges for every p≥1.

The tightest subclass of lower bounds Under the convergence condition, the tightest bounds are Repeat for all x and h – sampling period

The tightest subclass of lower bounds Under the convergence condition, the tightest bounds are Properties: The bound exists The bound becomes tighter by decreasing p. For p→1+, the tightest bound is h – sampling period

General class of MSE lower bounds The probability of outage error and MSE are related via: Chebyshev's inequality Known probability identity

General class of MSE lower bounds New MSE lower bounds can be obtained by using and lower bounding the probability of outage error For example: General class of MSE bounds: The tightest MSE bound:

General class of lower bounds on different cost functions Arbitrary cost function C(·) that is non-decreasing and differentiable satisfies Thus, it can be bounded using lower bounds on the probability of outage error Examples: the absolute error, higher moments of the error.

Properties: Relation to the ZZLB Theorem The proposed tightest MSE bound is always tighter than the extended ZZLB. The extended ZZLB is The tightest proposed MSE bound can be rewritten as

For any converging sequence of non-negative numbers Properties: Relation to the ZZLB ZZLB The proposed bound 4 2 8 1 7 4 2 8 1 7 2 2 1 1 max out 4 2 1 7 6 14 For any converging sequence of non-negative numbers Therefore,

Properties: unimodal symmetric pdf Theorem: A. If the posterior pdf f θ| x(θ| x) is unimodal, then the proposed tightest outage error probability bound coincides with the minimum probability of outage error for every h>0. B. If the posterior pdf f θ| x(θ| x) is unimodal and symmetric, then the proposed tightest MSE bound coincides with the minimum MSE.

Example 1 Statistics

Example 2 The model Statistics

Conclusion The concept of probability of outage error criterion is proposed. New classes of lower bounds on the probability of outage error and on the MSE in Bayesian parameter estimation were derived. It is shown that the proposed tightest MSE bound is always tighter than the Ziv-Zakai lower bound. Tightness of the bounds: Probability of outage error- condition: Unimodal posterior pdf. MSE – condition: Unimodal and symmetric posterior pdf.