Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition Ex. : estimation of the parameter of an exponential relaxation Signal without noise : Additive Gaussian noise si is a Gaussian RV of mean and and variance s2 Exponential noise ? si is an exponential RV of mean

Signal without noise : Additive Gaussian noise Exponential noise

Solving an estimation problem 4 steps 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB. Additive Gaussian noise: 1/SNR Exponential noise : Independent of alpha !

Solving an estimation problem 4 steps 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) ML estimator Additive Gaussian noise ? = Mean-square estimator = 0 Exponential noise ? = 0 ML estimator

Solving an estimation problem 4 steps 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations. Additive Gaussian noise ? ML estimator ~ unbiased and efficient Exponential noise ? ML estimator ~ unbiased and efficient What happens if ML estimator adapted to additive Gaussian noise is applied to data perturbed with exponential (multiplicative) noise ?

Solving an estimation problem 4 steps 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations. Gaus. data / Gaus. estimator 0.0063 0.1003 0.0064 - Gaus. data / Exp. estimator 0.32 0.55 Exp. data / Exp. estimator 0.0017 0.1001 0.0017 - Exp. data / Gaus. estimator 0.12 0.15

Gaus. data / Gaus. estimator

ML estimator not robust Gaus. data / Exp. estimator ML estimator not robust Gaus. estimator

Exp. data / Exp. estimator

Exp. data / Gaus. estimator Mean-square estimator is not efficient in the present of multiplicative exponential noise Exp. estimator

3.5 CRLB for vectorial parameters

Cramér-Rao lower bound Scalar parameter

3.5 CRLB for vectorial parameters

3.5.2 Example : Linear fit y 1 2 3 … i Question : what is the equation of the line that “best” fits the data ?

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

3.5.2 Example : Linear fit There are 2 parameters : a and b 1. The model of the signal is The noise n is Gaussian, zero mean, variance s2 There are 2 parameters : a and b 2. The loglikelihood :

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

3.5.2 Example : Linear fit The Fisher matrix : The inverse of Fisher matrix : The CRLB for parameters a and b ? CRLB depends on s and N b “more difficult” to estimate than a

The CRLB always increases with the number of estimated parameters 3.5.2 Example : Linear fit The CRLB of a when b is known ? N large : N large : The CRLB of b when a is known ? The CRLB always increases with the number of estimated parameters

b is called a nuisance parameter. The CRLB of a when b is known ? Assume that the parameter of interest is a, but b is unknown. In order to estimate a, the parameter b must also be estimated, although its knowledge is of no interest. b is called a nuisance parameter. NB : there are several ways to deal with a nuisance parameter. Considering it as a parameter to estimate is just one way.

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

3.5.2 Example : Linear fit Loglikelihood: ML estimator ?

3.5.2 Example : Linear fit Bias ? Unbiased Variance ? Efficient These ML estimators are efficient since they are unbiased and the Gaussian noise belongs to the exponential family

Conditions of strict efficiency of the ML estimator

TP 3 and project : PALM microscopy Objective: make an image of an object with an optical microscope, with better resolution than diffraction limit. Diffraction limit ?

Resolution of an optical imaging system Resolution of an imaging system = capacity to separate two points in an image. Assume that the point spread function (PSF) of the imaging system is of width d, and two objects are separated (in the image plane) by distance |x2-x1| PSF Measured intensity x1 x2 x1 x2 x1 x2 Resolution limit : |x1-x2|=d/2 The two objects are « resolved » The two objects are not resolved The minimal resolvable distance is half the dimension of the diffusion function d/2 (Rayleigh criterion). l > 400 nm NA < 1 d/2 > 200 nm In a microscope :

TP 3 and project : PALM microscopy Objective: image an object with a optical microscope, with resolution better than the diffraction limit. 50 nm

TP 3 and project : PALM microscopy Objective: image an object with a optical microscope, with resolution better than the diffraction limit. 50 nm 400 nm

TP 3 and project : PALM microscopy Fluorescence microscopy : inject in the medium fluorophores of nanometric size. They settle on the objects in the scene 50 nm

TP 3 and project : PALM microscopy Fluorescence microscopy : inject in the medium fluorophores of nanometric size. They settle on the objects in the scene When excited by a laser, they emit light which is observed by the microscope 50 nm

TP 3 and project : PALM microscopy But … The problem is not solved ! 50 nm 200 nm

TP 3 and project : PALM microscopy Solution : photo-activable fluorophores They can be in two states : State A : they emit fluorescent light if illuminated with laser at wavelength lA State B : they do not emit light One can « switch » them from state B to state A by illuminating them with laser at wavelength lB After a while, they naturally relax back into state B.

TP 3 and project : PALM microscopy Fluorophores are all in state B 50 nm

TP 3 and project : PALM microscopy Fluorophores are all in state B Illuminate with laser lB 50 nm

TP 3 and project : PALM microscopy Fluorophores are all in state B Illuminate with laser lB Some fluorophores go into state A 50 nm

TP 3 and project : PALM microscopy Switch off laser lB 50 nm

TP 3 and project : PALM microscopy Illuminate with laser lA 50 nm

TP 3 and project : PALM microscopy Only fluorophores in state A emit fluorescent light 50 nm

TP 3 and project : PALM microscopy Observed image

TP 3 and project : PALM microscopy Observed image Since the fluorophores are are no longer superposed, their position can be estimated with a precision much better than the width of the PSF

+ + TP 3 and project : PALM microscopy Observed image Since the fluorophores are are no longer superposed, their position can be estimated with a precision much better than the width of the PSF Estimation precision is only limited by noise + + Estimated positions

TP 3 and project : PALM microscopy The positions of the emitting fluorophores are recorded 50 nm + +

TP 3 and project : PALM microscopy The fluorophores naturally relax in state B and no longer emit light 50 nm + +

TP 3 and project : PALM microscopy The fluorophores naturally relax in state B and no longer emit light 50 nm + +

TP 3 and project : PALM microscopy Illuminate again with laser lB 50 nm + +

TP 3 and project : PALM microscopy Illuminate again with laser lB Some other fluorophores go into state A 50 nm + +

TP 3 and project : PALM microscopy Switch off laser lB Illuminate with laser lA 50 nm + +

TP 3 and project : PALM microscopy Switch off with laser lB Illuminate with laser lA Fluorophores in state A emit fluorescent light 50 nm + +

TP 3 and project : PALM microscopy Estimate the position of emitting fluorophores 50 nm + + + +

TP 3 and project : PALM microscopy Estimate the position of emitting fluorophores 50 nm + + + +

TP 3 and project : PALM microscopy By repeating the process a sufficient number of times, the positions of all fluorophores are measured 50 nm + + + + + + + + + + + + + + + + + + + + + + +

TP 3 and project : PALM microscopy In the final image, the shape of the objects are observed with a resolution related to the estimation precision of position of the fluorophores 50 nm + + + + + + + + + + + + + + + + + + + + + + +

TP 3 and project : PALM microscopy In the final image, the shape of the objects are observed with a resolution related to the estimation precision of position of the fluorophores PALM = Photo-Activated Localization Microscopy 50 nm + + + + + + + + Obtained resolution : ~10 nm + + + + + + + + + + + + + + +

Standard fluorescencemicroscopy Example 1 Image of stress fibers (actin bundles) in a live HeLa cell. Scale bar is 2 μm. Taken from: P. N. Hedde, et. al. Nature Methods 6, 689-690 (2009). PALM Standard fluorescencemicroscopy

Position estimation and sequence of PALM images Example 2 Observation of DNA bundle with PALM microscopy, Sujet de TIPE, PC, 2008 Position estimation and sequence of PALM images Position of fluorophores along DNA bundle Image with standard microscope Final PALM image

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

TP 3 and project : PALM microscopy Continuous signal for 1 fluorophore (in 1D to simplify the problem): with Hypothesis: Gaussian PSF of width w q

TP 3 and project : PALM microscopy Continuous signal for 1 fluorophore (in 1D to simplify the problem): with Measured signal : has to take into account pixel integration q D D

TP 3 and project : PALM microscopy Continuous signal for 1 fluorophore (in 1D to simplify the problem): with Measured signal : has to take into account pixel integration with and bi a white Gaussan noise of variance s2 si q D D

TP 3 and project : PALM microscopy Continuous signal for 1 fluorophore (in 1D to simplify the problem): with Measured signal : has to take into account pixel integration with and bi a white Gaussan noise of variance s2 si OBJECTIVE : estimate q from si D

Solving an estimation problem 4 steps 1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise) -> one needs to take into account the physics of signal acquisition 2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB 3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …) 4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

TP 3 and project : PALM microscopy Expression of the log-likelihood Expression of the CRLB when a is assumed known

Layout of the project TP 3 : Estimation precision of q Algorithm for subpixel estimation of q in a 1D signal Optimization of the magnification (ratio size of PSF / size of pixel) Lecture 4 : How to write a scientific paper ? Example : Libres Savoirs, article_astro.pdf TP 5 : Peak detection in an image « By yourself » : ML algorithm for subpixel estimation of q in an 2D image Report under the form of a scientific paper

Read the text …