Media Cybernetics Deconvolution Approaches and Challenges Jonathan Girroir

Slides:



Advertisements
Similar presentations
Today’s Topic (Slight ) correction to last week’s lecture
Advertisements

A Crash Course in Radio Astronomy and Interferometry: 4
From Images to Answers A Basic Understanding of Digital Imaging and Analysis.
Nick Beavers Project Manager Deconvolution from Andy Molnar Software Engineer.
Dial in: (319) Access: # Unmute/mute: #6 Deconvolution of Microscopy Images Issues to Consider for Widefield and Confocal Microscopy Jonathan.
Deconvolution of Laser Scanning Confocal Microscope Volumes: Empirical determination of the point spread function Eyal Bar-Kochba ENGN2500: Medical Imaging.
Deconvolution of Laser Scanning Confocal Microscopy Volumes: Empirical determination of the point spread function Eyal Bar-Kochba ENGN2500: Medical Imaging.
Image-Pro Premier Basic Training Course
Today’s Topic: Lec 3 Prep for Labs 1 & 2 3-D imaging—how to get a nice 2D Image when your samples are 3D. (Deconvolution, with point scanning or with Wide-field.
Fluorescence microscopy – Principle and practical consideration Hiro Ohkura.
Optics. Thomas Young ( ) Wave Optics Diffraction & Interference Christiaan Huygens ( )
Image restoration by deconvolution
EE465: Introduction to Digital Image Processing
Supervisor: Prof. Anil Kokaram Co-Supervisor: Dr. David Corrigan Student: Yun feng Wang Deblurring in Atomic Force Microscopy (AFM) images.
Digital Image Processing Chapter 5: Image Restoration.
Index of Refraction is many scatterers Transmitted plus scattered Scattered radiation cancels out All energy ends up in forward direction with phase shift.
FROM IMAGES TO ANSWERS Deconvolution of Widefield and Confocal images Quantatitive and Qualitative Deconvultion, 3D filters and 3D Analysis. Autoquant.
Maurizio Conti, Siemens Molecular Imaging, Knoxville, Tennessee, USA
lecture 2, linear imaging systems Linear Imaging Systems Example: The Pinhole camera Outline  General goals, definitions  Linear Imaging Systems.
Microscope.
Detecting Electrons: CCD vs Film Practical CryoEM Course July 26, 2005 Christopher Booth.
Digital Image Processing Lecture 4 Image Restoration and Reconstruction Second Semester Azad University Islamshar Branch
Live Cell Fluorescence Imaging Presented by: Nicholas Beavers – Media Cybernetics, Inc. Welcome to Sponsored by Starting Soon…
FROM IMAGES TO ANSWERS Portfolio de Productos de Media Cybernetics Olympus Latin America Q
TIRF Total Internal Reflection Fluorescence Microscopy specialized fluorescence microscopy technique specifically images a very thin optical section (50-250nm)
IPC Friedrich-Schiller-Universität Jena 1 Reduction of out of focus light  Excitation light excites fluorescence more or less within the whole sample.
Chapter 7 Case Study 1: Image Deconvolution. Different Types of Image Blur Defocus blur --- Depth of field effects Scene motion --- Objects in the scene.
Deconvolution, Visualization and Analysis Software for Widefield and Confocal Microscopy.
Chapter 3: Image Restoration Introduction. Image restoration methods are used to improve the appearance of an image by applying a restoration process.
Introduction to Deconvolution Image Processing Introduction to Light and Electron Microscopy Neu259 Spring 2006 Spring 2006 James Bouwer UCSD.
1)Adaptive optics: optimization and wavefront sensing 2)Novel microscope enhancements.
Biophotonics lecture 11. January Today: -Correct sampling in microscopy -Deconvolution techniques.
Basics of Deconvolution
From Images to Answers A Basic Understanding of Digital Imaging and Analysis.
Arbitrary and Dynamic Patterning in a Programmable Array Microscope
Molecular Cell Biology Light Microscopy in Cell Biology Cooper Modified from a 2010 lecture by Richard McIntosh, University of Colorado.
FROM IMAGES TO ANSWERS Live Cell Imaging - Practical Issues Silver Spring & San Diego, June 2005.
FROM IMAGES TO ANSWERS Portfolio de Productos de Media Cybernetics Olympus Latin America Q
Physics 114: Lecture 20 2D Data Analysis Dale E. Gary NJIT Physics Department.
Non-linear Filters Non-linear filter (nelineární filtr) –spatial non-linear operator that produces the output image array g(x,y) from the input image array.
1 Correct Sampling. What is SAMPLING? Intensity [a.u.] X [µm] 1.
Use Snell’s law to determine the angle of total internal reflection in the coverslip (without oil). Oil immersion objectives More light (no total internal.
Today Defocus Deconvolution / inverse filters. Defocus.
Designing a Microscopy Experiment Kurt Thorn, PhD Director, Image from Susanne Rafelski, Marshall lab.
Designing a Microscopy Experiment Kurt Thorn, PhD Director, Image from Susanne Rafelski, Marshall lab.
Confocal Microscopy Kurt Thorn NIC.
Introduction to Digital Image Analysis Kurt Thorn NIC.
FROM IMAGES TO ANSWERS Deconvolution of Widefield and Confocal images The growing role of deconvolution NE 2007.
All-sky source search Aim: Look for a fast method to find sources over the whole sky Selection criteria Algorithms Iteration Simulations Energy information.
Microscope.
Lasers and Confocal.
Oil immersion objectives
Digital Image Processing Lecture 10: Image Restoration
Image Deblurring and noise reduction in python
Deconvolution , , Computational Photography
Optical Coherence Tomography
NANO 230 Micro/NanoFabrication
Digital Holographic Microscopy for Quantitative Visualization
What Is Spectral Imaging? An Introduction
Introduction to Digital Image Analysis Part II: Image Analysis
LIGHT MICROSCOPY variations
Solving an estimation problem
Three-Dimensional FRET Reconstruction Microscopy for Analysis of Dynamic Molecular Interactions in Live Cells  Adam D. Hoppe, Spencer L. Shorte, Joel.
Applications of Fourier Analysis in Image Recovery
Image Restoration for Confocal Microscopy: Improving the Limits of Deconvolution, with Application to the Visualization of the Mammalian Hearing Organ 
Modern Observational/Instrumentation Techniques Astronomy 500
Advanced deconvolution techniques and medical radiography
Theoretical Background Challenges and Significance
Deblurring Shaken and Partially Saturated Images
Volume 85, Issue 6, Pages (December 2003)
Presentation transcript:

Media Cybernetics

Deconvolution Approaches and Challenges Jonathan Girroir

Overview 1.Image Formation 2.Deconvolution 3.Blind and Non-Blind Approaches 4.Spherical Aberration 5.Dealing with Noisy Data 6.Summary and Discussion

1. Image Formation The object The image What happened?

1D - Diffraction

2D - Airy Disk

3D - Point Spread Function (PSF) An impulse response.. Characterizes your microscope.

Effects of the PSF An object is a collection of point sources. Mathematical operation called a convolution. The Microscope is a convolution operator. * = ObjectPSFImage

2. Deconvolution Widefield – Single Channel

Multiple wavelengths - Widefield

Confocal Microscopy Confocal PSF

Types of Deconvolution Multiple ways, two categories: “Deblurring” algorithms – No neighbors and nearest neighbors. Subtractive. “Deblurring” algorithms – No neighbors and nearest neighbors. Subtractive. Image restoration algorithms – Inverse filter, non-blind deconvolution, blind deconvolution. Image restoration algorithms – Inverse filter, non-blind deconvolution, blind deconvolution. Trade off between speed and quality

3D Inverse Filter One pass – fast. Limited by noise amplification. Blur removal traded against gain in noise. ImageObjectPSF Imaging : DeblurredImage PSF 1 Inverse Filter:

Improving signal to noise

2D Blind deconvolution

2D Blind deconvolution – TIRF

When to use deconvolution In widefield fluorescent Microscopy Improve contrast of 2D and 3D images Improve contrast of 2D and 3D images Improve resolution in three dimensions Improve resolution in three dimensions Improve 3D rendering and visualization Improve 3D rendering and visualization

When to use deconvolution In confocal microscopy Improve signal to noise ratios Improve signal to noise ratios increase dynamic range increase dynamic range Improve image grayscale resolution (contrast) Improve image grayscale resolution (contrast) Increase axial resolution Increase axial resolution

When to use deconvolution Documentation Images and projections will show more detail and greater contrast Images and projections will show more detail and greater contrast Improvements can be striking Improvements can be striking

When to use deconvolution Analysis Deconvolution retains quantitative accuracy Deconvolution retains quantitative accuracy Improves contrast facilitating analysis Improves contrast facilitating analysisCountingTracking Area measurements Intensity measurements

Why does this work? The microscope is a well understood optical system The behavior of light is well modeled

The Point Spread of a Microscope Compare a widefield PSF to a confocal PSF

Looking forward to Deconvolution YFP Labeled Motor Neurons (Mouse) Laura Baylor, Harvard University Laboratory of Dr. Jeff Lichtman Emissive Wavelength Emissive Wavelength Numerical Aperture Numerical Aperture Spherical Aberration Spherical Aberration

3. Blind and Non-Blind Approaches Non-Blind Fixed PSF. Fixed PSF. PSF normally measured for each objective, and camera. PSF normally measured for each objective, and camera.Blind Adaptive PSF. Adaptive PSF. PSF normally theoretical. PSF normally theoretical. Requires NA, RI, wavelength, xyz spacing. Requires NA, RI, wavelength, xyz spacing.

Non-Blind deconvolution estimates the original object from the measured PSF and the collected data (2D or 3D image). Non-Blind Deconvolution Non Blind Deconvolution

Blind deconvolution separates the original object and the point spread from the collected data (2D or 3D image). Blind deconvolution Blind Deconvolution

What happens during deconvolution? How it’s done (8/8)

Non-Blind Deconvolution Reblurred Initial Object Estimate PSF Original Blurred - UpdateError Improved Object Estimate

Blind deconvolution

Initial Object Estimate Initial Object Estimate Blind Deconvolution PSF Original Blurry Data - Update Error PSF ? ? ? Iteration: 1 - PSF

Initial Object Estimate Initial Object Estimate Blind Deconvolution PSF Original Blurry Data - Update Error Object Estimate Initial Object Estimate ? ? ? Iteration: 1 - Object

Object Estimate Object Estimate Blind Deconvolution PSF Original Blurry Data - Update Error PSF ? ? ? Iteration: 2 - PSF

Blind deconvolution

Blind vs. Non-Blind Non-blind Must maintain library of PSFs. Must maintain library of PSFs. PSFs need to be optimally measured and noise free. PSFs need to be optimally measured and noise free. Faster because only estimating image. Faster because only estimating image.Blind Does not require imaging of beads. Does not require imaging of beads. Can respond to changes in optical model: Specimen, thermal expansion, aberrations and noise. Can respond to changes in optical model: Specimen, thermal expansion, aberrations and noise.

4. Spherical Aberration Changes in refractive index Incorrect cover slip thickness Thick specimens

Results in an asymmetry of the PSF. Adds significantly more blur. Reduces signal. Effects of Spherical Aberration

Match refractive indexes. Use a SA correction collar. Bias the PSF with similar aberration and deconvolve. Bias the PSF and allow it to adapt using blind deconvolution. Correction of Spherical Aberration

Raw Image Max Intensity 2,614

Deconvolution with no SA Max Intensity 15,134

Deconvolution with SA Correction Max Intensity 23,938

5. Dealing with Noisy Data Noise comes from: Photon detection noise. Photon detection noise. Gaussian instrument noise. Gaussian instrument noise. Scatter. Scatter. Noise is an issue in: Live cell imaging / low light. Live cell imaging / low light. Confocal imaging. Confocal imaging. Update step is important.

Gold’s Method Apply a smoothing every few iterations Quickly can deconvolve images Does not work well for noisy data. Either noise is amplified or small features are removed (over smoothing).

Maximum Likelihood Estimation Statistical approach which tests what is the most likely to occur in the specimen using a known noise model. Finds the “most likely” possibility for each portion of the image. Models noise extremely well. Can actually remove noise instead of suppress. MLE is more versatile than Gold’s which just uses a smoothing factor.

6. Summary and Discussion Deconvolution is a powerful tool that can increase resolution, contrast and signal to noise. There are multiple methods available to accommodate a variety imaging needs. Trade offs exist in acquisition and processing.

Media Cybernetics