Partially Coherent Ambiguity Functions for Depth-Variant Point Spread Function Design Roarke Horstmeyer 1 Se Baek Oh 2 Otkrist Gupta 1 Ramesh Raskar 1 PIERS Marrakesh, 3/20/11 1 MIT Media Lab 2 MIT Department of Mechanical Engineering

Conventional camera PSF measurement: Conventional PSF Blur defocus Circular Aperture I(z 0 )I(z 1 )I(z 2 ) z0z0 z1z1 z2z2 f Pt. Source Problem Statement

Design apertures for specific imaging tasks Determine mask Aperture mask: amplitude/phase defocus Desired set of PSFs I(z 0 ), I(z 1 ), I(z 2 ) z0z0 z1z1 z2z2 3 Problem Statement

Examples Defocus PSFs:Depth-InvariantRotatingArbitrary Cubic phase Gauss-Laguerre modes Iterative Design ?

PSF Design Similar to Phase Recovery d z0z0 z1z1 z2z2 Diffractive Element f z0z0 z1z1 z2z2 Aperture Element 3D PSF Design Measurement for phase recovery I(z 0 ), I(z 1 ), I(z 2 ) General Goal: Find (A, ϕ ) from multiple intensities (A,ϕ)(A,ϕ) (A,ϕ)(A,ϕ) Similar to Models: Fresnel propagation, k-space, light fields, phase space

I(x 1,z 1 )I(x 0,z 0 ) U(A, ϕ ) (a) Phase Retrieval iterative I(x n,z n ) I(x 0,z 0 ) … (c) Phase Space Tomography U(A, ϕ ) simultaneous I(x 1,z 1 )I(x 0,z 0 ) (d) Mode Selective (proposed) U(A, ϕ ) simultaneous iterative Overview of 3D Design Techniques I(x 1,z 1 )I(x 0,z 0 ) (b) Transport of Intensity U(A, ϕ ) simultaneous

I(x 1,z 1 )I(x 0,z 0 ) U(A, ϕ ) (a) Phase Retrieval iterative I(x n,z n ) I(x 0,z 0 ) … (c) Phase Space Tomography U(A, ϕ ) simultaneous I(x 1,z 1 )I(x 0,z 0 ) U(A, ϕ ) simultaneous iterative I(x 1,z 1 )I(x 0,z 0 ) U(A, ϕ ) simultaneous Phase space extends nicely to partially coherent design (b) Transport of Intensity Overview of 3D Design Techniques (d) Mode Selective (proposed)

Phase Space Functions Wigner Distribution (WDF) Ambiguity Function (AF) AF “easier” than WDF Tu, Tamura, Phys. Rev. E 55, 1997 OTF(z 1 ) OTF(z 0 ) OTF(-z 1 ) F-slice PSF(z 0 ) PSF(-z 1 ) PSF(z 1 ) WDF Projections: PSFs x u u x' AF Slices: OTFs

z0z0 z1z1 f x U(x) Phase Space Camera Model r Δz z0z0 I xʹxʹ I xʹxʹ z1z1 OTFs Aperture mask

z0z0 z1z1 f x U(x) tan(θ 0 )=0 u xʹxʹ Phase Space Camera Model Why AF is useful: z0z0 I xʹxʹ I xʹxʹ z1z1 tan(θ 1 )=(W 20 k/π) OTFs r Δz 1. Polar display of the OTF z0z0 z1z1 Aperture mask

z0z0 z1z1 f x U(x) tan(θ 0 )=0 u xʹxʹ Phase Space Camera Model Why AF is useful: z0z0 I xʹxʹ I xʹxʹ z1z1 tan(θ 1 )=(W 20 k/π) OTFs r Δz 2. Convert AF to Mutual Intensity: inverse FT, 45° rotation, scale by 2 3. Recovery of U(x) from AF (up to constant Δ ϕ ) z0z0 z1z1 1. Polar display of the OTF

OTF(z 1 ) OTF(z 0 ) PSF OTF Inputs Output: Desired Aperture Mask, 1D (1) AF Population (2) One-time AF Interpolation OTF(z 2 ) θnθn (5) Optimized AF xʹxʹ xʹxʹ u 1 0 xʹxʹ u xʹxʹ u x2x2 x1x1 x1x1 x2x2 (3) Mutual Intensity J Rank Constraint (4) Optimized J Error Check Iterate &

Rank-constraint on Mutual Intensity, J = λ 1 + λ 2 +…+ λ 3 x1x1 x2x2 x1x1 x2x2 Represent J with coherent mode decomposition 1 Coherent, orthogonal modes from singular value decomposition J = UΛV T = Σ λ i U i (x 1 )U i * (x 2 ) Imperfect J guess: Many coherent modes Assume: J symmetric, nxn λ i = Singluar Values U i orthogonal to U j for all i≠j i=1 n J 1 E. Wolf, JOSA 72 (3), 1982

= λ 1 + λ 2 +…+ λ 3 1 st Mode: Coherent x1x1 x2x2 x1x1 x2x2 PSF = response to a point source: restricted to 1 mode Rank-constraint on Mutual Intensity, J J est = λ 1 U 1 (x 1 )U 1 * (x 2 ) J Represent J with coherent mode decomposition 1 Coherent, orthogonal modes from singular value decomposition 1 E. Wolf, JOSA 72 (3), 1982

Ground Truth AFReconstructed AF Computed Phase Mask Ground Truth and Reconstructed OTFs W 20 =0W 20 =λ/2W 20 =λ -π-π +π+π xʹxʹ u xʹxʹ xʹxʹ xʹxʹ xʹxʹ u Reconstruction Example: Cubic Phase Mask Example aperture mask function: exp(jαx 3 ), α=40, 20 iterations

Simulation Experiment Amplitude One (fixed) mask Simple Example: Arbitrary Input Input z 1 =50mm z 2 =50.1mm Rank-1 constraint 25μ – resolution, 1cm 2 binary mask in 50mm f/1.8 Nikkor, 200μ z=4m z 3 =50.2mm 50μ

-π-π +π+π No Constraints on (A, ϕ ) Phase (rad.) x (cm) Amplitude (AU) Amplitude-only constraint Phase-only constraint Constrained Decompositions In Experiment: Amplitude-only or Phase-only required MSE vs. # iterations MSE # of iterations Aperture mask constraints: -Varied performance -Algorithm still converges

Keeping More than One Mode = λ 1 + λ 2 +…+ λ 3 Several Modes: Partially Coherent x1x1 x2x2 x1x1 x2x2 -More accurate estimate found with n > 1 modes J n = 3: (J - Σ λ i U i (x 1 )U i (x 2 )) 2 = global minimum Eckert-Young Thm.: 1 st n-modes of SVD(J) = optimal rank-n estimate SVD = Optimal estimate (L 2 norm, no prior knowledge) i=1 3

Simulating Partial Coherence = λ 1 + λ 2 +…+ λ 3 x1x1 x2x2 x1x1 x2x2 -More accurate estimate found with n > 1 modes Multiple modes can be multiplexed over time 1,2 J Several Modes: Partially Coherent 1 P. De Santis, JOSA 3 (8), 1986, 2 Z. Zhang, private communication, 2011 Spatial Light Modulator: Vary over time CPU

Simulation Example: Benefit of Several Modes Input MSE improvement ~100x (modes contain both A and ϕ ) 12 3 Display 3 Optimal masks z 1 =50mm z 2 =50.1mm z 3 =50.2mm 50μ 1cm 2 masks Rank-3 constraint Experimentally: A, ϕ over time = hard “Weights”:

Adding a Constraint to Several Modes SVD & constrain in separate operations: No convergence = μ 1 + μ 2 +μ3+μ3 x1x1 x2x2 On(J)On(J) x1x1 x2x2 General solution: convex optimization e.g.: amplitude-only, phase-only, spatial 1 and coherence constraints min || J – Σ μ i W i W i * || 2 subject to constraints on W, given n i=1 n 1 Flewett et al., Optics Letters 34 (14) 2010 W 1 =?W 2 =?W 3 =? Operation O n (J) = find closest n rank-1 outer-products, constrained

Example Constraint: Amplitude-only Problem: n optimal coherent modes that are real, positive min || J – WW T || 2 W ≥ 0, real (J = kxk, W=kxn)

Example Constraint: Amplitude-only Problem: n optimal coherent modes that are real, positive min || J – WW T || 2 W ≥ 0, real (J = kxk, W=kxn) Solution: Non-negative matrix factorization 1 -e.g. Netflix challenge: low-rank rep. of 0-5 star movie scores Symmetric NMF: add to update rules (solve for W &H, W≈H) Note: Optimal “Coherent modes” are no longer orthogonal Update Rules: Add line 1 δ=tiny value, error ~2-5% 1. H=W T 2. W=W.*(H T J)./((HH T )H+δ) 3. H=H.*(W T J) T./H(WW)+δ) 1 Lee and Seung, Nature 401, 2001

A Simple Example: Multiple Amplitude Modes cm 2 masks Amplitude-only, 3 masks Amp-only masks: Sym. NMF “Weights”: Simulation Input MSE improvement ~7x (vs. 1 Amp. mode) Buildup of a baseline bias… z 1 =50mm z 2 =50.1mm z 3 =50.2mm 50μ

Conclusion & Future Work -Phase space functions = intuitive window into 3D PSF design -Multiple modes (partially coherent) = increased flexibility -Constrained searches can be achieved w/ convex methods -Amplitude-only: Symmetric NMF -Other constraints: Phase-only (another convex implementation), coherence length (weighted SVD) -Subtracting modes: J=U 1 U 1 * ± U 2 U 2 * ±… (take 2 images) -Current: Initial Experimental tests using an SLM -Next Step: Find a nice application Thanks! Questions?

J pc (x 1,x 2 ) AF pc (x',u) x 1 (cm) x2x2 u.5 3 Coherent Modes 1cm Mask x'(cm -1 ) 5e4-5e x(cm) u x2x2 Partially Coherent Reconstruction: 3 Modes x 1 (cm) x'(cm -1 ) 5e4-5e4 Ground TruthReconstructed

Constraining Several Modes Apply Constraint: Amplitude-only, Phase-only, prior knowledge, etc. = λ 1 + λ 2 +… + λ 3 x1x1 x2x2 Sum=No longer optimal (localized constraints will not converge) individual constraint x1x1 x2x2 x1x1 x2x2 + λ 2 + λ 3 λ1λ1 SVD(J) amplitude-only Example: Amplitude-only mask individual constraint individual constraint

A Simple Example: Prior Knowledge A B C 3 modes hitting unknown structure (A=.4, B=.5, C=.7) + SVD(J) = 2 orthogonal modes x SVD(J) = x2x2 x1x1 Negative values = phase U 1 U 1 * ε (0,.9) U 2 U 2 * ε (-.1,.4)

A Simple Example: Prior Knowledge A B C 3 modes hitting unknown structure (A=.4, B=.5, C=.7) + SNMF(J) SVD(J) = 2 orthogonal modes = x x2x2 x1x1 Symmetric NMF: Assume no phase change - 3 modes>0, more info about structure ++ U 1 U 1 * ε (0,.47) U 2 U 2 * ε (0,.6) U 3 U 3 * ε (0,.6) Negative

Keeping More than One Mode = λ 1 + λ 2 +…+ λ 3 Several Modes: Partially Coherent x1x1 x2x2 x1x1 x2x2 -More accurate estimate with n>1 mutual intensity modes -J est = Σ J i AF est = Σ AF i = Σ L(J i ) 1,2 (L=linear transformation) -If J est more accurate, then AF est more accurate J Multiple Modes can be: a. Multiplexed over time (Desantis, Zheng) b. Could also multiplex over space and/or angle 1 M. Bastiaans, JOSA 3(8) 1986, 2 Lohmann and Rhodes, Appl. Opt. 17, 1978