Slides prepared on 09 September, 2018 Exploiting occlusion to image hidden scenes Christos Thrampoulidis joint work with G. Shulkind, F. Xu, W. Freeman, J H. Shapiro, A. Torralba, F. Wong, and G. Wornell Reference: “Exploiting occlusion in non-line-of-sight active imaging”, C.T. , G. Shulkind, F. Xu, W. Freeman, J H. Shapiro, A. Torralba, F. NC. Wong, and G. W Wornell. IEEE Transactions on Computational Imaging, 2018. Slides prepared on 09 September, 2018

Seeing around the corner Conventional imaging Lens: (physically) maps each scene element x0,i to a distinct pixel value yi on the sensor sensor Seeing around the corner visible (diffuse) wall Applications: defense, search & rescue, robotic vision, autonomous vehicles, …

Challenges Conventional imaging Seeing around the corner Lens: (physically) maps each scene element x0,i to a distinct pixel value yi on the sensor sensor Seeing around the corner visible diffuse wall Challenges Measurements linear but uninformative: 2. very weak signal sensor

Active NLOS imaging Challenge: Weak signal Solution: Probe the scene with an active light source sensor laser Challenge: Uninformative measurements Previous approach: Ultrafast time-of-flight measurements [Velten et al. ’12; Buttafava et al. ‘15; Klein et al. ’16, …]

Active NLOS imaging Challenge: Weak signal Solution: Probe the scene with an active light source laser sensor Challenge: Uninformative measurements Previous approach: Ultrafast time-of-flight information Drawbacks: expensive and sensitive camera extremely fast laser slow: very long-acquisition times The curse of light speed! [Velten et al. ’12; Buttafava et al. ‘15; Klein et al. ’16, …]

Opportunistic NLoS imaging Question: Can we see in a hidden room without ultrafast ToF measurements? What are additional structural features that we can exploit? Our Idea: Opportunistically exploit visible occluders visible occluders visible diffuse wall

Opportunistic NLoS imaging Question: Can we see in a hidden room without ultrafast ToF measurements? Our idea: Opportunistically exploit visible occluders laser

Opportunistic NLoS imaging Question: Can we see in a hidden room without ultrafast ToF measurements? Our idea: Opportunistically exploit visible occluders Occlusions create diversity . 0 0 0 0 0 0 0 0 0 0 0 laser

NLoS in action Computational method: Raw photon counts visible wall hidden pattern Occluder Image structure Computational method: Binomial likelihood (Photon-efficient) [Thrampoulidis,Shulkind,Xu,Freeman,Shapiro,Torralba,Wong,Wornell, IEEE Trans. Comp. Imaging, 2018] [Xu*, Thrampoulidis*,Shulkind*,Shapiro,Torralba,Wong,Wornell, Optics Express, 2018]

More examples

Modeling gains VS ~ 70 PPP ~ 1100 PPP Photons Per Pixel (PPP) 40 min Binomial likelihood VS Gaussian likelihood ~ 70 PPP 2.5 min ~ 1100 PPP 40 min Photons Per Pixel (PPP)