1. Macroscopic Interferometry
Achuta Kadambi, MIT Media Lab
Joint work with Jamie Schiel and Ramesh Raskar

2. Computer Vision/Graphics

3. Computer Vision/Graphics Real-Life
700 nanometer wavelength
Electric field only shown

4. Microscopic wave phenomena  Macro scenes
Computer Vision/Graphics
Real-Life
700 nanometer wavelength
Microscopic wave phenomena  Macro scenes
Electric field only shown

5. Multi-depth 3D Camera

6. Multi-depth 3D Camera
10 meter scene

7. Multi-depth 3D Camera
10 meter scene
Performance competitive low SNR

8. Previous Work
ICCV15: Uses wave phenomena of polarization to enhance 3D shape
Kadambi et al. ICCV 2015

9. Previous Work
ICCV15: Uses wave phenomena of polarization to enhance 3D shape
Today: Use wave phenomena to create multi-depth 3D
Kadambi et al. ICCV 2015

10. Scene

11. Scene

12. Scene

13. Scene

14. Scene

15. Scene

16. Scene

17. Coherent Source
Scene
Interference graphic from sciencetalenter.dk

18. Interferometry Intentionally superimposing light to get scene clues
Figure from ligo.Caltech.edu

19. Interferometry uses a reference to probe the scene
Reference Signal
Sample
Signal
Phase Delay

20. Tiny Wavelength Good for Microscopic Scenes
Reference Signal
700 nanometer wavelength
Sample
Signal
Phase Delay

21. Tiny Wavelength Good for Microscopic Scenes
Reference Signal
700 nanometer wavelength
Corneal OCT
Sample
Signal
Phase Delay
Corneal OCT from Carl Zeiss Visante ®

22. Tiny Wavelength Good for Microscopic Scenes
Reference Signal
700 nanometer wavelength
Corneal OCT
Sample
Signal
Phase Delay
Corneal OCT from Carl Zeiss Visante ®

23. Smaller the measurement ..
Measure sample deviations less than the size of a proton
LIGO Project to Detect Grav Waves

24. Smaller the measurement .. Bigger the instrument
Measure sample deviations less than the size of a proton
.. But kilometers in size
LIGO Project to Detect Grav Waves

25. Vision goal: small device, large scene

26. Vision goal: small device, large scene
Optical Phase
700 nanometer wavelength

27. Vision goal: small device, large scene
Optical Phase
Electronic Phase
Kinect uses electronic phase
700 nanometer wavelength

28. Terminology
Primal-Domain: Original Frame a Signal is Sampled In
Dual-Domain: Frequency transform with respect to Primal
Phase ToF: existing Kinect-style ToF cameras

29. Phase ToF (Kinect)

30. Phase ToF (Kinect)

31. How the Phase ToF (Kinect) Works
dt = ;
N = 20000;
t = 0:dt:(N-1)*dt;
phi1 = pi/3; phi2 = 0;
s1 = sin(2*pi*1*t + phi1);
s2 = sin(2*pi*1*t+ phi2);
xcFFT = conj( fft(s1) ) .* fft(s2);
[~, fundamental_idx] = max(xcFFT);
phase_difference = angle( xcFFT(fundamental_idx) );

32. Block Diagram of Phase ToF
Phase ToF Refer.
Phase ToF Sample
Primal-domain is \tau
For clarity, all constants removed

33. Block Diagram of Phase ToF
Phase ToF Refer.
Phase ToF Xcorr
Phase ToF Sample
For clarity, all constants removed

34. Block Diagram of Phase ToF
Phase ToF Refer.
Phase ToF Xcorr
Compute FFT wrt.
& Take angle of fundam.
Phase ToF Sample
Primal-domain is \tau
For clarity, all constants removed

35. Problems with Phase ToF
Multipath Interference
Reflections of two or more sine waves come back and mix
Phase Wrapping
Phase is periodic – long-range depths will wrap
Low SNR
Calculating phase requires multiple steps (prop. of errors)

36. Computational ToF Imaging
By count of recent papers, appears to be a hot area in comp. photo
[Godbaz et al. 2008]
Find more details at ICCV 2015 course:
[Heide and Hullin et al. 2013]
[Kadambi et al. 2013]
[Bhandari et al. 2014]
[Jayasuriya et al. 2015]
[Gkioulekas et al. 2015]
[Tsai et al. 2016]
[Su et al. 2016] Wed AM posters
[Shreshtha et al. 2016] next month’s SIGGRAPH

37. Computational ToF Imaging
By count of recent papers, appears to be a hot area in comp. photo
[Godbaz et al. 2008]
Find more details at ICCV 2015 course:
[Heide and Hullin et al. 2013]
[Kadambi et al. 2013]
[Bhandari et al. 2014]
Our Contribution: change primal-domain
[Jayasuriya et al. 2015]
[Gkioulekas et al. 2015]
[Tsai et al. 2016]
[Su et al. 2016] Wed AM posters
[Shreshtha et al. 2016] next month’s SIGGRAPH

38. Changing the Primal Domain
Phase ToF Refer.
Phase ToF Xcorr
Compute FFT wrt.
Phase ToF Sample
Primal-domain is \tau
For clarity, all constants removed

39. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

40. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

41. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

42. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

43. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

44. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

45. Changing the Primal Domain
Phase ToF Xcorr
For clarity, all constants removed

46. Changing the Primal Domain
Phase ToF Xcorr
Frequency ToF Xcorr
For clarity, all constants removed

47. Changing the Primal Domain
Phase ToF Xcorr
Frequency ToF Xcorr
Primal-domain is f_M
Dual-domain corresponds to pathlength.
For clarity, all constants removed

50. Macroscopic Scenes

51. Macroscopic Scenes

52. Macroscopic Scenes

53. Multi-depth 3D Camera
Need only 4 frequencies to recover both reflections

54. Multi-depth 3D Camera
Need only 4 frequencies to recover both reflections

55. Multi-depth 3D Camera
Need only 4 frequencies to recover both reflections

56. Resolving Multipath Interference
Scene

57. How close can the reflections be?

58. How close can the reflections be?

59. Analyzing multipath estimation in low SNR

60. Conclusion Collaborators Jamie Schiel Ramesh Raskar Acknowledgements:
Formalize link between ToF cameras and optical interferometry
Changed primal-domain for multi-depth, low SNR, long-range 3D
Collaborators
Jamie Schiel Ramesh Raskar
Acknowledgements:
Ayush Bhandari Suren Jayasuriya Vage Taamazyan