1. Polarization-based Shape Estimation of Transparent Objects by Using Raytracing and PLZT Camera Daisuke MiyazakiThe University of Tokyo Noriyuki TakashimaFuruuchi Chemical Corporation Akira YoshidaFuruuchi Chemical Corporation Eiki HarashimaFuruuchi Chemical Corporation Katsushi IkeuchiThe University of Tokyo

2. The 1st part of this talk PLZT polarization camera Measures the polarization state (Stokes vector) of the light Is controllable from the computer Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(11)Introduction(1/2)

3. The 2nd part of this talk Inverse polarization raytracing Estimates the 3D shape of transparent objects Solves the inverse problem of the polarization raytracing Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(11)Introduction(2/2)

4. PLZT polarization camera

5. Mueller calculus Light:4D vector(Stokes vector) Material:4x4 matrix(Mueller matrix) Intensity Power of 0  linear polarized light Power of 45  linear polarized light Power of right circular polarized light DOP (degree of polarization) Introduction(2)Conclusion(2) Algorithm(6)Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(1/12) Intro(1/2)

6. Mueller matrix ND (neutral density) filter W N : alpha value (0~1) Retardation δ: retardation value Horizontal linear polarizer W L : 0~0.5 (ideally 0.5) Rotation Introduction(2)Conclusion(2) Algorithm(6)Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(2/12) Intro(2/2)

7. PLZT Lanthanum-modified lead zirconate titanate Made from 4 kinds of metal compound Pb: lead La: lanthanum Zr: zirconium Ti: titanium Transparent ceramics Birefringent media depending on the voltage Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(3/12) Algorithm(1/6)Intro(2)

8. PLZT and ND filter Camera Target scene (Light, object,...) PLZTND filter x y z +90  Optical axis Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(4/12) Algorithm(2/6)Intro(2)

9. Mueller matrix of the system  : Amount of the phase shift of PLZT (depends on the voltage) System ND filter PLZT rotated 90  RotationRetardation Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(5/12) Algorithm(3/6)Intro(2)

10. PLZT and linear polarizer CameraTarget scene (Light, object,...) PLZTLinear polarizer +22.5  x y z +90  Optical axis Optical axis Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(6/12) Algorithm(4/6)Intro(2)

11. Mueller matrix of the system PLZT rotated 90  Rotation  : Amount of the phase shift of PLZT (depends on the voltage) System Linear polarizer rotated 22.5  Horizontal linear polarizer Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(7/12) Algorithm(5/6)Intro(2)

12. Computing Stokes vector PLZT with ND filter PLZT with linear polarizer [  retarder] PLZT with linear polarizer [1/4 waveplate] PLZT with linear polarizer [1/2 waveplate] Inverse matrix Stokes vector from 4 images Introduction(2)Conclusion(2) Experiment(4) Inverse polarization raytracing(11)PLZT polarization camera(8/12) Algorithm(6/6)Intro(2)

13. Experiment setup Camera Slider ND filter Linear polarizer PLZT unit Band-pass filter UV-cut filter IR-cut filter +x -x +y -y Introduction(2)Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(9/12) Intro(2)Algorithm(6)Experiment(1/4)

14. Experiment result s0s0 s1s1 s2s2 s3s3 DOP Introduction(2)Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(10/12) Intro(2)Algorithm(6)Experiment(2/4)

15. Evaluation DOP of linear polarizer True value = 1.0 Measurement result = 0.72 s 3 /s 0 of left circular polarizer True value = -1.0 Measurement result = -0.25 Introduction(2)Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(11/12) Intro(2)Algorithm(6)Experiment(3/4)

16. Related work Liquid crystal polarization camera Wolff, Mancini, Pouliqen, Andreou (1997) Fujikake, Takizawa, Aida, Kikuchi, Fujii, Kawakita (1998) Harnett, Craighead (2002) Our PLZT polarization camera Obtain whole Stokes parameters PLZT has higher response time than LC Introduction(2)Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(12/12) Intro(2)Algorithm(6)Experiment(4/4)

17. Inverse polarization raytracing

18. Reflection and transmission Normal Unpolarized Air Object Partially polarized Light Depends upon Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(1/11) Intro(1/4)Algorithm(3)Experiment(4)

19. Interreflection [Miyazaki 2004] Reflection only [This method] Reflection & transmission Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(2/11) Intro(2/4)Algorithm(3)Experiment(4)

20. Polarization raytracing Raytracing with polarization Gondek et al. 1994, Wolff & Kurlander 1990, Tannenbaum et al. 1994, Guy & Soler 2004, Chipman 1995, Wilkie 2001 Commercial software We use: raytracing + Mueller calculus Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(3/11) Intro(3/4)Algorithm(3)Experiment(4)

21. Reflection/Transmission matrix Transmission Reflection Fresnel coefficients: Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(4/11) Intro(4/4)Algorithm(3)Experiment(4)

22. Cost function min Calculate height and normal dxdy InputCalculated Relationship between normal & height Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(5/11) Experiment(4)Intro(4)Algorithm(1/3)

23. Update normal Light ray Object Ray changes Change normal Input DOP Calculated DOP Error Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(6/11) Experiment(4)Intro(4)Algorithm(2/3)

24. Algorithm overview Initial height Output height Normal from height Height from normal is small 2 InputCalc. Stop when Minimize Update normal 2 InputCalc. Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(7/11) Experiment(4)Intro(4)Algorithm(3/3)

25. Experimental setup Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(8/11) Intro(4)Algorithm(3)Experiment(1/4)

26. Experimental result Acrylic hemisphere (r=15mm)10 loop Initial (Miyazaki 2004)50 loop Error(height) 2.8mm 0.61mm Error(normal) 14  7.0  Frontal shape(estimated) Frontal shape(truth) Rear shape(known) Refractive index 1.5 & Illumination (known) Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(9/11) Intro(4)Algorithm(3)Experiment(2/4)

27. Experimental result Initial (Miyazaki 2004) 10 loop Glass (n=1.5) Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(10/11) Intro(4)Algorithm(3)Experiment(3/4)

28. Related work Shape estimation of transparent object Murase 1992, Hata et al. 1996, Ohara et al. 2003, Ben-Ezra & Nayar 2003, Kutulakos 2005, Saito et al. 1999, Miyazaki et al. 2002, Miyazaki et al. 2004 Shape from polarization Koshikawa & Shirai 1987, Wolff & Boult 1991, Rahmann 1999, Rahmann 2000, Rahmann & Canterakis 2001, Rahmann 2003, Drbohlav & Sara 2001, Miyazaki et al. 2003 Target is transparentEstimate arbitrary shape [Our method] Introduction(2)Conclusion(2)PLZT polarization camera(12)Inverse polarization raytracing(11/11) Intro(4)Algorithm(3)Experiment(4/4)

29. Conclusion

30. Summary [Inverse polarization raytracing][PLZT polarization camera] PLZTND/LP filter Stokes vector Voltage Shape Iteration min 2 InputCalc. Introduction(2)PLZT polarization camera(12)Inverse polarization raytracing(11)Conclusion(1/2)

31. Future work [Inverse polarization raytracing][PLZT polarization camera] Estimating refractive index ? Realtime measurement Improve the accuracy Introduction(2)PLZT polarization camera(12)Inverse polarization raytracing(11)Conclusion(2/2)

32. © Daisuke Miyazaki 2005 All rights reserved. http://www.cvl.iis.u-tokyo.ac.jp/ Daisuke Miyazaki, Noriyuki Takashima, Akira Yoshida, Eiki Harashima, Katsushi Ikeuchi, "Polarization-based Shape Estimation of Transparent Objects by Using Raytracing and PLZT Camera," in Proceedings of SPIE (Polarization Science and Remote Sensing II, Part of SPIE's International Symposium on Optics and Photonics 2005), Vol. 5888, pp. 1-14, San Diego, CA USA, 2005.8