December 11, 2002MVA20021 Determining Shapes of Transparent Objects from Two Polarization Images Daisuke Miyazaki Masataka Kagesawa Katsushi Ikeuchi The University of Tokyo, Japan

December 11, 2002MVA20022 Modeling transparent objects  Polarization-based vision system  Unambiguous determination of surface normal using geometrical invariant Transparent object VR

December 11, 2002MVA20023 Related works Koshikawa 1979 Koshikawa et al Wolff 1990 Wolff et al Saito et al Miyazaki et al Our method Rahmann et al Need many light sources Not search corresponding points Not solve ambiguity problem Not need camera calibration Not need infrared camera Spherical diffuser DOP Thermal radiation Binocular stereo Optimization method Searching corresponding points

December 11, 2002MVA20024 Outline Rotate the object Target object DOP (Degree Of Polarization) images Region segmentation Search corresponding points 3D model

December 11, 2002MVA20025 Polarization Polarizer Object Air Incident light Reflected light Surface normal Transmitted light  Incident angle Reflection angle DOP(Degree Of Polarization): the ratio of how much the light polarized DOPOrigin Unpolarized light0Sunlight / incandescent light Perfectly polarized light1The light transmitted the polarizer Partially polarized light0~1The light hit the object surface Light source Unpolarized light (DOP 0) Pefectly polarized light (DOP 1) Partially polarized light (DOP 0~1)

December 11, 2002MVA20026 Object Observation Surface normal Light source Polarizer Camera Light source Surface normal P Q PP Incident angle PP Reflection angle QQ Incident angle QQ Reflection angle PP Phase angle QQ

December 11, 2002MVA20027 Ambiguity of phase angle  Determination of phase angle  Propagate the determination from occluding boundary to the inner area (Assume C 2 surface) Intensity I min P 360  1P1P 2P2P Phase angle Azimuth angle   -ambiguity

December 11, 2002MVA20028 Ambiguity of reflection angle  Reflection angle Zenith angle  DOP (Degree Of Polarization)  1 PP 0 1P1P 2P2P Brewster angle BB 90   -ambiguity Determination of reflection angle  Explain in the following slides

December 11, 2002MVA20029 Object rotation  Rotate the object at a small angle  Solve the ambiguity from two DOP images taken from two directions Rotate Camera Object

December 11, 2002MVA Region segmentation Measure DOP of the object DOP image DOP 1: white DOP 0: black Result of region segmentation Divided into 3 regions Region segmentation Divide DOP image with curves of 1 DOP (Brewster angle)

December 11, 2002MVA Gauss’ map N B-E regionB-B region B-N region NNNN or E F B B: Brewster curve N: North pole E: Equator F: Folding curve BBB EEE F

December 11, 2002MVA B-E region & B-N region  B-E region (  B <  <90 o )  Definition: A region enclosed by occluding boundaries  Determine the occluding boundary from background subtraction  B-N region (0 o <  <  B )  Definition: A region where a point of 0 o is included   is 0 o or 90 o when DOP is 0  Assume there is no self-occlusion, so  is 0 o when DOP is 0

December 11, 2002MVA B-B region  B-B region (0 o <  <  B or  B <  <90 o )  Definition: A region which is not the previous two  Apply the following disambiguation method to this region

December 11, 2002MVA Folding curve  A curve (on G) that is a part of the boundary of the region (on G) and is not a Brewster curve (on G) is called a folding curve (on G) Folding curve Brewster curve Equator North pole Gaussian sphere G=Gaussian sphere

December 11, 2002MVA Parabolic curve  Theorem: Folding curve is parabolic curve  Parabolic curve = a curve where Gaussian curvature is 0  Folding curve = geometrical invariant North pole Folding curve Equator Gaussian sphere Object surface Folding curve

December 11, 2002MVA Corresponding point  Corresponding point  folding curve  great circle  arg min DOP, s. t. surface normal // rotation plane Corresponding point Rotate the object North side Corresponding point Rotate the object South side point [= rotation direction]

December 11, 2002MVA Difference of DOP Derivative of DOP DOP + – 1 0 BB 90  0 DOP before rotationDOP after rotationRotation angle Derivative of DOP Compare two DOPs at the pair of corresponding points

December 11, 2002MVA Acquisition system Object Camera Polarizer Light Optical diffuser

December 11, 2002MVA Precision Result of region segmentation Estimated shape Graph of DOP Reflection angle  DOP  Error (Average absolute difference) Error DOP0.17 Reflection angle 8.5  Height2.6mm Plastic transparent hemisphere [diameter 3cm]

December 11, 2002MVA Target object  Photo [Acrylic bell-shaped object]

December 11, 2002MVA DOP images  DOP image when the object is not rotated  DOP image when the object is rotated at a small angle Rotation direction We rotate the object about 8° DOP0:white DOP1:black

December 11, 2002MVA Region segmentation result  Result of region segmentation when the object is not rotated  Result of region segmentation when the object is rotated at a small angle Rotation direction We rotate the object about 8°

December 11, 2002MVA Disambiguation of B-B region Derivative of DOP Positive Negative BB 90  0 Negative Positive Surface normal was  Rotation direction was  Negative

December 11, 2002MVA Rendered image  Shading image

December 11, 2002MVA Rendered image  Photo  Raytracing image

December 11, 2002MVA Error  Comparison of true value and estimated value The diameter(width) of the object is 24mm Error is 0.4mm (Average of the difference of the height) True Estimated True value is made by hand

December 11, 2002MVA Conclusions  A method to measure the surface shape of transparent object based on the analysis of polarization and geometrical characteristics  Determined the surface normal with no ambiguity  Detected a pair of corresponding points of transparent surface  Determined the surface normal of the entire surface at once  Measured a transparent object which is not a hemisphere

December 11, 2002MVA Future works  Higher precision (dealing with interreflections)  Estimation of refractive index  More elegant method for determining phase angle 

December 11, 2002MVA (c) Daisuke Miyazaki 2002 All rights reserved. D. Miyazaki, M. Kagesawa, K. Ikeuchi, "Determining Shapes of Transparent Objects from Two Polarization Images," in Proceedings of IAPR Workshop on Machine Vision Applications, pp.26-31, Nara, Japan,