Daisuke Miyazaki Katsushi Ikeuchi The University of Tokyo Virtual Heritage --- Photometric Aspect Polarization-based Shape Estimation of Transparent Objects for Digitizing Cultural Assets Daisuke Miyazaki Katsushi Ikeuchi The University of Tokyo

Objective Estimate 3D shape of transparent object Introduction(1/3) Polarization raytracing(7) Shape estimation(6) Experiment(5) Conclusion(2) Objective Estimate 3D shape of transparent object Analyze the polarization phenomena Polarization analysis Virtual transparent object Real transparent object

Application fields Modeling cultural assets 3D catalog in web site Introduction(2/3) Polarization raytracing(7) Shape estimation(6) Experiment(5) Conclusion(2) Application fields Modeling cultural assets 3D catalog in web site Manufacturing robot Object recognition to recycle

Methods developed in this research Introduction(3/3) Polarization raytracing(7) Shape estimation(6) Experiment(5) Conclusion(2) Methods developed in this research Saito et al. 1999 Miyazaki et al. 2004 Miyazaki et al. 2002 Today’s talk

Polarization raytracing(1/7) Introduction(3) Polarization raytracing(1/7) Shape estimation(6) Experiment(5) Conclusion(2) Polarization Light = wave  oscillates Oscillates in certain direction  polarization DOP = degree of polarization Partially polarized (DOP 0~1) Incident Reflected Air Object Transmitted Unpolarized (DOP 0) Light Perfectly polarized (DOP 1) Polarizer

Reflection and transmission Introduction(3) Polarization raytracing(2/7) Shape estimation(6) Experiment(5) Conclusion(2) Reflection and transmission Normal Depends upon Light Partially polarized Unpolarized Air Object Partially polarized

Polarization raytracing(3/7) Introduction(3) Polarization raytracing(3/7) Shape estimation(6) Experiment(5) Conclusion(2) Tracing the light rays Calculate reflection and transmission

Polarization raytracing Introduction(3) Polarization raytracing(4/7) Shape estimation(6) Experiment(5) Conclusion(2) Polarization raytracing Ray tracing Ray tracing Calculate intensity Calculate polarization Conventional raytracing Polarization raytracing

Polarization raytracing(5/7) Introduction(3) Polarization raytracing(5/7) Shape estimation(6) Experiment(5) Conclusion(2) Mueller calculus Conventional raytracing Intensity: Scalar Reflectivity&transmissivity: Scalar Polarization raytracing Polarization state: 4D vector Reflection&transmisstion matrix: 4x4 matrix

Example of Mueller calculus Introduction(3) Polarization raytracing(6/7) Shape estimation(6) Experiment(5) Conclusion(2) Example of Mueller calculus Conventional raytracing Reflected intensity reflectivity incident intensity Scalar Scalar Scalar Polarization raytracing Reflection vector reflection matrix incidence vector

Polarization raytracing(7/7) Introduction(3) Polarization raytracing(7/7) Shape estimation(6) Experiment(5) Conclusion(2) Example of matrices Reflection Transmission Rotation Phase shift

Shape estimation Iterative computation with updating the shape Introduction(3) Polarization raytracing(7) Shape estimation(1/6) Experiment(5) Conclusion(2) Shape estimation Iterative computation with updating the shape Initial shape Caculated DOP with polarization raytracing 2 min Input DOP (degree of polarization) Final shape

Cost function Input Calculated Relationship between normal & height Introduction(3) Polarization raytracing(7) Shape estimation(2/6) Experiment(5) Conclusion(2) Cost function Input Calculated Relationship between normal & height min dxdy Calculate height and normal

Calculate normal from shape Introduction(3) Polarization raytracing(7) Shape estimation(3/6) Experiment(5) Conclusion(2) Calculate normal from shape Set initial height H properly Calculate gradient p&q by differentiating height H

Polarization raytracing(7) Introduction(3) Polarization raytracing(7) Shape estimation(4/6) Experiment(5) Conclusion(2) Update normal Input DOP Light ray Object Calculated DOP Change normal Ray changes Error

Calculate height from normal Introduction(3) Polarization raytracing(7) Shape estimation(5/6) Experiment(5) Conclusion(2) Calculate height from normal Updated normal Relaxation method Calculated height

Polarization raytracing(7) Introduction(3) Polarization raytracing(7) Shape estimation(6/6) Experiment(5) Conclusion(2) Algorithm overview Initial height Normal from height Minimize 2 Input Calc. Update normal Output height is small enough 2 Input Calc. Stop when Height from normal

Experimental setup Monochrome camera Camera adapter Computer Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(1/5) Conclusion(2) Experimental setup Camera adapter Computer Monochrome camera IR/UV cut-off filter Linear polarizer Geodesic dome Polarizer controller Transparent object inside 40W lamp Plastic sphere

Simulational result Initial 25 loop Initial 20 loop Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(2/5) Conclusion(2) Simulational result Initial 25 loop Initial 20 loop Frontal shape(estimated) Refractive index 1.5 & Illumination (known) Frontal shape(truth) Rear shape(known)

Polarization raytracing(7) Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(3/5) Conclusion(2) Experimental result Acrylic hemisphere Refractive index 1.5 Diameter 30mm Error(height):0.61mm Error(normal):7.0 Initial (previous method) 50 loop Error(height):2.8mm Error(normal):14 Initial (previous method) 10 loop 3000 50 25 1500 Error/loop

Polarization raytracing(7) Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(4/5) Conclusion(2) Experimental result Initial 5 loop 50 loop Acrylic object Diameter(base)24mm Refractive index 1.5 Error(height) 0.24mm

Experimental result Glass(refractive index 1.5) 10 loop Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(5/5) Conclusion(2) Experimental result Glass(refractive index 1.5) 10 loop Initial(previous method)

Summary Polarization raytracing Iteration Initial shape Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(5) Conclusion(1/2) Summary Polarization raytracing Iteration Initial shape Calculated polarization data 2 min Input polarization data Object Shape

Future work Realtime measurement Commercial product Introduction(3) Polarization raytracing(7) Shape estimation(6) Experiment(5) Conclusion(2/2) Future work Commercial product Realtime measurement Estimating refractive index ? Modeling cultural assets

Kamsa hamnida Supported by Ministry of Education, Culture, Sports, Science and Technology Special thanks to Robotics and Computer Vision Laboratory, Dept. of EECS, KAIST

Daisuke Miyazaki 2005 Creative Commons Attribution 4 Daisuke Miyazaki 2005 Creative Commons Attribution 4.0 International License. http://www.cvl.iis.u-tokyo.ac.jp/ Daisuke Miyazaki, Katsushi Ikeuchi, "Virtual Heritage - Photometric Aspect," in Proceedings of The 1st Joint Workshop of KAIST-RCV and U. Tokyo-Ikeuchi Lab. on Robust Vision Technology, Daejeon, Korea, 2005.4