1. Daisuke Miyazaki Katsushi Ikeuchi The University of Tokyo
Virtual Heritage --- Photometric Aspect Polarization-based Shape Estimation of Transparent Objects for Digitizing Cultural Assets
Daisuke Miyazaki Katsushi Ikeuchi
The University of Tokyo

2. Objective Estimate 3D shape of transparent object
Introduction(1/3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Objective
Estimate 3D shape of transparent object
Analyze the polarization phenomena
Polarization analysis
Virtual transparent object
Real transparent object

3. Application fields Modeling cultural assets 3D catalog in web site
Introduction(2/3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Application fields
Modeling cultural assets
3D catalog in web site
Manufacturing robot
Object recognition to recycle

4. Methods developed in this research
Introduction(3/3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Methods developed in this research
Saito et al. 1999
Miyazaki et al. 2004
Miyazaki et al. 2002
Today’s talk

5. Polarization raytracing(1/7)
Introduction(3)
Polarization raytracing(1/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Polarization
Light = wave  oscillates
Oscillates in certain direction  polarization
DOP = degree of polarization
Partially polarized
(DOP 0~1)
Incident
Reflected
Air
Object
Transmitted
Unpolarized
(DOP 0)
Light
Perfectly polarized
(DOP 1)
Polarizer

6. Reflection and transmission
Introduction(3)
Polarization raytracing(2/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Reflection and transmission
Normal
Depends
upon
Light
Partially polarized
Unpolarized
Air
Object
Partially polarized

7. Polarization raytracing(3/7)
Introduction(3)
Polarization raytracing(3/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Tracing the light rays
Calculate reflection and transmission

8. Polarization raytracing
Introduction(3)
Polarization raytracing(4/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Polarization raytracing
Ray tracing
Ray tracing
Calculate intensity
Calculate polarization
Conventional raytracing
Polarization raytracing

9. Polarization raytracing(5/7)
Introduction(3)
Polarization raytracing(5/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Mueller calculus
Conventional raytracing
Intensity: Scalar
Reflectivity&transmissivity: Scalar
Polarization raytracing
Polarization state: 4D vector
Reflection&transmisstion matrix: 4x4 matrix

10. Example of Mueller calculus
Introduction(3)
Polarization raytracing(6/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Example of Mueller calculus
Conventional raytracing
Reflected intensity reflectivity incident intensity
Scalar
Scalar
Scalar
Polarization raytracing
Reflection vector reflection matrix incidence vector

11. Polarization raytracing(7/7)
Introduction(3)
Polarization raytracing(7/7)
Shape estimation(6)
Experiment(5)
Conclusion(2)
Example of matrices
Reflection
Transmission
Rotation
Phase shift

12. Shape estimation Iterative computation with updating the shape
Introduction(3)
Polarization raytracing(7)
Shape estimation(1/6)
Experiment(5)
Conclusion(2)
Shape estimation
Iterative computation
with updating the shape
Initial shape
Caculated DOP with polarization raytracing 2
min
Input DOP
(degree of polarization)
Final shape

13. Cost function Input Calculated Relationship between normal & height
Introduction(3)
Polarization raytracing(7)
Shape estimation(2/6)
Experiment(5)
Conclusion(2)
Cost function
Input
Calculated
Relationship between
normal & height
min
dxdy
Calculate height and normal

14. Calculate normal from shape
Introduction(3)
Polarization raytracing(7)
Shape estimation(3/6)
Experiment(5)
Conclusion(2)
Calculate normal from shape
Set initial height H properly
Calculate gradient p&q by differentiating height H

15. Polarization raytracing(7)
Introduction(3)
Polarization raytracing(7)
Shape estimation(4/6)
Experiment(5)
Conclusion(2)
Update normal
Input DOP
Light ray
Object
Calculated DOP
Change
normal
Ray changes
Error

16. Calculate height from normal
Introduction(3)
Polarization raytracing(7)
Shape estimation(5/6)
Experiment(5)
Conclusion(2)
Calculate height from normal
Updated normal
Relaxation method
Calculated height

17. Polarization raytracing(7)
Introduction(3)
Polarization raytracing(7)
Shape estimation(6/6)
Experiment(5)
Conclusion(2)
Algorithm overview
Initial height
Normal from
height
Minimize 2
Input
Calc.
Update normal
Output height
is small enough 2
Input
Calc.
Stop when
Height from
normal

18. Experimental setup Monochrome camera Camera adapter Computer
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(1/5)
Conclusion(2)
Experimental setup
Camera adapter
Computer
Monochrome camera
IR/UV cut-off filter
Linear polarizer
Geodesic dome
Polarizer controller
Transparent object inside
40W lamp
Plastic sphere

19. Simulational result Initial 25 loop Initial 20 loop
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(2/5)
Conclusion(2)
Simulational result
Initial
25 loop
Initial
20 loop
Frontal shape(estimated)
Refractive index 1.5
& Illumination (known)
Frontal shape(truth)
Rear shape(known)

20. Polarization raytracing(7)
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(3/5)
Conclusion(2)
Experimental result
Acrylic hemisphere
Refractive index 1.5
Diameter 30mm
Error(height):0.61mm
Error(normal):7.0
Initial
(previous method)
50 loop
Error(height):2.8mm
Error(normal):14
Initial
(previous method)
10 loop
3000 50 25
1500
Error/loop

21. Polarization raytracing(7)
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(4/5)
Conclusion(2)
Experimental result
Initial
5 loop
50 loop
Acrylic object
Diameter(base)24mm
Refractive index 1.5
Error(height)
0.24mm

22. Experimental result Glass(refractive index 1.5) 10 loop
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5/5)
Conclusion(2)
Experimental result
Glass(refractive index 1.5)
10 loop
Initial(previous method)

23. Summary Polarization raytracing Iteration Initial shape
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5)
Conclusion(1/2)
Summary
Polarization raytracing
Iteration
Initial shape
Calculated polarization data 2
min
Input polarization data
Object
Shape

24. Future work Realtime measurement Commercial product
Introduction(3)
Polarization raytracing(7)
Shape estimation(6)
Experiment(5)
Conclusion(2/2)
Future work
Commercial product
Realtime measurement
Estimating refractive index ?
Modeling cultural assets

25. Kamsa hamnida
Supported by Ministry of Education, Culture, Sports, Science and Technology
Special thanks to Robotics and Computer Vision Laboratory, Dept. of EECS, KAIST

26. Daisuke Miyazaki 2005 Creative Commons Attribution 4
Daisuke Miyazaki 2005 Creative Commons Attribution 4.0 International License.
Daisuke Miyazaki, Katsushi Ikeuchi, "Virtual Heritage - Photometric Aspect," in Proceedings of The 1st Joint Workshop of KAIST-RCV and U. Tokyo-Ikeuchi Lab. on Robust Vision Technology, Daejeon, Korea,