Download presentation
Presentation is loading. Please wait.
Published byPierce Lamb Modified over 9 years ago
1
Active Lighting for Appearance Decomposition Todd Zickler DEAS, Harvard University
2
Appearance Decomposition I = f ( shape, reflectance ) Appearance f -1 ( I ) = ? illumination,
3
Appearance Decomposition Research Overview APPEARANCE CAPTURE COLOR IMAGE FILTERING 3D RECONSTRUCTION PHOTOMETRIC INVARIANTS
4
Appearance Decomposition Getting 3D Shape: Image-based Reconstruction I = f ( shape, reflectance, illumination ) f -1 ( I ) = ?
5
Appearance Decomposition Reflectance: BRDF Bi-directional Reflectance Distribution Function
6
Appearance Decomposition Conventional 3D Reconstruction: Restrictive Assumptions LAMBERTIAN: IDEALLY DIFFUSE
7
Appearance Decomposition Example: Conventional Stereo ASSUMPTION: I l = I r IlIl IrIr
8
Appearance Decomposition Example: Conventional Stereo IlIl IrIr ASSUMPTION: I l = I r
9
Appearance Decomposition Conventional 3D Reconstruction: Restrictive Assumptions Shape from shading [Tsai and Shaw, 1994] Variational Stereo [Faugeras and Keriven, 1998] Space Carving [Kutulakos and Seitz, 1998] Multiple-window stereo [Fusiello et al., 1997]
10
Appearance Decomposition Reflectance: BRDF
11
Appearance Decomposition Reflectance: BRDF
12
Appearance Decomposition Helmholtz Reciprocity [Helmholtz 1925; Minnaert 1941; Nicodemus et al. 1977]
13
Appearance Decomposition Stereo vs. Helmholtz Stereo STEREOHELMHOLTZ STEREO
14
Appearance Decomposition Stereo vs. Helmholtz Stereo STEREOHELMHOLTZ STEREO
15
Appearance Decomposition Stereo vs. Helmholtz Stereo STEREOHELMHOLTZ STEREO
16
Appearance Decomposition Reciprocal Images Specularities “fixed” to surface IlIl IrIr Relation between I l and I r independent of BRDF
17
Appearance Decomposition Reciprocity Constraint ^ n vlvl ^ vrvr ^ p olol oror = vlvl ^ vrvr ^ p olol oror ^ n
18
Appearance Decomposition Reciprocity Constraint ^ n vlvl ^ vrvr ^ p olol oror = vlvl ^ vrvr ^ p olol oror ^ n Arbitrary reflectance Surface normal
19
Appearance Decomposition Reciprocal Acquisition CAMERA LIGHT SOURCE
20
Appearance Decomposition Recovered Normals [Zickler et al. 2002]
21
Appearance Decomposition Recovered Surface [Zickler et al., ECCV 2002]
22
Appearance Decomposition In Practice 1.Arbitrary Reflectance 2.Off-the-shelf components 3.Direct surface normals 4.Images aligned with recovered shape 5.Self-calibrating (coming…)
23
Appearance Decomposition Ongoing Work: Auto-calibration [Zickler et al., CVPR 2003, CVPR 2006,…]
24
Appearance Decomposition Research Overview APPEARANCE CAPTURE COLOR IMAGE FILTERING 3D RECONSTRUCTION PHOTOMETRIC INVARIANTS
25
Appearance Decomposition Reflectance Decomposition DIFFUSE =+ SPECULAR [Phong 1975; Shafer, 1985]
26
Appearance Decomposition Reflectance Decomposition [Shafer, 1985]
27
Appearance Decomposition Reflectance Decomposition: Simplifies the Vision Problem =+ LAMBERTIAN: IDEALLY DIFFUSE =+
28
Appearance Decomposition Reflectance Decomposition: A Difficult Inverse Problem DIFFUSE =+ SPECULAR [Bajscy et al., 1996; Criminisi et al., 2005; Lee and Bajscy, 1992; Lin et al., 2002; Lin and Shum, 2001; Miyazaki et al., 2003; Nayar et al., 1997; Ragheb and Hancock, 2001; Sato and Ikeutchi, 1994; Tan and Ikeutchi, 2005; Wolfe and Boult, 1991,…] =+
29
Appearance Decomposition Known Illuminant: Still Ill-posed G S I RGB B R D?
30
Appearance Decomposition Known Illuminant: Still Ill-posed G S I RGB B R D?
31
Appearance Decomposition Observation: Explicit Decomposition not Required G r2r2 r1r1 S I RGB B R J 1.INVARIANT TO SPECULAR REFLECTIONS 2.BEHAVES ‘LAMBERTIAN’
32
Appearance Decomposition Observation: Explicit Decomposition not Required G r2r2 r1r1 S I RGB B R J || J || [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005]
33
Appearance Decomposition Generalization: Mixed Illumination G r2r2 r1r1 S I RGB B R J r1r1 S1S1 B G R S2S2 J SINGLE ILLUMINANTMIXED ILLUMINATION [Zickler, Mallick, Kriegman, Belhumeur, CVPR 2006]
34
Appearance Decomposition Generalization: Mixed Illumination
35
Appearance Decomposition Example: Binocular Stereo [Algorithm: Boykov, Veksler and Zabih, CVPR 1998] Conventional Grayscale (R+G+B)/3 Specular Invariant, ||J|| (blue illuminant) Specular Invariant, ||J|| (blue & yellow illuminants) One image from input stereo pair Recovered depth
36
Appearance Decomposition Example: Optical Flow [Algorithm: Black and Anandan, 1993] Conventional Grayscale (R-+G+B)/3 Specular Invariant, ||J|| (blue illuminant) Specular Invariant, ||J|| (blue & yellow illuminants) Ground truth flow
37
Appearance Decomposition Example: Photometric Stereo [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005] J behaves ‘Lambertian’ Linear function of surface normal
38
Appearance Decomposition Example: Photometric Stereo [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005] J behaves ‘Lambertian’ Linear function of surface normal
39
Appearance Decomposition Example: Photometric Stereo [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005] J behaves ‘Lambertian’ Linear function of surface normal
40
Appearance Decomposition Example: Photometric Stereo [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005]
41
Appearance Decomposition Example: Photometric Stereo [Mallick, Zickler, Kriegman, Belhumeur, CVPR 2005]
42
Appearance Decomposition Generalized Hue G r2r2 r1r1 S I RGB B R J ψ
43
Appearance Decomposition Example: Material-based Segmentation [Zickler, Mallick, Kriegman, Belhumeur, CVPR 2006] Input image Conventional GrayscaleSpecular Invariant ||J|| Conventional Hue Generalized Hue
44
Appearance Decomposition Active lighting can provide: 1. Precise shape (surface normals) for a broad class of (non-Lambertian) surfaces 2. Specular and/or shading invariance (e.g., optical flow, tracking, segmentation) 3. Minimal hardware requirements Active Lighting for Image-guided Surgery? Endoscopic imagery: 1. Illuminant(s) is/are controlled and known 2. Non-Lambertian surfaces 3. Lack of texture
45
Appearance Decomposition Acknowledgements Satya Mallick, UCSD Peter Belhumeur, Columbia University David Kriegman, UCSD Sebastian Enrique, Columbia University Ravi Ramamoorthi, Columbia University zickler@eecs.harvard.edu http://www.eecs.harvard.edu/~zickler
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.