Measuring Bidirectional Texture Reflectance with a Kaleidoscope Jefferson Y. Han & Ken Perlin Media Research Laboratory New York University Slides by Ingrid Montealegre 34:15 total + 30% = ~45 minutes

Introduction Recent work in realistic image synthesis focuses on use of actual data measurements of surfaces and materials Want to capture reflectance properties of a surface (exitant and incident angle of light) High dimensionality makes dense sampling difficult New technique for measuring the appearance of a textured surface as it is viewed or illuminated from different directions

Texture Reflectance Reflectance of a surface characterized by its Bidirectional Reflectance Distribution Function (BRDF) [Nicodemus et al. 1977] BRDF is a 4 dimensional function that describes light from it’s incident and exitant angle BRDF(i,i,e,e) Real objects are not uniform and are not accurately represented by a single BRDF function

Texture Reflectance Dana et al. [1999] introduce Bidirectional Texture Function (BTF) which parameterizes BRDF to allow for spatially varying reflectance BTF accurately captures surface subtleties (including self-occlusion and self-shadowing) It is a large 6D function and is difficult to obtain a dense sample - BTF(u,v,i,i,e,e)

BTF Measurement Seminal work by Dana et al. presents a 3DOF robotic arm that incrementally rotates/tilts a sample in front of a light source Produces 205 total samples w/even distribution requires sample to be affixed to a device, in situ measurements not possible Perlin and Han’s approach measures textured surfaces (BTF) in situ as viewed or illuminated from different directions

BTF using a Kaleidoscope Based on the principle of the kaleidoscope [Brewster 1819] Hollow tube of polygonal cross-section whose inner walls are lined with front surface mirrors Object at end of kaleidoscope appears to multiply into replicated images of itself

View through a Kaleidoscope

BTF using a Kaleidoscope When far end is tapered, sample will look like a faceted virtual sphere since each successive reflection reorients the sample a little further from the perpendicular Analogous to having an entire array of cameras pointing toward a surface Optimal for measuring the BTF

Optical Schematic Optical paths of camera and projector are merged using a 45º beam splitter

Illumination Also an illumination technique When projector is pointed down kaleidoscope, different pixels of the image will arrive at the projected sample after having reflected in different ways, approaching the sample from various directions Different regions of the projected image behave like different light sources By keeping only certain pixels of projected image bright, can choose direction from which to illuminate the sample

Procedure BTF measurement proceeds by taking a sequence of sub-measurements At each sub-measure exactly one region of the illumination image is bright Each region corresponds to a unique sequence of reflections of light off the kaleidoscope walls Sample will be illuminated from a unique sub-range of incoming light directions Complete measurement uses all regions to illuminate the sample

Advantages No moving parts guaranteeing registration of sub-measurements & improved accuracy Can measure in situ Can be use for skin and loose items such as pebbles Requires a single camera, lower cost Richly samples the BTF Initial prototype captured 484 illuminations (comp. 205)

Design Parameters In general, kaleidoscope can be made as a regular polygon of n sides, for n>=3 Not every virtual facet is complete, # of fragmented facets varies with n Image processing is most easily performed on rectangular images, for n!=4, use largest inscribable square Focused on n=3 for simplicity and has largest number of whole facets

Simulations of n

Choice of Taper Angle Large taper angle causes reflections to tilt further away from normal, producing fewer facets that are visible Narrow taper angle causes more visible facets, capturing more facets in a single view results in fewer pixels per view, reducing spatial resolution

Choice of Taper Angle Chose taper angle that tilts from vertical by 9º Provides 4 orders of magnitude of reflections from the horizon, a final effect angle of 76º Can measure 22 complete views, providing 222 = 484 distinct view/illumination pairs

Experimental Setup

Calibration Used a planar 3x3 checkerboard pattern & performed corner detection to id sub-pixel coordinates Used to compute best homography transform that maps each patch to the unit square Each transformation applied to the 22 imaging shots and saved to disk Result is a 22x22 array of images indexed by projector facet and camera facet Correction for lens distortion of the camera needs to be done once using the technique by Zhang [1999]

Full 22x22 BTF Measurements

Illumination Alignment Needed to determine which pixels in the projected image illuminated each kaleidoscopically reflected image of the surface In current work, done manually Image from a video camera peering into kaleidoscope as a guide Ideally done automatically Needs to be done once

2 Different Illumination Angles

BSSTF Trials For surfaces with appreciable sub-surface scattering measure BSSRDF (Bidirectional Scattering Surface Reflectance Distribution Function) Illuminate a small spot on the sample and then measure surrounding region [Jensen et al. 2001] Incrementally moving the spot can measure Bidirectional Scattering Surface Texture Function BSSTF(ui,vi,ue,ve,i,i,e,e) Uses two dimensions for each of four measures Entry point of light on sample Exit point of light on sample Incoming spherical angle Outgoing spherical angle

Two BSSTF Measurements

Future Work Improve image extraction to utilize the data currently ignored Add high dynamic range (HDR) capture capability by taking multiple image captures at varying exposure lengths Adjust lenses and cameras used to make more compact Make a room sized version to capture movement