Radiometric Compensation in a Projector-Camera System Based on the Properties of the Human Visual System Dong WANG, Imari SATO, Takahiro OKABE, and Yoichi SATO June 25, 2005

Introduction desired imagedirectly projected our system our proposed method camera textured screen projector Our goal: Project onto textured surfaces while preserving good photometric quality of input images

Previous Work [Nayar03] Project onto the textured surface Make the output captured by the camera match the desired image system compensated image desired image textured screen Project Capture Match?? modified image Modify Project directly uncompensated

Limitation Physically limited dynamic range of the projector ⇒ Annoying artifacts cutoff modified image compensated image The output of the projector saturates

Key Idea Optimization problem: Minimize annoying artifacts by contrast reduction Maximize the contrast ⇒ Consider the properties of Human Visual System poor contrast cutoff remains

Visual Sensitivity ] [Bolin & Meyer 1998] Human Visual System is not sensitive to: High background illumination levels High spatial frequencies High contrast levels originaluniform sinusoidal straps noised sensitive not sensitive

How to Use the Loss of Visual Sensitivity? Less contrast reduction effort where humans are less sensitive ⇒ How to determine the maximum error that can be tolerated? Input Maximum error that can be tolerated

A Perceptually-Based Physical Error Metric Predicts maximum luminance error that can be tolerated Makes use of threshold sensitivity, contrast sensitivity, and contrast masking LUMINANCE-DEPENDENT PROCESSING SPATIALLY-DEPENDENT PROCESSING TVI CSF Masking threshold map from TVI elevation factor map input image threshold map [Ramasubramanian99 ]

Assumptions Gray-scale images Planar surface ⇒ homography for the geometric mapping of the projector-camera system Lambertian surface No ambient illumination camera textured screen projector

Our Proposed Method Error caused by artifacts Error caused by the degradation of the contrast Total error ⇒ Determine the optimal global scalar α by minimizing the total error α: global scalar for contrast reduction λ: constant parameter

Results desired imagetextured screen compensated output image without contrast compression uncompensated

Results desired image compensated output image without contrast compression compensated output image with contrast compression α = threshold map

Conclusions Contributions  Incorporate the properties of Human Visual System into radiometric compensation  Relax a severe limitation of the radiometric compensation system Physically limited dynamic range of the projector Future Work  Compensate color images  Use spatially varying scalars  Spatial temporal

Thank you

Previous Work I C Offline calibration:  A set of corresponding I and C  Radiometric (pixel value) correspondence between I and C (per pixel) Online radiometric compensation

Previous Work I C Off-line calibration:  A set of corresponding I and C  Radiometric (pixel value) correspondence between I and C (per pixel)

Limitation Limitation: Annoying artifacts  The physically limited dynamic range of the projector desired image textured screen uncompensated compensation image compensated image Radiometric compensation Project directly Project cutoff

Previous Work Off-line calibration:  A set of corresponding I and C  Radiometric (pixel value) correspondence between I and C (per pixel) I C desired imagecompensation imagetextured screen compensated image uncompensated Project

Human Visual System is not very “Precise” ] [Bolin &Meyer 1998] Just Noticeable Differences Loss of sensitivity: High background illumination levels High spatial frequencies High contrast levels Visual Discrimination Model Human Visual System is not very “Precise”