CPSC 6040 Computer Graphics Images Slides prepared by Ioannis Karamouzas ioannis@clemson.edu With modifications by Donald House

Tone Mapping & Bilateral Filtering Slide Credits: Frédo Durand Josh Levine Ioannis Karamouzas

The Dynamic Range Problem Two Challenges Recording the information Choosing which 6-12 bits of the world to include in your photograph (cell phone to professional SLR, respectively) Metering the world to help you make this decision, since the world has more dynamic range than any light meter Displaying the information Compressing 12 bits into 4 bits for print, or 10 for LCD

Challenge 1: Recording the Information The range of illumination levels is 10 to 12 orders of magnitude Negatives or sensors can record only 2 to 3 orders of magnitude A solution: multiple exposures with different light sensitivities

The Radiance Map

Problem 2: Displaying the Information How should we map scene radiances (up to 1:100,000) to display radiances (only around 1:100) to produce a satisfactory image? Goal: match limited contrast of the display medium while preserving details Solution: Tone Mapping

Simple Tone Mapping

Gamma Compression on Intensity Colors ok, but details in intensity are blurry

Gamma Compression on Intensity Colors ok, but details in intensity are blurry Solution: Reduce contrast of low frequencies Keep mid and high frequencies Mid-high frequency Low frequency

Tone Mapping Using Linear Filtering Reduce contrast of low-frequencies Keep mid and high frequencies

The Halo Nightmare For strong edges Because they contain high frequencies

Tone Mapping Using Bilateral Filter to eliminate the halo effect

Durand and Dorsey, 2002 Do not blur across edges Non-linear filtering

Framework Overview IN LOG Detail = input log – large scale

Bilateral Filtering (slides based on Durand)

Start with Gaussian filtering Input (I) is a step function + noise Output (J) is a convolution of I with a Gaussian kernel f

Gaussian filter as weighted average Weight of ξ depends on the distance to x

The problem of edges I(ξ) is too different from I(x) Pollutes our estimate of J(x)

Principle of bilateral filtering Reject (in a soft way) pixels that differ too much Add a Gaussian g that penalizes intensity differences

Bilateral filtering Combines Spatial Gaussian f (domain kernel) ⅇxp − 𝑥−𝜉 2 2 𝜎 2

Bilateral filtering Combines Spatial Gaussian f (domain kernel) ⅇxp − 𝐼(𝑥)−I(𝜉) 2 2 𝜎 2 Combines Spatial Gaussian f (domain kernel) Gaussian g on intensity differences (range kernel)

Normalization factor

Bilateral filtering is non linear The weights are different for each output pixel

See [Tomasi and Manduchi,1998] for more Putting it all together See [Tomasi and Manduchi,1998] for more

Contrast Reduction Input HDR Image Contrast too high!

Contrast Reduction Intensity = 0.30R + 0.59G + 0.11B R’ = R/intensity Input HDR Image Intensity Intensity = 0.30R + 0.59G + 0.11B Color R’ = R/intensity G’ = G/intensity B’ = B/intensity important to use ratios (makes color luminance invariant)

Contrast Reduction Bilateral filter in log Input HDR Image Intensity Large Scale Bilateral filter in log Color spatial sigma = 2% - 5% image size range sigma = 0.4 (in log 10)

Contrast Reduction Bilateral filter in log Detail = Input HDR Image Intensity Large Scale Bilateral filter in log Detail Color Detail = log intensity - large scale

Contrast Reduction Bilateral filter in log Input HDR Image Intensity Large Scale Large Scale Bilateral filter in log Detail newLog = a oldLog Color

Contrast Reduction Bilateral filter in log Input HDR Image Intensity Large Scale Large Scale Bilateral filter in log Detail Detail Preserve! Color

Contrast Reduction Bilateral filter in log Input HDR Image Output Intensity Large Scale Large Scale Bilateral filter in log Detail Detail Preserve! Color Color

Contrast Reduction in log inLog = log10Intensity inLogLarge = bilateralFilter(inLog) inLogDetail = inLog – inLogLarge Manipulate large scale and detail separately!

Contrast Reduction in log outLog = k(inLogLarge – max(inLogLarge)) Normalize so that 0 is the biggest value in log Set target large scale contrast - targetRange = log10100, if we want a target contrast of 1:100 Compute range of input layer - largeRange = max(inLogLarge) – min(inLogLarge) Compute scale factor: k = targetRange / largeRange

Contrast Reduction in log outLog = detailAmp*inLogDetail + k(inLogLarge – max(inLogLarge)) Normalize so that 0 is the biggest value in log Set target large-scale contrast - targetRange = log10100, if we want a target contrast of 1:100 Compute range of input layer - largeRange = max(inLogLarge) – min(inLogLarge) Compute scale factor: k = targetRange / largeRange We can boost details by detailAmp

Final Output Image outIntensity = 10outLog R’, G’, B’ are intensity invariant color channels - outR = R’*outIntensity - outG = G’*outIntensity - outB = B’*outIntensity

Recap Bilateral filter in log Detail = log intensity - large scale Input HDR Image Output Intensity Large Scale Large Scale Bilateral filter in log Detail Detail Color Detail = log intensity - large scale Color

Results

Important notes Spatial sigma not very important Range sigma: quite important Critical to work in the log domain Because HDR and because perception sensitive to multiplicative contrast Luminance computation (0.30R + 0.59G + 0.11B) See [Eisemann 2004] for smarter function

Improving Performance Direct bilateral filtering is slow (minutes) Fast algorithm: bilateral grid - http://groups.csail.mit.edu/graphics/bilagrid/ - http://people.csail.mit.edu/sparis/publi/2009/ijcv/ Paris_09_Fast_Approximation.pdf - http://graphics.stanford.edu/papers/gkdtrees/