computer graphics & visualization HDRI

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Problem The world and our visual system has a HDR… … but not our digital equipment

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Need to find ways to Record Store Process HDR Data Convert and output

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Recording Engineering versus Nature

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Lense-System

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Chromatic aberration HumanEngineer

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Chromatic aberration HumanEngineer

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Chromatic aberration

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Resolution Retina CCD - Charge-coupled Device

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Resolution Estimated overall resolution: 15 to 576 megapixels

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Resolution

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Dynamic Range From: 100:1 (single focus) to 1,000,000:1 (Purkinje effect) 16384:1

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Nature the clear looser? Not really  excessive post processing

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Post processing

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Post processing

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Post processing

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Post processing… …is executed parallel and hierarchical

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Alternatives HDR Cameras Multiple Exposures expensive dynamic scenes

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Merging multiple exposures camera exposure curve rescale and add weighted mean in overlap regions

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Storage, first a little bit about colors

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group …a little bit about colors

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Color Encoding Primary colors RGB, CMYK, XYZ with transformation formula (often a matrix) color gamut quantization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Encoding Schemes Pixar Log Encoding (TIFF) 11 bit log/exponential encoding Radiance RGBE Encoding (HDR) 8bit RGB mantissa, 8bit shared exponent SGI LogLuv (TIFF) 16bit logarithmic luminance+ 2 * 8bit u,v coords ILM OpenEXR (EXR) 16bit floating point format (half)

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Processing 1.load data 2.convert to 3 * 32bit float / 3 * 16bit float 3.compute whatever you want 4.encode down into preferred file format and store

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Output Solution 1: Get a HDR Screen (BRIGHTSIDE)

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Output Solution 2: Tone Mapping

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Apply standard arithmetic or logical operations to images (pixel values) – Output pixel only depends on corresponding pixels in input images – Images must have the same size – Very efficient in terms of performance and memory requirements No intermediate result needed

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Operators – Addition, Subtraction Overlay, background removal, noise reduction – Multiplication (Scaling) Brighten, darken – Blending/compositing (weighted average) Multi-modality fusion – Logical bitwise operations AND/NAND (Intersection) OR/NOR (Union) – Invert/Logical NOT

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Examples Intensity scaling A & B A B Detect object that did not move !A & !B

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Example – Outline edges in the original image + Wrap around of pixel values!

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Example (cont.) – Generate mask by thresholding and AND the inverted mask with the image

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Example (cont.) – Add unthresholded mask to original image

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image arithmetic Example – Blending X P0 + (1-X) P1 – Reduces contrast before adding X=0.5 X=0.7

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Thresholding – Cut arbitrary ranges Normalization (contrast stretching) – S out = (S in – Min) / (Max – Min) Logarithmic/exponential operators – Enhance/supress low intensity pixels Histogram equalization – monotonic, non-linear modification of the dynamic intensity range Histogram Histogram shows the distribution of pixels among intensity values

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Histogram equalization – Re-assign intensities – Output image has uniform intensity distribution Cannot be achieved in general Find transfer function that maps input values a to output values b such as to approximate a flat histogram – Relative order (in terms of intensity) of pixels will not be destroyed

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Consider probability density function p(x) and probability distribution function P(x) – P(a) is the probability that a brightness chosen from a region is less than or equal to a given brightness value a – P(a) =  x=0  a p(x) – The probability that a brightness in a region falls between a and a+  a is p(a)  a = dP(a)/da  a – Brightness probability density function is given by the histogram

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Mapping F (a  b) for histogram equalization – p a (a)da = p b (b)db  dF=db=p a (a)da/p b (b) – Because p b (b) should be constant to (1/(2 B -1)): F(a)=(2 B -1)P(a) – Digital implementation: f(a)=max(0,round(2 B -1 n a /N)-1) n a : number of pixels with intensity  a N: Number of pixels

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Before and after histogram equalization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Before and after histogram equalization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Before and after histogram equalization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point operators Before and after histogram equalization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Visibility is reproduced you can see an object on the display iff you can see it in the real scene Viewing the image produces the same subjective experience as viewing the real scene

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Setting the brightest pixel to 1 scaling the rest linearly – Light sources are multiple orders of magnitude brighter  Scene is almost entirely black Setting the brightest non light-source pixel to 1-e scaling the rest linearly/logarithmic – Light sources are still mapped to 1 – Visibility is preserved – but experience is not preserved – changes in light emission have no effect on the image

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping [Ward 97] Compute Luminance Histogram

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Compute Histogram Equalization

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Apply Linear Ceiling

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Apply just noticeable difference ceiling

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Tone Mapping Apply veiling luminance effects (blooming)

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group HDR

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group HDR

computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group HDR