Overview of Haze Removal Methods Matteo Pedone Machine Vision Group, University of Oulu, Finland

Overview of Haze Removal Methods 1.Description of the problem 2.Overview of current approaches found in literature 3.Strengths and weaknesses of present methods 4.Description of our method

The Atmospheric Scattering Model A particle scatters incident light The nature of scattering depends on material properties, shape and size The exact form and intensity of the scattering pattern varies dramatically with particle size

The Atmospheric Scattering Model Haze: constituted of aerosol (small particles suspended in gas) Main sources: volcanic ashes, foliage exudation, combustion products, sea salt… Haze particles are larger than air molecules but smaller than fog droplets. produce a distinctive gray or bluish hue and affects visibility. Extends to altitudes of several Km

The Atmospheric Scattering Model Fog: Same origins as haze, associated with an increase in relative humidity of an air Size of water droplets increases Haze can turn into fog (transition state: mist) Reduces visibility more than haze Extends to altitudes of few hundred meters.

The Atmospheric Scattering Model

Important physical mechanisms: Attenuation: radiance of a scene-point falls as its distance from the observer increases Airlight: Atmosphere behaving like a source of light. Due to multiple scattering. Increases with distance.

The Atmospheric Scattering Model Mathematical Model: I(x) is the observed radiance at x J(x) is the original scene radiance at x A is the airlight t(x), scalar called transmission: describes how the radiance of a point in the scene is attenuated according to its distance d from the observer Note that I, J, A are (R,G,B) triplets

The Atmospheric Scattering Model Mathematical Model: In order to remove the effect of haze, one must recover J(x) Quantities A and t are typically unknown I(x) is known

The Atmospheric Scattering Model References Narasimhan & Nayar, “Vision and the Atmosphere”, International Journal of Computer Vision, 2001

Overview of Haze Removal Methods 1.Description of the problem 2.Overview of current approaches found in literature 3.Strengths and weaknesses of present methods 4.Description of our method

Current Methods for Haze Removal Can be grouped into several categories 1.With multiple images 2.With one image + depth-map 3.Single image Subcategories of the ones above are: 1.Requires user interaction 2.Fully automatic We are mostly interested in Single-Image methods

Multiple Image Approaches Assume 2+ images of the same scene are taken:  Under different weather conditions [1] or  With different polarization filters [2] [1] Narasimhan & Nayar, “Vision and the Atmosphere”, 2001 [2] Schechner et al. 2003, “Polarization-based vision through haze”, Applied Optics 42

Multiple Image Approaches Narasimhan & Nayar’s method Assumes 2+ bad weather images are given Uses geometric constraints to estimate A The airlight component [1-t(x)] is estimated from corresponding pixels of the two bad weather images

Multiple Image Approaches Narasimhan & Nayar’s method, RESULTS (a),(b) Foggy images (c) Dehazed image, (d) Clear weather images

One Image + Depth + Texture Kopf et al. Method: Deep Photo project from SIGGRAPH 2008 Assumes a 3D model of the scene is given (e.g.: from Google Maps) Assumes textures of the scene are given (from satellite or aerial photos) Requires user interaction to align the 3D model with the scene Very accurate results

Single Image Approaches Do not require information extracted from additional images Do not require depth-information Typically rely upon statistical assumptions, and or the nature of the scene (e.g. part of the sky is visible) Sometimes they require user interaction Most relevant: Fattal’s ”Single-Image Dehazing”, SIGGRAPH 2008 He’s ”Single Image Haze Removal Using Dark Channel Prior”, CVPR 2009

Single Image Approaches He’s Method (with Dark Channel Prior) Assumes a portion of the scene is dominated by airlight STATISTICAL ASSUMPTION: ”in most of the non-sky patches, at least one color channel has very low intensity at some pixels. In other words, the minimum intensity in such a patch should have a very low value” The 1st assumption is used to estimate airlight, the 2nd assumption is used to estimate the transmission

Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: Airlight is estimated by picking up the pixels of the image corresponding to the 0.1% brightest pixels in the dark channel, and then choosing the one with maximum intensity.

Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: He shows that the transmission can be estimated by calculating:

Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: He shows that the transmission can be estimated by calculating: Dark channel of the image divided by the airlight color

Single Image Approaches He’s Method (with Dark Channel Prior) Airlight and transmission are sufficient to invert the model and retrieve the original radiance of the scene. Dark channel is computed on square neighborhoods  Block artifacts and halos are reduced by using a soft-matting algorithm.

Overview of Haze Removal Methods 1.Description of the problem 2.Overview of current approaches found in literature 3.Strengths and weaknesses of present methods 4.Description of our method

Summary Multiple image methods require special equipment (polarizers) or same scene under different weather conditions. They don’t necessarily produce better results than single-images approaches

Summary One Image + 3D model + textures Accurate and does not require special equipment Requires a considerable amount of special information (3D model, and aerial photos of the scene) Requires user interaction

Summary Single-Image methods do not require special equipment, nor extra information They either make assumption on the nature of the scene, or require little interaction by the user

Summary Single-Image methods It is known what are the consequences of a bad estimate for the transmission  haze is not completely removed, or it is removed where there is no haze (overboost contrast)

Summary None of the aforementioned authors shows what happens when the airlight estimate is inaccurate (motivation of our work)

Overview of Haze Removal Methods 1.Description of the problem 2.Overview of current approaches found in literature 3.Strengths and weaknesses of present methods 4.Description of our method

Estimation of Airlight Collect statistics of airlight colors from 100+ natural images (daylight and twilight hazy scenes) Manually select 32x32 pixel patch with “full haze” Airlight colors are scattered around a degrees line in hue-saturation plane, and most are close to the origin (=> low saturation).

Estimation of Airlight Extract patches of 13x13 pixels from hazy image according to the following criteria: 1. The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes) 2. The pixels in the patch do not have too low or too high transmission (avoid degenerate cases)

Estimation of Airlight Extract patches of 13x13 pixels from hazy image according to the following criteria: 1. The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes) 2. The pixels in the patch do not have too low or too high transmission (avoid degenerate cases) 3. Pixels in the patches do not have too low saturation (hue would not be reliable). 4. Pixels in the patches are not too dark or too bright in average, and variance should not be too high (noise) or too low (homogeneous areas with no shades).

Estimation of Airlight Solve minimization problem: find an airlight color vector that enforces: 1.Perpendicularity to patches albedos 2.Closeness to natural airlight hue line 3.Low saturation n i : normal to the plane containing the [R,G,B] values of the pixels in the i-th patch extracted n sky : unit-vector in RGB space having direction corresponding to the statistical hue of airlight in natural images w : unit vector [1,1,1]/3 1/2 c(n i ): scalar associated with the i-th patch (based of residual error, see paper) : weight parameters (respective default values: 3, 0.02)

Estimation of Airlight Results with artifical haze (Middlebury dataset)

Estimation of Airlight Results with real images

THANK YOU