Minimum Likelihood Image Feature and Scale Detection Kim Steenstrup Pedersen Collaborators: Pieter van Dorst, TUe, The Netherlands Marco Loog, ITU, Denmark

Gaussian Processes in Practice 2 What is an image feature? Marr’s (1982) primal sketch (edges, bars, corners, blobs) Marr’s (1982) primal sketch (edges, bars, corners, blobs) Geometrical features, Marr’s features defined by differential geometry: Canny (1986), Lindeberg (1998) Geometrical features, Marr’s features defined by differential geometry: Canny (1986), Lindeberg (1998) Iconic features: Koenderink (1993), Griffin & Lillholm (2005) Iconic features: Koenderink (1993), Griffin & Lillholm (2005) Observation: Features are usually points and curves, i.e. sparsely distributed in space (unlikely events). Features have an intrinsic scale / size. How blurred is the edge? What is the size if a bar?

Gaussian Processes in Practice 3 A probabilistic primal sketch Our definition: Features are points that are unlikely to occure under an image model. Similarly the scale of the feature is defined as the most unlikely scale. We use fractional Brownian images as a generic model of the intensity correlation found in natural images. Captures second order statistics of generic image points (non-feature points). We use fractional Brownian images as a generic model of the intensity correlation found in natural images. Captures second order statistics of generic image points (non-feature points). The model includes feature scale naturally. The model includes feature scale naturally. This leads to a probabilistic feature and scale detection. This leads to a probabilistic feature and scale detection. Possible applications: Feature detection, interest points for object recognition, correspondance in stereo, tracking, etc.

Gaussian Processes in Practice 4 Probabilistic feature detection Feature detection: Konishi et al. (1999, 2002, 2003) Konishi et al. (1999, 2002, 2003) Lillholm & Pedersen (2004) Lillholm & Pedersen (2004) Scale selection: Pedersen & Nielsen (1999) Pedersen & Nielsen (1999) Loog et al. (2005) Loog et al. (2005)

Gaussian Processes in Practice 5 Linear scale-space derivatives Scale-space derivatives:

Gaussian Processes in Practice 6 Scale Space k-Jet Representation We use the k-jet as representation of the local geometry: (The coefficients of the truncated Taylor expansion of the blurred image.) Biologically plausible representation (Koenderink et al., 1987)

Gaussian Processes in Practice 7 Probabilistic image models Key results on natural image statistics: Scale invariance / Self-similarity: Power spectrum, : Field (1987), Ruderman & Bialek (1994) Scale invariance / Self-similarity: Power spectrum, : Field (1987), Ruderman & Bialek (1994) In general non-Gaussian filter responses! In general non-Gaussian filter responses! Fractional Brownian images as model of natural images: Mumford & Gidas (2001), Pedersen (2003), Markussen et al. (2005) Mumford & Gidas (2001), Pedersen (2003), Markussen et al. (2005) Jet covariance of natural images resembles that of fractional Brownian images: Pedersen (2003) Jet covariance of natural images resembles that of fractional Brownian images: Pedersen (2003)

Gaussian Processes in Practice 8 Fractional Brownian images

Gaussian Processes in Practice 9 FBm in Jet space (Result from Pedersen (2003))

Gaussian Processes in Practice 10 Detecting Features and Scales Detecting points in scale-space that are locally unlikely (minima): (We could also have maximised.)

Gaussian Processes in Practice 11 Why minimum likeli scales? Lindeberg (1998) maximises polynomials of derivatives in order to detect features and scales. Similarly, we maximise in order to detect features and scales. The difference lies in the choice of polynomial! We use an image model and Lindeberg uses a feature model.

Gaussian Processes in Practice 12 Synthetic examples: Double blobs

Gaussian Processes in Practice 13 Synthetic examples: Blurred step edge

Gaussian Processes in Practice 14 Real Example: Sunflowers

Gaussian Processes in Practice 15 Sunflowers: Multi-scale

Gaussian Processes in Practice 16 Sunflowers: Fixed scale

Gaussian Processes in Practice 17 Summary Minimising the likelihood of an image point under the fractional Brownian image model detects feature points and their intrinsic scale. Minimising the likelihood of an image point under the fractional Brownian image model detects feature points and their intrinsic scale. There is a relationship between feature types and the  parameter. There is a relationship between feature types and the  parameter. Why over estimation of the scale? Why over estimation of the scale? Preliminary results look promising, a performance evaluation is needed (task based?). Preliminary results look promising, a performance evaluation is needed (task based?). The method is pointwise. How to handle curve features (edges, bars, ridges)? The method is pointwise. How to handle curve features (edges, bars, ridges)?