Image features and properties

Image content representation The simplest representation of an image pattern is to list image pixels, one after the other starting from the top left corner. Similar representation can be taken for image windows of any size. The vector representation has very high-dimension.

Features for image content representation Image features are meaningful, characterizing patterns of the image: “a distinctive characteristic of the data: signifies something to somebody”. They may refer to the image itself as a whole or more frequently to local, meaningful, detectable structures or parts of the image – Global (holistic) features summarize properties like color and texture of the whole appearance pattern of the image or image regions or represent meaningful structures that are present in the image, like region boundaries and shapes, edges, lines… (A). – Local features refer to properties of special image points and their surrounding regions referred to as interest points or keypoints (B). Local features makes recognition more robust to partial occlusions and viewpoint changes. Most of the recognition approaches today use suitable local features. (A) (B)

Image Features Features are used for the purpose of matching in image to image comparison and image retrieval, or to extract more meaningful content for the purpose of image understanding, semantic annotation and multimedia processing. To this end two interrelated tasks are of fundamental importance: – Feature detection i.e. the process of extracting/ locating such characteristic elements from raw image data. Appropriate algorithms or data transformations are used to this end. – Feature representation i.e. definition of suitable feature descriptors that capture the feature saliency and have nice properties of invariance for the purpose of recognition and retrieval. Features are usually represented in vector form.

Feature invariance When performing detection of local features it is important to compute stable metric with respect to (small) variations in position. Repeatability of feature detectors, i.e. the frequency with which local features detected in one image are found at a distance  pixels of the same location in a trasformed image, is important for matching. More in general we will talk about invariance as a fundamental property to matching. We are therefore interested especially to invariant descriptors that keep stable under several conditions. A feature F is invariant to condition K for object x, if it has the value F x regardless the effect of condition K. Fundamental invariance are wrt photometric and geometric transformations.

Photometric transformations ‒ Light-object interaction ‒ Affine intensity change ( I  aI+b ) Geometric transformations ‒ Rotation preserves angles, parallel lines and distances ‒ Similarity (rotation + uniform scale) preserves angles, parallel lines and distance ratios ‒ Affine (non uniform scale dependent on direction) valid for: orthographic camera, locally planar object preserves parallelism ‒ Projective preserves intersection and tangency only Fundamental transformations

Effects of main image transformations Illumination Scale Rotation Affine Full perspective camera motion and position light source light interaction with surface cover