Image Quality Measures Omar Javed, Sohaib Khan Dr. Mubarak Shah

Factors Affecting Registration Performance Mission image quality and content Reference image quality and content Mission-Reference differences Viewing geometry Quality of DEM Method of registration

Test Images 5 image sequences were used as test images –08 Oct 99 Image Sequence08 Oct 99 Image Sequence –13 Oct 99 Image Sequence13 Oct 99 Image Sequence –15 Oct 99 Image Sequence15 Oct 99 Image Sequence –16 Oct 99 Image Sequence16 Oct 99 Image Sequence –19 Oct 99 Image Sequence19 Oct 99 Image Sequence

In This Presentation... Factors affecting registration performance. Image quality and content measures –SNR estimation –Texture measures –Gabor filters

Properties of Mission Imagery Affecting Registration Performance Scene Content –Homogenous texture i.e. no distinctive features –Example Images

Properties of Mission Imagery Affecting Registration Performance Aperture Problem –Presence of roads or homogenous elongated features causes error in registration along the direction of elongation

Properties of Mission Imagery Affecting Registration Performance Extreme blur

Properties of Mission Imagery Affecting Registration Performance Spurious weather phenomenon e.g. clouds, haze..

Image Quality and Content Measures SNR estimation Texture Measures Gabor Filters

Blind SNR Estimation A method to estimate the quality of image is based on quantity Q=  2   f i dr The intensity image f i can be modeled by a mixture of Rayleigh pdfs

Algorithm For SNR Estimation Compute the horizontal and vertical derivatives of the image Calculate the gradient magnitude ‘ΔΙ’ from the derivatives. Obtain a Histogram of gradient intensity values from ΔΙ. Count the number of pixels > 2μ, where μ is mean of ΔΙ. Normalize by total number of pixels.

Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=20 # of false +ves=11 # of false -ves= 20 Misclassification Error= 44.28%

Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=21 # of false +ves=9 # of false -ves= 17 Misclassification Error= 30.95%

Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=0 # of false +ves=0 # of false -ves= 6 Misclassification Error= 5.21%

Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=25 # of false +ves=10 # of false -ves= 44 Misclassification Error= 31.95%

Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=19 # of false +ves=17 # of false -ves= 35 Misclassification Error= 30.23%

Discussion of Results Images labeled as low quality –Red squares indicates large registration error or exclusion from registration

Discussion of Results Images labeled as high quality –Red squares indicates large registration error or exclusion from registration

Suitability as an Image Metric Advantages –Extreme blur is detected and corresponds well with registration error. –Low computation time Disadvantages –Cloud detection is not robust. –Feature less images are a major cause of registration error. SNR is not able to detect these images robustly.

Texture Gray Level Co-occurrence Matrices (GLCMs) –2D histogram which encodes spatial relations parameters: direction, distance, quantization-level window-size –Measures are computed on the GLCM entropy, contrast, homogeneity, energy

Computing GLCM A GLCM P[i,j] is defined by –specifying displacement vector d=(dx,dy) –Counting all pairs of pixels separated by d having gray levels I and j. Input image Window size i j Distance and Direction Relationship d 1 ……… i ……… ….. j ……… P(i, j) Quantization level

GLCM Measures Entropy –Randomness of gray level distribution Energy: –uniformity of gray level in a region

GLCM Measures Contrast –Measure of difference between gray levels Homogeneity –Measure of similarity of texture

Contrast Entropy Homogeneity Energy Contrast GLCM measures

Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=19 # of false +ves=5 # of false -ves= 15 Misclassification Error= 28.57%

Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=12 # of false +ves=7 # of false -ves= 26 Misclassification Error= 39.28%

Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=15 # of false +ves=15 # of false -ves= 6 Misclassification Error= 18.26%

Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=32 # of false +ves=13 # of false -ves= 41 Misclassification Error= 31.95%

Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=47 # of false +ves=18 # of false -ves= 8 Misclassification Error= 15.11%

Discussion of Results Images labeled as low quality –Red squares indicates large registration error or exclusion from registration

Discussion of Results Images labeled as high quality –Red squares indicates large registration error or exclusion from registration

Suitability as an Image Metric Advantages –Homogeneous texture is detected though detection is not robust. Disadvantages –It is difficult to fine tune the several parameters of GLCM’s so that consistent results are obtained for a variety of images. –Clouds are not detected. –Blur is not detected.

Gabor Filter The Gabor function –is a complex sinusoid centered at frequency (U,V) modulated by a Guassian envelop. Gabor function can discriminate between textures

Gabor Filter Experiments were done with the following values –Variance of Guassian = 30 –Four Gabor kernels 1 Horizontal 1 Vertical 2 Diagonal

Gabor Kernels

Calculation of Quality metric Normalize image intensity values (0 to 255). –Calculate mean of intensity values. –Subtract mean from all intensity. –Add 128 (middle value). Determine Gabor response of the image. –Generate four Gabor kernels. –Convolve each kernel with the image. –Multiply the four results.

Calculation of Quality metric Perform connected component analysis and clean up small areas of response. Count the number of pixels N p in the response area. Normalize by total number of pixels. If N p <T low label image as low quality. If N p >T high label image as high quality.

Calculation of Quality metric If both the previous conditions are not met then calculate spatial covariance of Gabor response. If spatial covariance is < T s label image as low quality otherwise label image as high quality.

Results Images of Gabor response

Results Result after convolution from vertical kernel

Results Result after convolution from horizontal kernel

Results Result after convolution from diagonal kernel

Results Result after convolution from diagonal kernel

Results Results after multiplication and thresholding

Results Images of Gabor response

Results Images of Gabor response

Results Images of Gabor response

Results Images of Gabor response

Results Images of Gabor response

Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=26 # of false +ves=2 # of false -ves= 5 Misclassification Error= 10.00%

Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=12 # of false +ves=4 # of false -ves= 21 Misclassification Error= 29.76%

Results False +ves Difficulty –Correct Detection

Results False -ves

Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=0 # of false +ves=0 # of false -ves= 6 Misclassification Error= 5.21%

Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=46 # of false +ves=7 # of false -ves= 22 Misclassification Error= 17.15%

Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=49 # of false +ves=22 # of false -ves= 10 Misclassification Error= 18.6%

Results A Sample of Images labeled as low quality –Featureless images

Results A Sample of Images labeled as low quality –Cloudy Images –Blur

Results A Sample of Images labeled as high quality

Suitability as an Image Metric Advantages –Accurate estimation of amount of texture in an image. –It can identify hazy, cloudy or featureless images. –Prediction of success/failure of registration possible. Disadvantages –High computation time.