1. Image Quality Measures Omar Javed, Sohaib Khan Dr. Mubarak Shah

2. Factors Affecting Registration Performance Mission image quality and content Reference image quality and content Mission-Reference differences Viewing geometry Quality of DEM Method of registration

3. Test Images 5 image sequences were used as test images –08 Oct 99 Image Sequence08 Oct 99 Image Sequence –13 Oct 99 Image Sequence13 Oct 99 Image Sequence –15 Oct 99 Image Sequence15 Oct 99 Image Sequence –16 Oct 99 Image Sequence16 Oct 99 Image Sequence –19 Oct 99 Image Sequence19 Oct 99 Image Sequence

4. In This Presentation... Factors affecting registration performance. Image quality and content measures –SNR estimation –Texture measures –Gabor filters

5. Properties of Mission Imagery Affecting Registration Performance Scene Content –Homogenous texture i.e. no distinctive features –Example Images

6. Properties of Mission Imagery Affecting Registration Performance Aperture Problem –Presence of roads or homogenous elongated features causes error in registration along the direction of elongation

7. Properties of Mission Imagery Affecting Registration Performance Extreme blur

8. Properties of Mission Imagery Affecting Registration Performance Spurious weather phenomenon e.g. clouds, haze..

9. Image Quality and Content Measures SNR estimation Texture Measures Gabor Filters

10. Blind SNR Estimation A method to estimate the quality of image is based on quantity Q=  2   f i dr The intensity image f i can be modeled by a mixture of Rayleigh pdfs

11. Algorithm For SNR Estimation Compute the horizontal and vertical derivatives of the image Calculate the gradient magnitude ‘ΔΙ’ from the derivatives. Obtain a Histogram of gradient intensity values from ΔΙ. Count the number of pixels > 2μ, where μ is mean of ΔΙ. Normalize by total number of pixels.

12. Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=20 # of false +ves=11 # of false -ves= 20 Misclassification Error= 44.28%

13. Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=21 # of false +ves=9 # of false -ves= 17 Misclassification Error= 30.95%

14. Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=0 # of false +ves=0 # of false -ves= 6 Misclassification Error= 5.21%

15. Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=25 # of false +ves=10 # of false -ves= 44 Misclassification Error= 31.95%

16. Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=19 # of false +ves=17 # of false -ves= 35 Misclassification Error= 30.23%

17. Discussion of Results Images labeled as low quality –Red squares indicates large registration error or exclusion from registration

18. Discussion of Results Images labeled as high quality –Red squares indicates large registration error or exclusion from registration

19. Suitability as an Image Metric Advantages –Extreme blur is detected and corresponds well with registration error. –Low computation time Disadvantages –Cloud detection is not robust. –Feature less images are a major cause of registration error. SNR is not able to detect these images robustly.

20. Texture Gray Level Co-occurrence Matrices (GLCMs) –2D histogram which encodes spatial relations parameters: direction, distance, quantization-level window-size –Measures are computed on the GLCM entropy, contrast, homogeneity, energy

21. Computing GLCM A GLCM P[i,j] is defined by –specifying displacement vector d=(dx,dy) –Counting all pairs of pixels separated by d having gray levels I and j. Input image Window size i j Distance and Direction Relationship d 1 ……… i ………. 255 1 ….. j ………. 255 +1 P(i, j) Quantization level

22. GLCM Measures Entropy –Randomness of gray level distribution Energy: –uniformity of gray level in a region

23. GLCM Measures Contrast –Measure of difference between gray levels Homogeneity –Measure of similarity of texture

24. Contrast Entropy Homogeneity Energy Contrast GLCM measures

25. Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=19 # of false +ves=5 # of false -ves= 15 Misclassification Error= 28.57%

26. Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=12 # of false +ves=7 # of false -ves= 26 Misclassification Error= 39.28%

27. Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=15 # of false +ves=15 # of false -ves= 6 Misclassification Error= 18.26%

28. Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=32 # of false +ves=13 # of false -ves= 41 Misclassification Error= 31.95%

29. Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=47 # of false +ves=18 # of false -ves= 8 Misclassification Error= 15.11%

30. Discussion of Results Images labeled as low quality –Red squares indicates large registration error or exclusion from registration

31. Discussion of Results Images labeled as high quality –Red squares indicates large registration error or exclusion from registration

32. Suitability as an Image Metric Advantages –Homogeneous texture is detected though detection is not robust. Disadvantages –It is difficult to fine tune the several parameters of GLCM’s so that consistent results are obtained for a variety of images. –Clouds are not detected. –Blur is not detected.

33. Gabor Filter The Gabor function –is a complex sinusoid centered at frequency (U,V) modulated by a Guassian envelop. Gabor function can discriminate between textures

34. Gabor Filter Experiments were done with the following values –Variance of Guassian = 30 –Four Gabor kernels 1 Horizontal 1 Vertical 2 Diagonal

35. Gabor Kernels

36. Calculation of Quality metric Normalize image intensity values (0 to 255). –Calculate mean of intensity values. –Subtract mean from all intensity. –Add 128 (middle value). Determine Gabor response of the image. –Generate four Gabor kernels. –Convolve each kernel with the image. –Multiply the four results.

37. Calculation of Quality metric Perform connected component analysis and clean up small areas of response. Count the number of pixels N p in the response area. Normalize by total number of pixels. If N p <T low label image as low quality. If N p >T high label image as high quality.

38. Calculation of Quality metric If both the previous conditions are not met then calculate spatial covariance of Gabor response. If spatial covariance is < T s label image as low quality otherwise label image as high quality.

39. Results Images of Gabor response

40. Results Result after convolution from vertical kernel

41. Results Result after convolution from horizontal kernel

42. Results Result after convolution from diagonal kernel

43. Results Result after convolution from diagonal kernel

44. Results Results after multiplication and thresholding

45. Results Images of Gabor response

46. Results Images of Gabor response

47. Results Images of Gabor response

48. Results Images of Gabor response

49. Results Images of Gabor response

50. Results 08 Oct 99 Sequence Total Images = 70 Images with error=12 Unregistered Images=17 Images identified by metric as unregisterable=26 # of false +ves=2 # of false -ves= 5 Misclassification Error= 10.00%

51. Results 13 Oct 99 Sequence Total Images = 84 Images with error=18 Unregistered Images=11 Images identified by metric as unregisterable=12 # of false +ves=4 # of false -ves= 21 Misclassification Error= 29.76%

52. Results False +ves Difficulty –Correct Detection

53. Results False -ves

54. Results 15 Oct 99 Sequence Total Images = 115 Images with error=6 Unregistered Images=0 Images identified by metric as unregisterable=0 # of false +ves=0 # of false -ves= 6 Misclassification Error= 5.21%

55. Results 16 Oct 99 Sequence Total Images = 169 Images with error=19 Unregistered Images=39 Images identified by metric as unregisterable=46 # of false +ves=7 # of false -ves= 22 Misclassification Error= 17.15%

56. Results 19 Oct 99 Sequence Total Images = 172 Images with error=15 Unregistered Images=22 Images identified by metric as unregisterable=49 # of false +ves=22 # of false -ves= 10 Misclassification Error= 18.6%

57. Results A Sample of Images labeled as low quality –Featureless images

58. Results A Sample of Images labeled as low quality –Cloudy Images –Blur

59. Results A Sample of Images labeled as high quality

60. Suitability as an Image Metric Advantages –Accurate estimation of amount of texture in an image. –It can identify hazy, cloudy or featureless images. –Prediction of success/failure of registration possible. Disadvantages –High computation time.