1. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 1 PATTERN CLASSIFICATION WITH DISTANCE FUNCTIONS Prof. George Papadourakis, Ph.D.

2. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 2 Simple Classifiers (1/2)  Two approaches for designing a classifier.  Theoretical: Initially, a mathematical model of the problem is created, afterwards, based on the model, the best classifier is designed.  Practical Application: Initially, assumption of a possible solution, from category samples, afterwards, from real data, best escalation  Empirical method, practical applications,  Simple Solution of the problem, feature analysis, detection of flaws, gradually as complex as it gets (Practical Application)

3. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 3  A simpler and empirical approach:  Classification with the use of distance functions. YES NO  Simple Classifiers (2/2)

4. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 4 Minimum Distance Classifiers (1/5)  Affective for all categories well separable.  Each category C i has a feature vector z i Usually z i the mean vector of C i patterns  Euclidean Pattern Distance x with z i  x belongs to category C i if:

5. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 5  Instead of smaller D i the greater d i x T x doesn’t have information, so Μ d i Minimum Distance Classifiers (2/5)

6. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 6 Minimum Distance Classifiers (3/5) Patterns near Z i : separation simple Big Scatter separation complex z1z1 z1z1 z2z2 z2z2 C1C1 C2C2 Distance Limits and metrics for the Euclidean Distance

7. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 7 Minimum Distance Classifiers (4/5) Patterns near Z i : separation simple Big Scatter separation complex Distance Limits for the Euclidean Distance with overlay categories

8. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 8 Minimum Distance Classifiers (5/5) Patterns near Z i : separation simple Big Scatter separation complex

9. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 9 Distance Norms (1/8)  Euclidean pattern distance x with z dimension n

10. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 10 Distance Norms (2/8)  Integer values: Month (1..12), Binary elements, etc.  Hippodamian metrics (Hippodamus of Miletus 5 th B.C.) or first class metrics, Manhatan distance, city block

11. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 11 Distance Norms (3/8)

12. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 12 Distance Norms (4/8) Metrics in Hippodamian Distance

13. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 13  Qualitative data - binary form: YES-NO, 1 - 0  Hamming Distance – binary distance norm – EXOR  Hamming Distance  Subcase of the Hippodamian  Coincide for binary vectors. Distance Norms (5/8)

14. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 14  The metrics, special cases of distance Minkowsky  s=2 – Euclidean, s=1 - Hippdamian.  s=infinity - Chebysher Distance Distance Norms (6/8)

15. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 15  Useful properties of Minkowsky Distance Distance Norms (7/8)

16. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 16  Mahalanobis’ Distance: Statistical Pointers  C: Covariance matrix,  z: mean category feature vector  C=I Mahalanobis=Euclidean Distance Norms (8/8) z1z1 z2z2 C1C1 C2C2

17. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 17  Similarity Norms: how similar are two patterns  Similarity Norm high, then distance short  Internal product between two vectors:  Used x, z normalized, length 1.  Internal product depends on the angle.  Limits: Similarity Norms (1/6)

18. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 18  When x, z non normalized, angle cosine:  Internal product expresses correlation  Maximum value same direction  Useful when there sets exist, which grow along the primary axis. Similarity Norms (2/6)

19. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 19 X2X2 X2X2 X1X1 X1X1 x x Z1Z1 Z1Z1 Z2Z2 Z2Z2 θ2θ2 θ2θ2 θ1θ1 θ1θ1 (0,0) Similarity Norms (3/6)

20. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 20 Similarity Norms (4/6)  Unknown Pattern X belongs to category C

21. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 21  Tanimoto metrics: real and discreet values  Unkown Pattern Similarity Norms (5/6) X belongs to category C

22. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 22  Tanimoto Metrics used mainly with discreet values  Based on the comparison of two sets.  Common Elements number for two vectors by different elements number. Similarity Norms (6/6)

23. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 23 Matching with models (1/2)  Matching with models: natural approach for pattern recognition

24. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 24  Models: representative vectors  Hamming Distance (binary elements)  Used when category variation is due to noise: speech recognition, simple problems of pattern recognition. Matching with models (2/2)

25. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 25 Satellite Image Recognition System (1/4)  Applications  Weather forecast  Disease detection in crops  Military Applications  System which defines the existing land uses  The procedure is based on the way of absorption and reflection of the sun light to the ground, in several phasma areas  Features –> reflecting light,  two coloring bands: x 1 - Infrared: high reflection, areas with water. x 2 - Red: high absorption, areas with vegetation.

26. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 26  Terminus categories:  S – Sand area  H – Hay Cultivations  W - Water (rivers, lakes)  U – Urban area  C – Corn Cultivations  F – Forestral area Satellite Image Recognition System (2/4)

27. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 27 Satellite Image Recognition System (3/4)  W – water completely separable  H – Hay, F – Forests none- well separable  Minimun Distance Classifier  Point 1: C – Corn Cultivations  Point 2: S – Sand area  By shape,possibly U – Urban Area

28. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 28 Satellite Image Recognition System (4/4)  Minimum Distance Classifier – Simple approach, without complicated calculations.  However, properties of the statistical distribution patterns are not taken into consideration. Isometric Curves

29. Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 29  Duda, Heart: Pattern Classification and Scene Analysis. J. Wiley & Sons, New York, 1982. (2nd edition 2000).  Fukunaga: Introduction to Statistical Pattern Recognition. Academic Press, 1990.  Bishop: Neural Networks for Pattern Recognition. Claredon Press, Oxford, 1997.  Schlesinger, Hlaváč: Ten lectures on statistical and structural pattern recognition. Kluwer Academic Publisher, 2002.  Satosi Watanabe Pattern Recognition: Human and Mechanical, Wiley, 1985  E. Gose, R. Johnsonbaught, S. Jost, Pattern recognition and image analysis, Prentice Hall, 1996.  Sergios Thodoridis, Kostantinos Koutroumbas, Pattern recognition, Academiv Press, 1998. References