1. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 1 Prof. Heikki Kälviäinen Lappeenranta University of Technology, Finland

2. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 2 Applications of Hough Transform for Image Processing and Analysis Heikki Kälviäinen Professor, Computer Science * Machine Vision and Pattern Recognition Laboratory Department of Information Technology Lappeenranta University of Technology (LUT), FINLAND Heikki.Kalviainen@lut.fi http/www.lut.fi/~kalviai **Centre for Vision, Speech, and Signal Processing (CVSSP) University of Surrey, UNITED KINGDOMHeikki.Kalviainen@lut.fi

3. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 3

4. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY Hough Transform Shape detection –Lines, circles, ellipses, arbitrary shapes. Motion detection and estimation –Simple and robust methods in 2D. Mixed pixel classification –Large data sets of mixed pixels. Image compression –Compression and better image quality. Image databases –Matching of images.

5. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 5 Hough surveys and comparisons J. Illingworth, J. Kittler, A Survey of the Hough Transform, Computer Vision, Graphics, and Image Processing, 1988, vol. 44, pp. 87-116. V.F. Leavers, Survey: Which Hough Transform, CVGIP Image Understanding, 1993, vol. 58, no. 2, pp. 250 ‑ 264. H. Kälviäinen, P. Hirvonen, L. Xu, E. Oja, Probabilistic, non- probabilistic Hough transforms: overview and comparisons. Image, Vision Computing, 1995, vol. 13, no. 4, pp. 239 ‑ 251. N. Kiryati, H. Kälviäinen, S. Alaoutinen, Randomized or Probabilistic Hough Transform: Unified Performance Evaluation, Pattern Recognition Letters, 2000, vol. 21, nos. 13-14, pp. 1157-1164.

6. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 6

7. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 7 Compression, Similarity, Matching, Object Recognition

8. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 8 Feature extraction using Hough Transform

9. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 9 Hough Transform A method for global feature extraction: –y = a x + b => b = -x a + y. –For each pixel (x,y) compute a curve b = -x a + b into the parameter space. –Alternatively the normal presentation of a line: Hough Transform detects sets of pixels which represent geometric primitives in a binary image. Lines, circles, ellipses, arbitrary shapes, etc. Tolerant to noise and distortions in an image, but traditional versions suffer from problems with time and space complexities. New variants: probabilistic and deterministic Hough Transforms.

10. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 10 Hough Transform (SHT)

11. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 11 Kernel of the Hough Transform 1.Create the set D of all edge points in a binary picture. 2.Transform each point in the set D into a parameterized curve in the parameter space. 3.Increment the cells in the parameter space determined by the parametric curve. 4.Detect local maxima in the accumulator array. Each local maximum may correspond to a parametric curve in the image space. 5.Extract the curve segments using the knowledge of the maximum positions.

12. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 12 Randomized Hough Transform (RHT) Developed in Lappeenranta University of Technology (LUT), FINLAND. Xu, L., Oja, E., Kultanen, P, ”A New Curve Detection Method: Randomized Hough Transform (RHT), Pattern Recognition Letters, vol. 11, no. 5., 1990, pp. 331-338.

13. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 13 Kernel of the Randomized Hough Transform (RHT) 1.Create the set D of all edge points in a binary edge picture. 2.Select a point pair (d_i, d_j) randomly from the set D. 3.If the points do not satisfy the predefined distance limits, go to Step 2; otherwise continue to Step 4. 4.Solve the parameter space point (a, b) using the curve equation with the points (d_i, d_j). 5.Accumulate the cell A(a, b) in the accumulator space. 6.If the A(a, b) is equal to the threshold t, the parameters a and b describe the parameters of the detected curve; otherwise continue to Step 2.

14. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 14 1.Infinite scope parameter space. 2.Arbitrarily high parameter resolution. 3.High computational speed. 4.Small storage. Advances of RHT over SHT

15. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 15 RHT Extensions Kälviäinen, H., Hirvonen, P., Xu, L., Oja, E., ”Probabilistic and Non-probabilistic Hough Transforms: Overview and Comparisons,” Image and Vision Computing, Vol. 13, No. 4, 1995, pp. 239-251.

16. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 16 More complex images

17. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 17 Motion Detection by RHT (MDRHT) 2D motion detection as sets of moving pixels. A set of moving edge points is assumed to illustrate a moving object frame by frame. The majority of the points are assumed to move rigidly. Two moving points is the simplest version. Extensions: (a) rotation and scaling, (b) exploiting gradient information of each edge point, (c) using three or more moving points as evidence, and (d) detecting multiple moving objects.

18. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 18 Motion Detection Using RHT (MDRHT) Kälviäinen, H., ”Motion Detection Using the Randomized Hough Transform (RHT): Exploiting Gradient Information and Detecting Multiple Moving Objects,” IEE Proceedings--- Vision, Image and Signal Processing, Vol. 143, No. 6, 1996, pp. 361-369.

19. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 19 Kernel of Motion Detection Using Randomized Hough Transform (MDRHT) 1.Create the sets B and C of edge points, each in one of two consecutive frames. 2.Select point pairs (b_i,b_j) and (c_i,c_j) randomly from sets B and C, respectively. 3.If the point pairs correspond, calculate the x- and y-translations dx=c_{ix}-b_{ix} and dy=c_{iy}-b_{iy} and go to Step 4; otherwise, go to Step 2. 4.Accumulate the cell A(dx,dy). 5.If the A(dx,dy) is equal to the threshold t, motion (dx,dy) has been detected; otherwise, go to Step 2.

20. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 20

21. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 21 Detecting partially deformed motion

22. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 22 Detecting multiple objects

23. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 23 Mixed pixel classification What is in a mixed pixel?: The identification of the constituent components and their proportions in a mixed pixel. For applications with large pixels and/or with large sets of mixed pixels (remote sensing). Bosdogianni, P.*, Kälviäinen, H., Petrou, M.*, and Kittler, J.*, Robust Unmixing of Large Sets of Mixed Pixels, Pattern Recognition Letters, Vol. 18, 1997, pp. 415-424. *Centre for Vision, Speech, and Signal Processing (CVSSP), University of Surrey, UK

24. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 24 Linear mixing model w = ax + by + cz –w: reflectance of a mixed pixel (known). –x, y,z: reflectances of pixels that belong to three different pure classes (known). –a,b,c: proportions of the pure classes present in the mixed pixel (unknown). Assuming that a+b+c=1, we obtain w - z = (x-z)a + (y-z)b.

25. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 25 Pure classes with mixed pixels and outliers

26. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 26 Mixel pixel classification by RHT 1.Select one quadruple (x_1,y_1,z_1,w_1) from the first band and another quadruple (x_2,y_2,z_2,w_2) from the second band of the same pixel randomly. 2.Using two selected quadruples compute one (a,b) value in the parametric (a,b) space by w - z = (x-z)a + (y-z)b. 3.Accumulate the cell A(a,b) in the accumulator space. 4.If the A(a, b) is equal to the threshold t, the parameters a and b describe the parameters of the detected proportions; otherwise continue to Step 1.

27. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 27 Advantages and questions Fast computation and the small accumulator => the use of large datasets possible. Randomized Hough Transform needs less CPU time and memory than Standard Hough Transform when datasets are large. Hough methods are more robust than classical Least Square Methods in the presence of outliers. How high threshold? => e.g. with adaptive termination rules like a variable threshold according to data. More accuracy? => e.g. by averaging several RHT processes.

28. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 28 Image Compression with Hough Feature Extraction * P. Fränti, *E. Ageenko, S. Kukkonen, H. Kälviäinen, Using Hough Transform for Context-based Image Compression in Hybrid Raster/Vector Applications, Journal Of Electrical Imaging, 2002, vol. 11, no. 2, pp. 236-245 *Department of Computer Science University of Joensuu, Finland

29. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 29 Goal: To use vector features in context- based compression of binary images Context-based compression Feature extraction using Hough transform Feature-based context modeling Feature-based filtering Results Conclusions

30. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 30 Context-based compression

31. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 31 Feature extraction using Hough transform

32. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 32 Feature-based context modeling (HTC)

33. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 33 Feature-based filtering: Near-lossless compression system (HTF-JBIG)

34. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 34 Noise removal procedure

35. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 35 Filtering procedure

36. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 36 Results of the filtering procedure

37. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 37 Original, filtered, and difference images

38. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 38 Test images: Bolt, Plan, House Chair, Module, Plus

39. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 39 Effects of the feature-based context modeling for the Bolt image

40. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 40 Storage sizes in bytes ImageHybrid compressionFiltering only Filtering + Hybrid vector raster (JBIG) raster (HTC) (HTF- JBIG) (HTF-HTC) BOLT 6,43812,96611,51410,5369,287 PLAN 2,3705,0984,5784,3253,786 HOUSE 13,39815,68813,96113,33611,553 CHAIR 16,71052,38450,14051,52948,023 MODULE 3,4687,6717,2226,4316,057 PLUS 5,26817,60917,13216,27315,739 TOTAL 47,652111,416104,547102,43094,445

41. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 41 Computation times of the HT-based compression

42. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 42 Conclusions Two methods proposed for improving compression performance –Feature image as side information for compression –Feature-based filtering for removing noise Problems –Is an exact replica of the original image always needed? –How to improve the quality of vectorizing?

43. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY Image Databases and Image Matching with Hough Features *P. Fränti, A. Mednonogov, V. Kyrki, H. Kälviäinen Content-Based Matching of Line-Drawing Images Using Hough Transform International Journal on Document Analysis and Recognition (IJDAR) 2000, vol. 3, no. 2, pp. 117-124 *Department of Computer Science, University of Joensuu, Finland

44. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY Applications of Hough Transform for image databases Content-based matching of line-drawing images using Hough Transform. Similarity of images in image databases. Hough Transform as a feature extractor. –Translation-, –rotation-, and –scale-invariant features from the accumulator matrix.

45. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY Generated 3D images: query images

46. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY Generated 3D images: test database

47. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 47 Symbol library: noisy and rotated test images