1. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 1 Computer Vision: Fundamentals & Applications Heikki Kälviäinen Professor Computer Science Laboratory of Information Processing Heikki.Kalviainen@lut.fi http/www.it.lut.fi/~kalviai

2. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 2 Lappeenranta University of Technology, Finland

3. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 3

4. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 4 Contents Fundamentals of computer vision –Digital image processing –Pattern recognition & Machine vision –Fundamental steps in image processing Industrial applications –Application areas –Applications in Finland

5. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 5 Digital Image Processing R. C. Gonzalez & R.E. Woods, Digital Image Processing, Addison-Wesley, 1993 : “A digital image is an image f(x,y) that has been discretized both in spatial coordinates and brightness” f(x,y) is a 2D intensity function where x and y are spatial coordinates and the value of f at any point (x,y) is proportional to the brightness of the image at the point

6. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 6 Digital Image Processing A digital image consists of pixels (also called image elements, picture elements) For example: an image of a 256 x 256 array with 256 gray-levels (8 bits: 0 black, 255 white) –Binary images: only two values –Gray-level images: e.g. 256 values –Color images: three color components (e.g. RGB) –Spectral images: several components

7. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 7

8. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 8 Pattern Recognition and Machine Vision A digital image is just a set of pixels ? Pattern recognition = measurements and observations from natural scenes and their automatic analysis and recognition Computer vision = image analysis using pattern recognition techniques Machine vision = application oriented image analysis

9. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 9 Fundamental Steps in Image Processing Image acquisition Preprocessing Segmentation Representation and description Recognition and interpretation Image processing system

10. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 10 Robot Vision: Handling of Sheets in a Workshop

11. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 11 Robotized Handling of Objects

12. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 12 Video Video Automatic Cheese Factory (RTS, Ltd.) VideoVideo

13. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 13 Requirements for Successful Applications Fast – No delays Accurate – Assist/replace human vision Not too expensive – Return on investment Easy to implement and to use – End users are experts in their own field only!

14. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 14 Applications (some areas) Recognition, classification, and tracking of objects –Face recognition, fingerprint detection –Speech recognition, motion detection –OCR, document processing, image databases Industrial applications –Visual quality control –Process automation –Robotics

15. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 15 Applications (some areas) Telecommunications – Image compression, video technology. Military applications –Tracking of objects, surveillance systems. Remote Sensing –Analysis of satellite images, classification of airplanes,spying, weather forecasts, forest fire detection, missile control.

16. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 16 Applications (some areas) Medical image processing – X-ray images, ultrasound images, images of cells, chromosomes, proteins. –Detection of tumors, cancer; assistance in operations. Chemistry, Biology, Physics, Astronomy – DNA, molecules, particles, planets.

17. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 17 Applications in Finland TEKES technology programs –Machine Vision (1992-1996) & Intelligent and Adaptive Systems Applications (1995-2000) & Intelligent Automation Systems (2001-2004) Applications of –process control –robot vision –quality control in electronics, metal, forest, food manufacturing, etc., industry & applications in business

18. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 18 Visual Quality Control in Steel Manufacturing

19. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 19 Robot Positioning: Deflection Compensation

20. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 20 Visual Inspection on Wooden Surfaces

21. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 21 Visual Inspection on Wooden Surfaces

22. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 22 Sorting potatoes (Euroelektro International, Ltd.) Classification problem in automatic peeling: 1. Ready for a customer. 2. More peeling needed. 3. Manual peeling needed. 4. Totally rejected.

23. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 23 Detection of Forest Fires

24. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 24 Punch Press Quality Assurance (Lillbacka, Ltd., Nokia, Ltd., Abloy, Ltd., etc.)

25. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 25 Multispectral Images = Multicomponent Images

26. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 26 Multispectral Images: Compression

27. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 27 Multispectral Images: Video

28. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 28 Sorting Ceramic Tiles: Laboratory Experiments with RGB and Spectral Features (SPECIM, Ltd.)

29. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 29 Sorting Ceramic Tiles: Test Material Class 1 Class 2 Class 3 Tile 1Tile 2Tile 3Tile 4Tile 5 ::::

30. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 30 Sorting Ceramic Tiles: Classification

31. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 31 Molecular Computing: Bacteriorhodopsin-based Color Sensors Molecule structureFotocycle

32. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 32 Molecular Computing: Bacterial Camera Matrix elementCamera

33. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 33 Molecular Computing: Color Sensors ResponsesSelf-Organizing Map (SOM)

34. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 34 Applications of Hough Transform Randomized Hough Transform (RHT) Curve detection Motion detection Mixed pixel classification Image compression Vanishing point detection Image databases etc.

35. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 35 Compression, Similarity, Matching, Object Recognition

36. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 36 Feature extraction using Hough Transform

37. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 37 More complex images

38. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 38 Detecting partially deformed motion

39. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 39 Detecting multiple objects

40. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 40 Other Applications Industrial Robot for Windscreen Grinding Quality Control in Printing Industry Punch Press Quality Assurance Classification of Parquet Pieces Controlled Wood Cutting Automatic Cheese Production Detection of Food Fatness Baking Better Biscuits Sorting Ceramic Tiles Multispectral Video Image databases (see, e.g. PICSOM, http://www.cis.hut.fi/resea rch/demos.shtml)

41. LAPPEENRANTA UNIVERSITY OF TECHNOLOGY THE DEPARTMENT OF INFORMATION TECHNOLOGY 41 References R. C. Gonzalez & R.E. Woods, Digital Image Processing, Addison-Wesley, 1993. See more references for example at http://www.it.lut.fi/opetus/99-00/010588000/refs.html Applications: –Finland: Machine Vision 1992-1996. TEKES Technology Programme Report 15/96. Final Report, 1996. –LUT: http://www.it.lut.fi/research/ip/appl.html –Systems: for example, RTS Group (www.rts-group.com)