1. Ramakrishna Lanka Ramakrishna.lanka@mavs.uta.edu 1001126160
IMAGE ANALYSIS AND OBJECT DETECTION-RECOGNITION USING COMPUTER VISION ALGORITHMS
Ramakrishna Lanka

2. Contents Introduction to images and image processing
Overview of computer vision
Overview of image and video processing workflow
Image enhancement
Computer vision search algorithms and techniques
Tools to be used
References

3. PROPOSAL
The objective of this project is to implement computer vision algorithms to analyze an image and detect objects in the image based on object reference images.

4. Acronyms 3D – Three dimensional
CVIU – Computer vision and image understanding
FAST – Features from accelerated segment test
MSER – Maximally stable extremal regions
PCB – Printed circuit board
RANSAC – Random sample consensus
RGB – Red, green and blue
SURF – Speeded up robust features
VTP – Virtual terrain Project

5. What is an Image ?
Pictorial reproduction of an object in physical or digital space.
A digital Image is made up of Pixels.
More the pixels, much detailed is the Image as shown in figure(1).
Resolution of an Image is measured by the number of pixels it consists of. Ex: 1366x768
Fig(1). Illustration of image pixels. [19]

6. Need for Image and video processing.
Computer Vision: Object/Activity detection as shown in figure (15) and figure (16).
Medical and biological image processing as shown in figure (10) and figure (11).
After effects: Adding effects and image correction as shown in figure (2).
Efficient use of media in Consumer multimedia products: Storage, display and transfer.
Fig(2). Illustration of image after effects.

7. OVERVIEW OF COMPUTER VISION
Vision has given humans the natural ability to picture the world around in a three-dimensional (3D) space.
Computer vision gives the man-made device the ability to perceive the world.
The goal of computer vision is to extract meaningful information from the images and videos that the image sensors capture.

8. OVERVIEW OF THE IMAGE AND VIDEO PROCESSING WORKFLOW
Fig(2). Image and video processing workflow. [2]

9. OVERVIEW OF THE IMAGE AND VIDEO PROCESSING WORKFLOW
The raw images from the source may contain unwanted image source characteristics or noise.
Preprocessing to enhance the images will prepare it for further processing based on the requirement.
Image enhancement can be achieved using various filtering techniques and transforms.

10. Fig(3). Original image of a PCB [21]
Negative of an image
Fig(3). Original image of a PCB [21]
Fig(4). Histogram of the original PCB image
N(r, c) = 255 – I(r, c) where 0 <= r <= R and 0 <= c <= C (1.1)
Where, N is the negative of image I, both with dimension R x C.

11. Fig(7). Negative image of the PCB
Negative of an image
Fig(7). Negative image of the PCB
Fig(8). Histogram of the negative PCB image

12. Negative of an image
Fig(10). Original image of a knee scan [22]
Fig(11). Negative image of the knee scan figure (10)
Fig(9). Histogram of the knee scan
Fig(12). Histogram of the negative knee scan

13. Thresholding Transformation
(1.2)
Where s is the value of the resultant pixel intensity and r is the actual pixel intensity.
Fig(13). Image of some coins [28]
Fig(14). Threshold image of figure(13)

14. Thresholding Transformation
Multilevel thresholding
Fig(15). Image of obstacles [24]
Fig(16). Multilevel threshold image of figure(15)

15. Thresholding Transformation
Thresholding – color labelling
Fig(17). Image of my cluttered desk
Fig(18). Threshold segmentation of figure(17)

16. Powers-Law Transformations / Gamma correction
s = crγ (1.3)
Where, s is the corrected pixel intensity, r is the actual pixel intensity, c and γ are gamma correction constants.
Fig(19). Image of the gamma palette [27]
Fig(20). Gamma corrected image of figure(19) with γ=0.7
Fig(21). Gamma corrected image of figure(19) with γ=0.5

17. Histogram Equalization
h(rk) = nk (1.4)
Where, h(rk) is the histogram function, rk is the intensity value and nk is the number of pixels with the intensity rk.[4]

18. Computer Vision search algorithms and techniques
Edge feature detection
The Sobel operator
The Prewitt operators
The Robert operator
Laplacian/Laplacian of Gaussian operator
Canny edge detection
Fig(24). Reference object image [25]
Fig(25). Object in scene image [25]

19. Edge feature detection
The Sobel operator
(1.5)
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]
Figure(26) and figure(27) show the edge detections of figure(24) and figure(25).
Fig(26). Sobel edge detection of figure(24)
Fig(27). Sobel edge detection of figure(25)

20. Edge feature detection
The Prewitt operators
(1.6)
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]
Figure(28) and figure(29) show the edge detections of figure(24) and figure(25).
Fig(28). Prewitt edge detection of figure(24)
Fig(29). Prewitt edge detection of figure(25)

21. Edge feature detection
The Robert operator
(1.7)
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]
Figure(30) and figure(31) show the edge detections of figure(24) and figure (25).

22. Edge feature detection
The Laplacian/Laplacian of Gaussian operator
The Laplacian of an image highlights regions of rapid intensity change.
The Laplacian is often applied to an image that has first been smoothed with something approximating a Gaussian smoothing filter in order to reduce its sensitivity to noise. [14][15]

23. Edge feature detection
Canny edge detection
Low error rate.
Edge points should be well localized.
Single edge point response.

24. Computer Vision search algorithms and techniques
Local point/region detection
Corner point detection – FAST (Features from accelerated segment test )
MSER(Maximally stable extremal regions) feature detection The Robert operator
SURF(Speeded up robust features) feature detection.
Fig(24). Reference object image [25]
Fig(25). Object in scene image [25]

25. Local point/region feature detection
Corner point feature detection
- FAST corner detector uses a circle of 16 pixels (a Bresenham circle of radius 3) to classify whether a candidate point p is actually a corner. [31]

26. Local point/region feature detection
Maximally stable extremal regions (MSER)
MSER is a method for blob detection in images. The MSER algorithm extracts from an image a number of co-variant regions, called MSERs: an MSER is a stable connected component of some gray-level sets of the image. [29]

27. Local point/region feature detection
Speeded up robust features (SURF)
SURF is a fast and robust algorithm for local, similarity invariant representation and comparison.
SURF selects interest points of an image from the salient features

28. Object recognition Recognition [9][10]
Recognition is a process that follows detection and/or segmentation.
Recognition approaches in this project,
Object feature recognition [10]
Recognition with point feature matching
Recognition with edge feature matching
Object Segmentation recognition [1]
Recognition with segmentation

29. Object feature recognition
Recognition with point feature matching [1][10]
Match the extracted feature points of reference and scene images.
The RANSAC (Random sample consensus ) algorithm works by estimating the best set of point matches between the images. This is shown in figure(43).
Fig(43). Feature matching between figure(25) and figure(26)

30. Object feature recognition
Recognition with point feature matching [1][10]
Fig(44). Object in figure(25) recognized in figure(26)

31. Project status
The aim of this project is to analyze and enhance images; and design a system to identify objects in an image. The identity of the objects will be achieved by using feature detection and recognition based on the features of the reference objects that the system would have learnt before recognition.
Image analysis : Done
Image Enhancement : Done
Feature detection : Done
Feature Extraction : Done
Segmentation : Done
Performance comparison : Ongoing
Recognition with point feature matching : Done
Recognition with edge feature matching : Done (yet to document)
Recognition with segmentation : Done (yet to document)
Multiple object recognition : Done (yet to document)

32. Tools being used Matlab – Image processing tool box [13]
Matlab – Computer vision tool box [9]
Matlab – R2013a
OpenCV v [10]
Microsoft Visual Studio v2010
Windows bit
Central processor – Intel® Core™ i Ghz
Graphics processor – NVidia GT 720M 2GB
Memory – 8.00 GB

33. References
[1] R. Szeliski, “Computer Vision: Algorithms and Applications”, 2010 Springer, Draft: 3, September [2] A. Thé, “Image Processing Made Easy”, MathWorks Videos and Webinars, Recorded: 15 Oct [3] VTP software tutorials webpage (Source for hazy image): [4] R. Maini and H. Aggarwal, “A Comprehensive Review of Image Enhancement Techniques”, Journal of Computing, Volume 2, Issue 3, pp 8-12 March : [5] T. Gupta, “Seeing through Chaos: Noise Filtering”, ahumaninmachinesworld.blogspot, January 2014 (Internet Blog on Noise Filtering): [6] Computer Vision Algorithms, “COMPUTER VISION TUTORIAL”. Link: [7] H. Spont´on and J. Cardelino, “A Review of Classic Edge Detectors”, Image Processing On Line, Preprint Issue, pp 1-8, June Link:

34. References
[8] ‘Performance criteria’, Document in the Course website of EE5356 Digital Image processing (rate the performance of the filtered/enhanced image). Link: [9] Matlab computer vision toolbox: [10] Open source Computer vision software: OpenCV v3.0 – download: [11] Open source Computer vision software: OpenCV v3.0 – Reference Manual: [12] R.C. Gonzalez, R.E.Woods and S.L. Eddins, “Digital Image Processing Using MATLAB”, Pearson Education, [13] Matlab image processing toolbox: [14] A. K. Jain, “Fundamentals of Digital Image Processing”. Englewood Cliffs, NJ: Prentice Hall, 1989.

35. References
[15] R.C. Gonzalez and R.E. Woods, “Digital Image Processing”. 3nd edition, Prentice Hall, [16] Line detection image: [17] Open source Computer vision software: OpenCV v3.0 – User Guide: [18] Image segmentation example link: [19] Test images: [20] Computer vision test image: [21] PCB image:

36. References
[22] Medical image – knee: [23] Geometric shapes image: [24] Need for image processing, computer vision path planning image: [25] Stop reference image: [26] Stop scene image: [27] Gamma correction image: [28] Coins image:

37. References
[29] Ppt on MSER : [30] H. Bay, A. Ess, T.Tuytelaars and L.V. Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp , June [31] PowerPoint presentation on FAST: