1. Scott Clements, Monash University Software Engineering, Copyright 2003. Web Document Analysis- Improving Search Technology using Image Processing Scott Clements Bachelor of Software Engineering Monash University www.csse.monash.edu.au/~sdcle1/ Supervisor: Dr. Sid Ray

2. Scott Clements, Monash University Software Engineering, Copyright 2003. Interests and Expertise Dr. Sid Ray Image Processing expert Scott Clements Internet Technology Software Engineering Database Management Interface Design

3. Scott Clements, Monash University Software Engineering, Copyright 2003. Union of Expertise Engineering a product which uses: Image processing & Internet Technology

4. Scott Clements, Monash University Software Engineering, Copyright 2003. Primary Goals To improve search quality using Image Processing. To investigate – Image histogram matching to find similar images – Colour predominance in images

5. Scott Clements, Monash University Software Engineering, Copyright 2003. Secondary Goals To Engineer a product which has industry potential. –Project Management –Interface Design –Database Management –Information Retrieval

6. Scott Clements, Monash University Software Engineering, Copyright 2003. Background Popular search technology [mcbryan94, brin98, pinkerton00] – Text based – Quality of results can be poor – Difficult to find images Multimedia search technology [ogle95, smith97] – Text, image and video based – Poor interface design Aimed at Image Processing experts – Good use of Databases Management systems

7. Scott Clements, Monash University Software Engineering, Copyright 2003. Software Engineering Methods Stages Initial program: Grey-scale image matching Refinement 1: Colour image matching Refinement 2: Colour predominance image matching

8. Scott Clements, Monash University Software Engineering, Copyright 2003. Image processing technique Data types: Histogram Data Colour Predominance Data

9. Scott Clements, Monash University Software Engineering, Copyright 2003. System Architecture

10. Scott Clements, Monash University Software Engineering, Copyright 2003. System Architecture continued

11. Scott Clements, Monash University Software Engineering, Copyright 2003. Colour histogram matching Method: –Using: Group 16 configuration Total difference Algorithm Requirements –Database design –Histogram analysis Investigate: –Interface design –Relevance Feedback

12. Scott Clements, Monash University Software Engineering, Copyright 2003. Histograms (Group 16 Configuration) Colour Histograms -Count the number of occurrences of each colour intensity -256 intensities for each RGB component. (24bit image) -Insert this information into the database Problem: Excessive amount of information Solution: Convert to Group 16 Configuration.

13. Scott Clements, Monash University Software Engineering, Copyright 2003. Database Design

14. Scott Clements, Monash University Software Engineering, Copyright 2003. Algorithm Aim: To find other similar images Method: Compare each of the histograms with the query histogram Algorithm: Total difference

15. Scott Clements, Monash University Software Engineering, Copyright 2003. Total Difference Algorithm -Query Image versus images in the database -Compare each histogram -Find the positive difference between each histogram (Total Difference) -Convert 0-300% range to a similarity rating between 0-100% -Return the results which are within a user defined similarity rating

16. Scott Clements, Monash University Software Engineering, Copyright 2003. Interface Design

17. Scott Clements, Monash University Software Engineering, Copyright 2003. Relevance Feedback User Feedback: –Clicking the similarity button –Proving interest in a particular image Relevance –Sorting results: most similar to least similar

18. Scott Clements, Monash University Software Engineering, Copyright 2003. Results and Findings MethodAccuracy Grey-scale Histogram matching64% Colour Histogram Matching84% Test Set: –Real life photos –Computer generated images Weakness –Grey-scale histogram matching. (Unacceptable results) –Images with many different colours –Spatial Arrangements –Needing to resize the images. (standardisation for histograms)

19. Scott Clements, Monash University Software Engineering, Copyright 2003. Colour predominance Assign each pixel a colour value (if possible) Found that RGB was not suitable in this case HSB was much easier to find colour ranges Method: Using an image program find the Hue, Saturation and Brightness ranges for each colour.

20. Scott Clements, Monash University Software Engineering, Copyright 2003. Algorithm Design Analysis Count each occurrence of a certain colour Convert the occurrence result to a percent of predominance between 0-100% Query Query the database to find images which have predominant colours.

21. Scott Clements, Monash University Software Engineering, Copyright 2003. Database Refinement

22. Scott Clements, Monash University Software Engineering, Copyright 2003. Interface design

23. Scott Clements, Monash University Software Engineering, Copyright 2003. Interface design continued

24. Scott Clements, Monash University Software Engineering, Copyright 2003. Relevance Feedback Not fully suitable for Colour predominance Use a subset of Relevance Feedback to improve useability Sort the result from most to least relevant

25. Scott Clements, Monash University Software Engineering, Copyright 2003. Results and Findings Test set: –Real life photos –Computer generated images –Easy method to understand for users –Less information stored in the database –Accurate and efficient method to use AlgorithmSimilarity results Colour Predominance86%

26. Scott Clements, Monash University Software Engineering, Copyright 2003. Conclusion and Applications Small to Medium sized system Example: local image database Colour histogram matching Colour predominance Medium to Large system Example: Internet search engine Only Colour predominance –More efficient –Less information to store about images –Easy to understand

27. Scott Clements, Monash University Software Engineering, Copyright 2003. Future Research Parallelism in image analysis Alternative image data for histogram matching (e.g. HSB) Replace or extend Monash Image Library (MIL) to directly support popular internet image formats. Improve the documentation for colour image manipulation in MIL. More extensive testings of colour predominance Addition of predominance levels

28. Scott Clements, Monash University Software Engineering, Copyright 2003. Questions? Are there any questions?