1. Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009

2. Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information – Kim, Faloutsos and Heberts Approach using global information Comparison of Two Approaches Summary Questions and Comments

3. Background: Complex Network Source: cs790: complex network lecture

4. Background: Image Source: CS674 Image Processing Lecture

5. Background: Image Processing Source: CS674 Image Processing Lecture

6. Background: Image Representation Source: CS674 Image Processing Lecture

7. Outline Background Motivation

8. Belief: Computer vision is one of the most difficult problem remains, how can we represent and characterize image in the way of complex network so that we analysis it? For a given problem, if it can be described in the way of mathematics, it is half way to solve the problem.

9. Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information

10. CS: Backes Approach Construction of graph, – Vertices: points of shape boundary are modeled as fully connected network, – Weight: the Euclidean distance d – through a sequential thresholds T l (d< T l ), the fully connected network becomes a dynamic complex network, the topological features of the growth of the dynamic network are used as a shape descriptor (or signature)

11. CS: Backes Approach

12. Properties of the complex network – High clustering coefficient – The small world property

13. CS: Backes Approach

14. Dynamic evolution signature F: T T where T ini and T Q, respectively, the initial and final threshold

15. CS: Backes Approach Characterization – Degree descriptor k μ average degree, K k max degree

16. CS: Backes Approach Evolution by a threshold T=0.1,.15,.20

17. CS: Backes Approach Process of extraction of degree descriptor from an Image

18. CS: Backes Approach Advantage of Degree Descriptors – Rotation and scale inveriance – Noise tolerance – Robustness

19. CS: Backes Approach Representation of rotate invariance

20. CS: Backes Approach Representation of scale invariance

21. CS: Backes Approach Characterization – Joint Degree descriptor Is the concatenation of the entropy(H), energy(E), and average joint degree(P) at each instant threshold T

22. CS: Backes Approach Advantage of Joint Degree Descriptors – Rotation and scale inveriance – Noise tolerance – Robustness – Normalization of vertex is irrelevant because the joint degree concerns the probability distribution P(ki,k)i

23. CS: Backes Approach

28. Weakness of Backes Approach: – Initial and final threshold

29. Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information – Kim, Faloutsos and Heberts Approach using global information

30. CS: Kims Approach Construct Visual Similarity Network (VSN) – Vertices (V): features of from training images – Edges (E): link features that matched across images – Weights (W): consistence of correspondence with all other correspondences in matching image I a and I b VSN = (V, E, W)

31. CS: Kims Approach Construction of VSN – Vertices: can be any unit of local visual information. In this approach, features detected using Harris-Affine point detector and the SIFT descriptor

32. CS: Kims Approach Construction of VSN – Edges: established between features in different images. Spectral matching algorithm is used to each pair of image (I a, I b ) A new edge is established between feature a i and b j

33. CS: Kims Approach

35. Construction of VSN – Edge weights – M n*n is a spare weight matrix, M(a i, b j ) is the weight value

36. A small part of VSN

37. CS: Kims Approach Characterization – Ranking of information Remove noisy Measure the importance P is the PageRank vector

38. CS: Kims Approach Characterization – Structural similarity similar nodes are highly likely to exhibit similar link structures in the graph p.4 The similarity is computed by using link analysis algorithm

39. CS: Kims Approach Characterization Link analysis algorithm Given a VSN G, a node a i, the neighborhood subgraph Gai either pointed to a i or point to by a i M, the adjacency matrix of G a i.

40. CS: Kims Approach The left image is extracted features, the right image shows top20% high-ranked features

41. CS: Kims Approach Weakness of Kims Approach – Using threshold in computing edge weights – Mystery constant α =0.1 – Category partition to pre-determined K groups – The difference of objects appearance in the training data set is too big, make the conclusion weak

42. Outline Background Motivation Current States (CS): Comparison of Two Approaches

43. Comparison Backess Approach – Unsupervised approach – using local information – Dynamic complex network – More task on complex network, less work on image processing Kims Approach – Supervised approach – using global information – Static complex network – More work on image processing, less work on complex network Both using threshold, but Backes approach based on initial and final value,

44. Outline Background Motivation Current States (CS): Comparison of Two Approaches Summary

45. In both approaches using complex network for representation and characterization of image, – provide a unique way for object classification and analysis, – present better results than traditional and state- of-art methods, – demonstrate the potential of complex network analysis to computer vision.

46. Questions and Comments

47. Thanks