1. System for Comparing Images Allison Deal April 27, 2004

2. Problem The human eye can determine similarities among images easily, but they cannot do this for large numbers of pictures. Humans can miss subtle patterns among images. It can be hard to find trends among images without looking through most or all of them.

3. Specific Problem Analyze Chinese art images from three different time periods and 23 different artists. Find similarities among artists paintings, color choices, and styles. Uncover any hidden structures in the paintings, that only the computer can find.

4. Goal Design a system that uses automated computations to compare images. This system can be put to use by people interested in learning about artists’ painting styles.

5. Overview Create models for each picture. Compute the distances between all pairs of pictures. Use Multi-Dimensional Scaling for visualization purposes. Analyze the data.

6. Mixture Models for Colors Use K-means to determine the main colors of each image. Algorithm converges once distances are within a certain small threshold. (1.0e-4) Each pixel of the image is assigned to a specific cluster based on its color similarity.

7. Mixture Models for Colors Determine the background color and remove it from the analysis because it does not provide useful information. Use K-means results to compute mean and covariance vectors and weights for each cluster of each picture. f(x) =  N i=1 a i  (x |  i,  i )

8. Mixture Models for Colors The EM algorithm would have been more accurate in this context because of its accuracy, but K-means is faster for computational purposes.

9. Mixture Models for Colors Gao Fenghan 4 clusters 0.097 [54.17 52.74 51.56] 0.474 [159.39 149.95 137.06] 0.263 [129.92 122.64 112.89] 0.166 [98.30 92.53 85.74]

10. Mixture Models for Colors Chen Hongshou 2 clusters 0.295 [106.37 82.09 45.32] 0.705 [138.05 109.24 59.16]

11. Mixture Models for Colors Zhang Daqian 5 clusters 0.065[85.39 79.46 54.42] 0.290 [165.58 139.72 78.82] 0.247 [140.44 144.21 103.25] 0.176 [115.87 119.18 81.35] 0.222 [181.11 164.36 120.64]

12. Distance Calculations Kullback-Leibler distance measures difference between two probability distributions (f(x) & g(x)) Difficult to calculate because of integration. D(f ||g) =  f(x) * log [ f(x)/g(x) ] dx

13. Distance Calculations Use Monte-Carlo Method instead generate random samples from a distribution defined by f(x), g(x) Use altered calculation of Kullback- Leibler Distance. D(f ||g)  1/n *  log [ f(x j )/ g(x j ) ]

14. Distance Calculations This distance is not symmetric, so we calculated D(f || g) and D(g || f) and took the average. ½ { D(f ||g) + D(g||f) } By doing this for each picture, we can see where similarities exist and gain more information about the artists.

15. Multi-Dimensional Scaling To visualize the result, we can use Multi-Dimensional Scaling and embed the information into a vector space. Dr. Li is working more on this part of our project.

16. Results 1177 images from 23 artists and 4 different dynasties Around 50 images per artist Focus on 10 artists Dimensions of images: (100 to 150 pixels) x (around 200)

17. Results (1) Jin Nong: two-16, three-15, four-11, five-3 (2) Shen Zhou: three-20, four-18, five-8 (3) Gao Fenghan: two-2, three-8, four-23, five-10, six-4 (4) Chen Hongshou:one-1, two-17, three-20, four-6, five-2 (5) Qi Baishi: two-2, three-19, four-14, five-15, six-10, seven-2, eight-2 (6) Zhang Daqian: two-1, three-4, four-16, five-39, six-24, seven-11, eight-1 (7) Chou Ying: two-4, three-10, four-7, five-5, six-5, seven-2, eight-1, nine-4, ten-2 (8) Dong Qichang: two-4, three-23, four-15, five-3, six-1 (9) Huang Shen: two-5, three-14, four-11, five-6 (10) Yun Shouping: two-1, three-10, four-19, five-6, six-7, eight-1, ten-2 (# of clusters - number of pictures with this many clusters)

18. Results Ming Dynasty all tended to not use many colors (often used ink) (2) Shen Zhou: three-20, four-18, five-8 (4) Chen Hongshou:one-1, two-17, three-20, four-6, five-2 (8) Dong Qichang: two-4, three-23, four-15, five-3, six-1

19. Results Qing Dynasty artists began to use more colors (1) Jin Nong: two-16, three-15, four-11, five-3 (3) Gao Fenghan: two-2, three-8, four-23, five-10, six-4 (7) Chou Ying: two-4, three-10, four-7, five-5, six-5, seven-2, eight-1, nine-4, ten-2 (9) Huang Shen: two-5, three-14, four-11, five-6 (10) Yun Shouping: two-1, three-10, four-19, five-6, six- 7, eight-1, ten-2

20. Results After the Qing Dynasty more variation and more colors being used (5) Qi Baishi: two-2, three-19, four-14, five-15, six-10, seven-2, eight-2 (6) Zhang Daqian: two-1, three-4, four-16, five-39, six- 24, seven-11, eight-1

21. Results (9) Huang Shen 36 3.500 0.941 (10) Yun Shouping 45 4.522 1.656 Number= number of pictures, Mean=mean number of clusters, Variation= variation in number of clusters

22. Results Zhang Daqian used the most colors, but his variation was not that great compared to other artists. Chou Ying had the greatest amount of variation and the second highest mean. Many of the other artists tended not to vary much from painting to painting; they have a style and stick with it.

23. Summary We can learn about artists’ styles without looking through all of their paintings by using automated computations.

24. Future Work Instead of using the RGB colorspace, it might be better to use an LUV colorspace. The LUV colorspace is easier for the human eye to perceive.