1. Artificial Neural Network For Automated Prediction of Popularity of Digitized Images David Oranchak doranchak@gmail.com

2. Objective Flickr.com ranks photographs based on their popularity in the sites user base. Interesting: An image with high rank Not Interesting: An image with low or nonexistent rank For any image, can a neural network predict which group it belongs to?

3. Approach Obtain sample images from Flickr.com 559 total samples in the training set Very Interesting: Ranking in the top 25 Somewhat Interesting: Ranking between 300 and 500. Not Interesting: No ranking data assigned by Flickr

4. Very Interesting: 25 samples

5. Somewhat Interesting: 11 samples

6. Not Interesting: 36 samples

7. Approach Input data sets based on original image data Raw pixel data, resampled for performance reasons 10x10 RGB pixels 20x20 Grayscale pixels Color analysis data One-dimensional color counts (histogram) RGB: three channels, 256 entries per channel Gray scale: one channel (luminosity), 256 entries Texture data Contrast, correlation (inertia), dissimilarity, energy, entropy, homogeneity, correlation matrix sum, symmetry Input data derived using JIU, a free set of Java image tools

8. Approach Select a suitable neural network architecture Feedforward backprop architecture? Result: difficult to train based on input data Hard to determine suitable number of hidden neurons Kohonen unsupervised learning? Result: outputs do not naturally cluster based on interestingness No mapping between clusters and desired outputs. Counter Propagation Network? Result: Very easy to train on input data.

9. Approach Training the CPN 559 input patterns 221 patterns for Very Interesting 88 patterns for Somewhat Interesting 250 patterns for Not Interesting Network simulated using CPN algorithm in JavaNNS, the Java-based successor to SNNS. Five networks trained successfully; one for each type of input Raw RGB pixel data, raw gray scale pixel data, 1D RGB histogram, 1D gray scale histogram, texture

10. Experiment 1: Comparison against Flickr images with known rankings 2381 images from 67 different days obtained from Flickr 1373 Very Interesting images 557 Somewhat Interesting images 451 Not Interesting images

11. Experiment 1: Comparison against Flickr images with known rankings Results: 32% error rate when at least one network classifies images as Very Interesting 28% error rate when at least two networks classify images as Very Interesting 23% error rate when at least three networks classify images as Very Interesting 14% error rate when at least four networks classify images as Very Interesting 3% error rate when all five networks classify images as Very Interesting

12. Experiment 1: Comparison against Flickr images with known rankings Results are greatly improved when we combine the categories Very Interesting and Somewhat Interesting into a single category: Interesting When one network classifies: 9% error rate When two networks classify: 9% error rate When three networks classify: 7% error rate When four networks classify: 4% error rate When five networks classify: 2% error rate Downside: As number of networks go up to reduce noise, number of missed Interesting photos goes up.

13. Experiment 2: Flickr photos with unknown rankings 250 photos sampled at random from recently uploaded Flickr photos All five networks classify Interesting for 14 of the 250 photos

14. Experiment 2: Flickr photos with unknown rankings Result

15. Experiment 2: Flickr photos with unknown rankings Relaxing the constraint to 4 out of 5 networks produces 57 images

16. Experiment 2: Flickr photos with unknown rankings

17. Very subjective results. In my opinion, most of the photos are interesting!

18. Experiment 3: Personal photo collection 2912 samples from personal photo collection When all 5 networks classify Interesting, 98 images result. Flickr results are better. Personal collection experiment resulted in many ordinary-looking photos. Test data setup may contribute to lack of success in this case (resizing of input photos, differences between Flickr image management and personal photo formats)

19. Conclusions Current CPN technique is very successful within the Flick image data at locating interesting photographs Further experimentation must be performed to improve success in locating interesting photographs outside of Flick More experimentation and refinement must be done to improve detection rates and reduce false positives