Automatic Advertisement Ratings Discussion Methods Problem and Motivation The goal is to automatically generate an objective score or ranking for an advertisement. Method 1 Motivation Human ratings are time consuming Human ratings are expensive Provide insight into elements of a good advertisement Applications Aid in designing advertisements Predict how well advertisements will do Difficulties Advertisements contain more than just vision elements Advertisements are subjective Goals Collect Advertisements (video and Image) Collect annotations Train Convolutional Neural Network (CNN) Compare CNN to Support Vector Machine Regression (SVR) and handcrafted features Extend to video advertisements Apply and evaluate manual attributes Apply TRECVID concept detectors Features Tested VGG-16 AlexNet Gist Annotated Scores “How much do you like this advertisement? 5 people rated from 1-10, average becomes annotated Score Issues Not enough data Annotated score metrics were too subjective Datasets No existing dataset Videos collected: 276 commercials 9 brands Geico, Heineken, Mazda, McDonalds, T-Mobile, Toyota, Burger King, Budweiser, Coca-Cola YouTube annotations: likes, views, date uploaded Images collected: 1000 advertisements 5 Product categories Cars, Clothes, Cosmetics, Food, Technology 5 people rated each advertisement from 1-10 Thank you to Dr. Shah and Dr. Lobo for overseeing this program. Acknowledgements Human surveying was time-consuming Limitations Future Work Expanding Dataset Applying other mid-level representations Results Brian Mora University of Central Florida Dong Zhang University of Central Florida Amir Mazaheri University of Central Florida For Images: Annotated Scores Advertisement CNN Predicted Score Annotated Scores Extracted Features SVR Predicted Score Proposed Method Baseline Methods Method 2 For Videos: VideoFrames Apply Detectors rankSVM Get Rankings Methods Original DataAugmented DataPCA and Augmented Data PCA New Data Alexnet VGG Gist SVR Results from Extracted Features (Images) Reporting Root Mean Square Error (RMSE) (the smaller the better) IterationsRMSE Initial Results from fine-tuning Alexnet Reporting Root Mean Square Error (RMSE) (the smaller the better) Original AttributesNew AttributesVGG Features E from SVM (likes) E on random Ranking SVM Results (Videos) Learned Weights From Ranking SVM Results (top left) Alexnet Features SVR on train, (top right) Alexnet Features SVR on test, (bottom left) Manual Attributes SVR on train, (bottom right) Manual Attributes SVR on test Separate brands yielded best results, brands are different Relative Improvement (on RankSVM): 15% Raw Features (VGG) yielded best results Audio Features are very important SVR Results on Images SVR Results on Video Likes/views (Red = Ground Truth, Blue = TRECVID, yellow = VGG, Purple = Original Attributes) Each X value is the i-th image. Ideally, all of these points would be over each other) GeicoHeinekenMazdaMcDonaldT-mobileToyotaBurger KingBudweiserCoca-ColaAverage Original Attributes Refined Attributes VGG TRECVID E on random Ranking SVM Results (Videos) Reporting Root Mean Square Error (RMSE) (the smaller the better) Single SVM For all brands Reporting Root Mean Square Error (RMSE) (the smaller the better) Separate SVM for each brand Relative Improvement (on RankSVM): 15%