1. Automatic Advertisement Ratings Discussion Methods Problem and Motivation The goal is to automatically generate an objective score or ranking for an advertisement. Method 1 Motivation Human ratings are time consuming Human ratings are expensive Provide insight into elements of a good advertisement Applications Aid in designing advertisements Predict how well advertisements will do Difficulties Advertisements contain more than just vision elements Advertisements are subjective Goals Collect Advertisements (video and Image) Collect annotations Train Convolutional Neural Network (CNN) Compare CNN to Support Vector Machine Regression (SVR) and handcrafted features Extend to video advertisements Apply and evaluate manual attributes Apply TRECVID concept detectors Features Tested VGG-16 AlexNet Gist Annotated Scores “How much do you like this advertisement? 5 people rated from 1-10, average becomes annotated Score Issues Not enough data Annotated score metrics were too subjective Datasets No existing dataset Videos collected: 276 commercials 9 brands Geico, Heineken, Mazda, McDonalds, T-Mobile, Toyota, Burger King, Budweiser, Coca-Cola YouTube annotations: likes, views, date uploaded Images collected: 1000 advertisements 5 Product categories Cars, Clothes, Cosmetics, Food, Technology 5 people rated each advertisement from 1-10 Thank you to Dr. Shah and Dr. Lobo for overseeing this program. Acknowledgements Human surveying was time-consuming Limitations Future Work Expanding Dataset Applying other mid-level representations Results Brian Mora bmora@knights.ucf.edu University of Central Florida Dong Zhang dzhang@cs.ucf.edu University of Central Florida Amir Mazaheri amirmazaheri@knights.ucf.edu University of Central Florida For Images: Annotated Scores Advertisement CNN Predicted Score Annotated Scores Extracted Features SVR Predicted Score Proposed Method Baseline Methods Method 2 For Videos: VideoFrames Apply Detectors rankSVM Get Rankings Methods Original DataAugmented DataPCA and Augmented Data PCA New Data Alexnet0.819335780.822790420.749032950.83789653 VGG0.819335780.823347410.707827750.84354067 Gist0.822020830.820874120.743016480.84176287 SVR Results from Extracted Features (Images) Reporting Root Mean Square Error (RMSE) (the smaller the better) IterationsRMSE 1001.77769148975 2001.29043424634 7001.00233565894 Initial Results from fine-tuning Alexnet Reporting Root Mean Square Error (RMSE) (the smaller the better) Original AttributesNew AttributesVGG Features E from SVM (likes) 0.5046840.4741830.481944 E on random 0.4987090.4979620.498035 Ranking SVM Results (Videos) Learned Weights From Ranking SVM http://sellingwithinsight.com/wp-content/uploads/2016/04/car-magazine-advertisement-astonmartinadcopy.jpg https://s-media-cache-ak0.pinimg.com/236x/07/72/ca/0772ca4044c8f30959009bf9bbeb5680.jpg http://designmodo.com/wp-content/uploads/2011/01/Tech-Advertisements-21.jpg Results (top left) Alexnet Features SVR on train, (top right) Alexnet Features SVR on test, (bottom left) Manual Attributes SVR on train, (bottom right) Manual Attributes SVR on test Separate brands yielded best results, brands are different Relative Improvement (on RankSVM): 15% Raw Features (VGG) yielded best results Audio Features are very important SVR Results on Images SVR Results on Video Likes/views (Red = Ground Truth, Blue = TRECVID, yellow = VGG, Purple = Original Attributes) Each X value is the i-th image. Ideally, all of these points would be over each other) GeicoHeinekenMazdaMcDonaldT-mobileToyotaBurger KingBudweiserCoca-ColaAverage Original Attributes 0.4929630.4922220.498480.5002820.4964050.5020140.50503510.4904760.494780.49696184 Refined Attributes 0.4846670.5303330.5011930.5007140.5035480.5051850.5051851850.4962640.4942220.50236792 VGG 0.4 0.3684210.3809520.4912280.6023390.4248366010.329670.40.42193862 TRECVID 0.30.70.4315790.3263160.6140350.4678360.5032679740.3956040.4945050.47034932 E on random 0.4929630.4922220.498480.5002820.4964050.5020140.50503510.4904760.494780.49696184 Ranking SVM Results (Videos) Reporting Root Mean Square Error (RMSE) (the smaller the better) Single SVM For all brands Reporting Root Mean Square Error (RMSE) (the smaller the better) Separate SVM for each brand Relative Improvement (on RankSVM): 15%