© 2006 Nielsen BuzzMetrics, A VNU business affiliate Natalie Glance Senior Research Scientist Nielsen BuzzMetrics

© 2006 Nielsen BuzzMetrics, A VNU business affiliate Background  Nielsen BuzzMetrics aggregates consumer opinion expressed in message boards, weblogs, Usenet and other online discussions  Parent company behind BlogPulse, blog search and analytics website

© 2006 Nielsen BuzzMetrics, A VNU business affiliate What drives weblog spam?  Same goal as any other website spam: SEO  Weblog hosts provide:  Free hosting for link farms to promote affiliate sites  Free hosting for web pages with sponsored ads  Types of weblog spam  spam blogs – (pollute ping servers)  spam comments on legitimate blogs  spam trackback pings to legitimate blogs

© 2006 Nielsen BuzzMetrics, A VNU business affiliate Collateral damage: blog search result contamination  Search results for ‘mortgage’ :

© 2006 Nielsen BuzzMetrics, A VNU business affiliate Collateral damage: trend graphs  Explain the peaks: are they real or artifacts of spam?

© 2006 Nielsen BuzzMetrics, A VNU business affiliate Collateral damage: real-time monitoring  Spikes in keyword clusters  2006/07/28 10:39 a.m. {deleted myspace account}  2006/07/27 10:55 a.m. {landis tested yesterday}  2006/08/07 3:22 a.m. {investing debt directory}  2006/08/07 6:54 a.m. {adsense cents makers}  2006/08/07 1:11 p.m. {wwdc keynote}  Breaking news or spam attack?

© 2006 Nielsen BuzzMetrics, A VNU business affiliate Spam filtering challenges  Different analytics, different trade-offs  weblog search requirements: high coverage, clean results, minimize false positives  trend search: high precision to eliminate spurious artifacts  real-time monitoring: high coverage w/human oversight  Different timeframes, different approaches  real-time search: highly efficient classification algorithms; automated identification of spam attacks  historic search: offline spam identification can use combination of approaches; sandbox for new weblogs