1. Harini Sridharan Stephen Duraski

2.  Behavioral Targeting (BT) is a technique that uses a user’s web-browsing behavior to determine which ads to display to a user.  BT is a technique used by online advertisers to increase the effectiveness of their campaigns.  As of the writing of this paper, it is underexplored in academia how much BT can truly help online advertising in search engines.

3.  First, we aim to empirically answer the question of whether Behavioral Targeting truly has the ability to help online advertising.  Second, we aim to answer the question of how much BT can help online advertising using commonly used evaluation metrics.  Finally, we aim to answer the question of which BT strategy can work better than others for ads delivery.

4.  We use 7 days’ ads click-through log data coming from a commercial search engine, dated from June 1 st to 7 th 2008.  The dataset includes web page clicks and ad clicks of users. We do not include any demographic or geographic data to be clear of any privacy concerns.

5.  “BT uses information collected on an individual's web-browsing behavior, such as the pages they have visited or the searches they have made, to select which advertisements to display to that individual. Practitioners believe this helps them deliver their online advertisements to the users who are most likely to be influenced by them.” [15]  We measured the effectiveness of BT using the click- through rate (CTR).

6. 1. Does BT truly have the ability to help online advertising? To answer this question, we validate the basic assumption of BT, i.e. whether the users who clicked the same ad always have similar browsing and search behaviors and the users who clicked different ads have relatively different Web behaviors.

7. 2. How much can BT help online advertising using commonly used evaluation metrics? To answer this question, we use the difference between ads CTR before and after applying BT strategies as the measurement, i.e. the degree of CTR improvement is considered as a measurement of how much BT can help online advertising. The statistical t-test is utilized to secure the significance of our experiment results

8. 3. What BT strategy works better than others for ads delivery? We consider two types of BT strategies, which are (1) represent user behaviors by users’ clicked pages and (2) represent user behaviors by users’ search queries respectively. In addition, how long the user behaviors have occurred in the log data is also considered for user representation.

10.  We also build the user behavioral profile by simply considering all terms that appear in a user’s search queries as his previous behaviors.  With this method we can represent each user in the Bag of Words (BOW) model.  We use Porter Stemming [3] to stem terms and then remove stop words and terms that only appear one in a user’s query text.  470,712 terms are removed and 294,208 are reserved  We use the same TFIDF indexing to index the users by query terms.

11.  Many commercial BT systems use long-term user behavior, while many others use short term behavior.  There is no existing evidence to show which strategy is better.  As a preliminary survey, we will consider 1 days’ user behavior as their short term profile, and 7 days’ user behavior as their long-term profile.

12.  We will validate and compare four different BT strategies in this paper. They are: 1. LP: using Long term user behavior all through the seven days and representing the user behavior by Pageviews 2. LQ: using Long term user behavior all through the seven days and representing the user behavior by Query terms 3. SP: using Short term user behavior (1 day) and representing user behavior by Pageviews 4. SQ: using Short term user behavior (1 day) and representing user behavior by Query terms

13.  7 days CTR data from June 1 st to 7 th 2008.  We use user IDs associated with cookies stored on the users’ OS to identify individual users.  No other user information, such as demographic or geographic data, is used.  To filter robots, any user with >100 clicks per day is filtered out.  Data covers 6,426,633 unique users and 335,170 unique ads within 7 days.  Ads with <30 clicks over 7 days are removed, leaving 17,901 ads, the results are averaged over that number.

17.  We first represent the users by their behaviors using different types of BT strategies.  After that, we group the users according to their behaviors by the commonly used clustering algorithms.  Finally, we evaluate how much BT can help online advertising by delivering ads to good user segments

18.  Within- and Between- Ads User Similarity  The basic assumption with BT is that users with similar browsing behavior will have similar interests and therefore be inclined to click the same ads.  If this is true, the similarity between users who clicked the same ad must be larger than the similarity between users who clicked different ads  Click-through rate  Once we have validated the basic assumption that similar users click similar ads we have to show that BT can help online advertising.  Ad performance is generally measured with either click-through rate or revenue, since advertiser revenue is difficult for us to obtain, we use CTR.  If user segments exist where the CTR substantially improves over the same ad shown without user segmentation, then BT is valuable.

22. R LP 0.14170.025228.9217 LQ 0.22390.019644.2908 SP 0.15320.028124.5086 SQ 0.25940.016191.1890  The fact that average Sw is greater than average Sb implies that users who clicked the same ad are more similar than the ones that clicked different ads. To validate whether the difference between Sw and Sb, statistical paired t-test was implemented and the results were all less than 0.05. This in turn implied that, statistically, the within ad user similarity is always larger than the between ads similarity.

28.  In this work, we try and provide a systematic study to understand the click-through log of a commercial search engine, so the one can validate and compare different strategies of behavioral targeting.  This also happens to be the first systematic study for BT on real world ads click-through in academia.

29. 1. Demographic and geographic data were not considered during the study due to privacy concerns. 2. Ranking of user segments for a given ad has not been explored. 3. In general, user behavior modelling for BT is underexplored and hence we did not have a lot of resources to start with. 4. Study of algorithms for large sets of user data and their rapidly changing user behavior was not possible as BT requires large scale user data which is incremental.

30.  The following conclusions can be drawn from our experiments: 1. Users who clicked the same ad will be more similar than users who clicked different ads. 2. 2. The CTR of ads can be averagely improved as high as 670% over all the ads we collected, using fundamental clustering algorithms. 3. 3.Tracking short-term user behavior can perform better than tracking long-term user behavior for the user representation strategies.

31. Modeling the Impact of Short and Long Term Behavior on Search Personalization Paul N. Bennett, Ryen W. White, Wei Chu, Susan T. Dumais, Peter Bailey, Fedor Borisyuk, and Xiaoyuan Cui  This is a paper from 2012 that studies customizing results in search engines based on user behavior. While their focus was not ads, it did come to the conclusion that short term behavior was more effective than long term behavior when a session has been ongoing.

32.  Learning to Rank audience for behavioral targeting in display ads - Jian Tang, Ning Liu, Jun Yun, Yelong Shen, B Gao, S Yan,Ming Zhang  Online Targeting, behavioral advertising and privacy - Avi Goldfarb,Catherine E Tucker Transfer learning for behavioral targeting - Tianqi Chen, Jun Yan, Guirong Xue, Zheng Chen Analyzing content-level properties of the web adversphere - Yong Wang, Daniel Burgener, Aleksandar Kuzmanovic, Gabriel Maciá- Fernández ISP-enabled behavioral ad targeting without deep packet inspectionGabriel Maciá-Fernández, Yong Wang, Rafael Rodríguez-Gómez, Aleksandar Kuzmanovic Linking visual concept detection with viewer demographics - Adrian Ulges, Markus Koch, Damian Borth

33.  Using advanced user segmentation algorithms can lead to better results.  Study of better user representation strategies such as search sessions, content of user clicked pages and user browsing trials can be explored.