1. Statistic Models for Web/Sponsored Search Click Log Analysis The Chinese University of Hong Kong 1 Some slides are revised from Mr Guo Fan’s tutorial at CIKM 2009.

2. Index Background. A Simple Click Model. – Dependent click model [WSDM09]. Advanced Design. – Five extension directions. Advanced Estimation. – Bayesian framework and the rationale. – Bayesian browsing model (BBM) [Liu09]. – Click chain model (CCM) [Guo09]. Course Project. 2

3. Scenario: Web Search 3

4. User Click Log 4 36 23 18 11 36 1 2 3 4 5

5. Eye-tracking User Study Users have bias to examine the top results. 5

6. Position-bias Identification 6 Higher positions receive more user attention (eye fixation) and clicks than lower positions. This is true even in the extreme setting where the order of positions is reversed. “Clicks are informative but biased”. Normal Position Percentage Reversed Impression Percentage [ Joachims07 ]

7. Answer to Previous Example Result 5 is more relevant compared with Result 1. Because Result 5 has less opportunity to be examined. 7 36 23 18 11 36 1 2 3 4 5

8. Click Model Motivation Modeling the user’s click behavior in an interpreted manner and estimate the pure relevance of a query-document/ad pair regardless of bias. – Position-bias is the main problem. – Other kinds of bias. Influence among documents/ads Attractiveness bias Search intent bias … Pure relevance of a query-document/ad pair intuition. – When the query is submitted to the search engine and only one single document/ad is shown, what is the click-through rate of this query- document/ad pair? 8

9. Examination Hypothesis [Richardson07] A document must be examined before a click. The probability of click conditioned on being examined depends on the pure relevance of the query-document/ad pair. The click probability could be decomposed. – Global component. the examination probability which reflects the position-bias. – Local component (pure relevance). click probability of the (query, URL) pair conditioned on being examined. 9

10. Click Models Key tasks. – How to design the user examination behavior? – How to estimate the relevance of a query-doc/ad pair? Desired Properties. – Effective: aware of the position-bias/other-bias and address it properly. – Scalable: linear complexity for both time and space, easy to parallel. – Incremental: flexible for model update based on new data. 10 From this slide, “relevance” is equal to “pure relevance”.

11. Importance of Understanding Logs Better matching query and documents/ads. All the participants would benefit. – Users: better relevance. – Search engines: more revenue from advertisers and more users. – Advertisers: more return on investment (ROI). 11 Advertiser User Publisher Better Match

12. Growth of Web Users 12

13. Growth of Web Revenue 13

14. Index Background. A Simple Click Model. – Dependent Click Model [WSDM09]. Advanced Design. Advanced Estimation. Projects. 14

15. Notations – E i binary r.v. for Examination Event on position i; – C i binary r.v. for Click Event on position i; – r i = p(C i = 1| E i = 1) relevance for the query-document pair on position i. 15

16. Click Model Design 16 Dependent Click Model (DCM) [GUO09]

17. Parameters in DCM r=p(C=1|E=1) is local parameter. – Modeling the relevance of a query-document/ad pair. – The position-bias has been modeled by p(E=1). λ is global parameter. – Modeling p(E i+1 =1|C i =1,E i =1). 17 Parameters estimation Maximum log-likelihood method

18. Estimation of r: Step 1 Define as last click position. When there is no click, is the last position. 18 Query cikm PosURLClick 1cikm2008.org0 2www.cikm.org1 3www.fc.ul.pt/cikm0 4cikmconf.org0 5www.cikm.com/...1 6Ir.iit.edu/cikm20040 Query cikm PosURLClick 1cikm2008.org0 2www.cikm.org0 3www.fc.ul.pt/cikm0 4cikmconf.org0 5www.cikm.com/...0 6Ir.iit.edu/cikm20040

19. Estimation of r: Step 2 Log-likelihood of a query session. 19

20. Estimation of r: Step 3 By maximizing the lower bound of the log-likelihood, we have 20 Suppose the current pair has occurred in different sessions. For M sessions, it occurs before/on l and has been clicked; for N sessions, it occurs before/on l and is not clicked.

21. Estimation of λ For a specific, By maximizing the lower bound of the log- likelihood, we have 21 Suppose there are totally A sessions. In B sessions, the position l is large than position i and click event happens in position i. In C sessions, the position l is just equal to position i. Other cases happen in the other A-B-C sessions.

22. Property Verification Effective. Scalable and Incremental. 22

23. Evaluation Criteria for DCM Log-likelihood. – Given the document impression in the test set. – Compute the chance to recover the entire click vector. – Averaged over different query sessions. 23

24. Experimental Result for DCM 24

25. Some Other Evaluations Log-likelihood. – http://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood http://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood Perplexity. – http://en.wikipedia.org/wiki/Perplexity http://en.wikipedia.org/wiki/Perplexity Root mean square error (RMSE). – http://en.wikipedia.org/wiki/Root-mean-square_deviation http://en.wikipedia.org/wiki/Root-mean-square_deviation Area under ROC curve. – http://en.wikipedia.org/wiki/Receiver_operating_characteristic http://en.wikipedia.org/wiki/Receiver_operating_characteristic 25

26. Index Background. A Simple Click Model. Advanced Design. – Five extension directions. Advanced Estimation. Project. 26

27. 1 Dependency from Previous Docs/Ads For position 4 in the following two cases, do they have the same chance to be examined? Intuitively, the left one has less chance, since user may find the URL he/she wants in position 2 and stops the session. 27 Query cikm PosURLClick 1cikm2008.org0 2www.cikm.org1 3www.fc.ul.pt/cikm0 4cikmconf.org0 5www.cikm.com/...0 6Ir.iit.edu/cikm20040 Query cikm PosURLClick 1cikm2008.org0 2www.cikm.org0 3www.fc.ul.pt/cikm0 4cikmconf.org1 5www.cikm.com/...0 6Ir.iit.edu/cikm20040

28. Solution: Click Chain Model [Guo09] The chance of being examined depend on the relevance of previous documents/ads. Other similar work includes [Dupret08][Liu09]. 28

29. 2 Perceived v.s. Actual Relevance After clicking the docs/ads, the actual relevance, by judging from the landing page, might be different from user’s perceived relevance. 29 Pizza Query Ad1 Ad2 before examination after examination

30. Solution: Dynamic Bayesian Network [Chapelle09] For each ad, two kinds of relevance are defined, perceived relevance r and actual relevance s. s would influence the examination probability of the latter docs/ads. 30

31. 3 Aggregate v.s. Instance Relevance Users might have different intents for the same query. The click event could indicate the intent. 31 Aggregate search. E.g., learn the parameters Instance search. E.g., buy a camera Canon Query Ad1 Ad2 Canon Ad1 Ad2 Canon Ad1 Ad2

32. Solution: Joint Relevance Examination Model [Srikant10] Add a correction factor, which is determined by the click events of other docs/ads. Other similar work includes [Hu11]. 32

33. 4 Competing Influence in Docs/Ads When co-occurred with a high-relevant doc/ad, the perceived relevance of the current doc/ad would be decreased. 33

34. Solution: Temporal Click Model [Xu10] The docs/ads are competed to win the priority to be examined. 34

35. 5 Incorporating Features Feature example: dwelling time. 35

36. Solution: Post-Clicked Click Model [Zhong 10] Incorporating features to determine the relevance. Other similar work include [Zhu 10]. 36

37. Index Background. A Simple Click Model. Advanced Design. Advanced Estimation. – Bayesian framework and the rationale. – Bayesian browsing model. – Click chain model. Project. 37

38. Limitation of Maximum Log-likelihood Cannot fit the scalable and incremental properties. – It has difficulty in getting closed-form formula, when the model is complex. – Even in DCM as shown in this page, we need to approximate a lower bound for easy calculation. No prior information could be utilized in such sparse data environment. 38 Log-likelihood of DCM

39. An Coin-Toss Example for Bayesian Framework Scenario: to estimate the probability of tossing a head according to the following five training samples. The probability is a variable X = x. Each training sample is denoted by C i, e.g., C 1 = 1, C 4 =0. According to Bayesian rule, we have 39

40. Bayesian Estimation of Coin-tossing 40 X C1C1 C2C2 C3C3 C4C4 C5C5 Bayesian rule: Uniform prior: Independent sampling : Distribution : Estimation:

41. Density Function Update of Coin- tossing 41 Prior Posterior Density Function (not normalized) x 1 (1-x) 0 x 2 (1-x) 0 x 3 (1-x) 0 x 3 (1-x) 1 x 4 (1-x) 1

42. Click Data Scenario 42 a b c d a c e a b a c b a f g query Bayesian rule: Uniform prior: Independent sampling : Distribution :

43. Factor Trick If the factors of p(C|X) are arbitrary, for each training sample, a unique factor of p(X) must be stored. Thus it is space consuming; However if the factors of p(C|X) are from a small discrete set, only the exponents are needed to be stored. 43 Distribution :

44. Updating Example 44 Prior Density Function (not normalized) x 1 (1-x) 0 (1-0.6x) 0 (1+0.3x) 1 (1-0.5x) 0 (1- 0.2x) 0 … x 1 (1-x) 1 (1-0.6x) 0 (1+0.3x) 1 (1-0.5x) 0 (1- 0.2x) 0 … x 2 (1-x) 1 (1-0.6x) 0 (1+0.3x) 2 (1-0.5x) 0 (1- 0.2x) 0 … x 3 (1-x) 1 (1-0.6x) 1 (1+0.3x) 2 (1-0.5x) 0 (1- 0.2x) 0 … x 3 (1-x) 1 (1-0.6x) 1 (1+0.3x) 2 (1-0.5x) 1 (1- 0.2x) 0 …

45. How to realize the factor trick? Setting a global parameter for all cases. – Bayesian browsing model (BBM) [Liu09]. Assuming all other docs/ads follows the same distribution and integrating them. – Click chain model (CCM) [Guo09]. 45 In the following two example, we only concern the estimation of r using Bayesian framework. The estimation of other parameters are all based on maximizing the log-likelihood similarly as shown in DCM. Please refer the original paper for details.

46. Index Background. A Simple Click Model. Advanced Design. Advanced Estimation. – Bayesian framework and the rationale. – Bayesian browsing model. – Click chain model. Project. 46

47. BBM Variable Definition 47 For a specific query session, let – r i, the relevance variable at position i. – E i, the binary examination variable at position i. – C i, the binary click variable at position i. – n i, last click position before position i. – d i, the distance between position i and its previous clicked position.

48. Small Discrete Set of Beta Suppose M = 3 for simplicity illustration. There are only 6 values of beta. 48 n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1

49. Estimation Algorithms 49 How many times the Doc/ad was clicked How many times the Doc/ad was not clicked with the probability of beta n,d

50. Toy Example Step 1 50 Only top M=3 positions are shown, 3 query sessions and 4 distinct URLs. 41 4 3 13 312 Position 1 2 3 Query Session 3 Query Session 2 Query Session 1

51. Toy Example Step 2 51 Initialize M(M+1)/2+1 counts for each URL. URL Clicks n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1 4 0000000

52. Toy Example Step 3 52 Update counts for URL 4. – If not impressed, do nothing; – If clicked, increment “clicks” by 1; – Otherwise, locate the right r and d to increment. URL Clicks n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1 4 0000000

53. Toy Example Step 4 53 Update counts for URL 4. – If not impressed, do nothing; – If clicked, increment “clicks” by 1; – Otherwise, locate the right r and d to increment. URL Clicks n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1 4 0000001

54. Toy Example Step 5 54 Update counts for URL 4. – If not impressed, do nothing; – If clicked, increment “clicks” by 1; – Otherwise, locate the right r and d to increment. URL Clicks n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1 4 1000001

55. Toy Example Step 6 55 The posterior for URL 4. Interpretation: – The larger the probability of examination, the stronger the penalty for a non-click. URL Clicks n=0 d=1 n=0 d=2 n=0 d=3 n=1 d=1 n=1 d=2 n=2 d=1 4 1000001

56. Algorithm Complexities 56 Let Initializing and updating the counts: – Time: Space: Linear to the size of the click log Almost constant storage required

57. Index Background. A Simple Click Model. Advanced Design. Advanced Estimation. – Bayesian framework and the rationale. – Bayesian browsing model. – Click chain model. Project. 57

58. User Behavior Description 58 Examine the Document Click? See Next Doc? Done No Yes No Yes See Next Doc? Done No

59. Estimation Algorithms By assuming other docs/ads in a session follow the same distribution and integrate them, the factors f p(C|R) could be described from a small discrete set. 59

60. Five Cases The current doc/ad may occur in five different cases. For each case, there would be unique factors for p(C|R i ). 60

61. Case 1 61 The doc/ad must be examined. Other R can seen as constants.

62. Case 2 62

63. Case 3 63

64. All Cases 64 By assuming other docs/ads in a session follows the same distribution and integrate them, the factors f p(C|R) could be described from a small discrete set.

65. Index Background. A Simple Click Model. Advanced Design. Advanced Estimation. Project. 65

66. Description Fake dataset. Format. – queryId – ad1Id, click – ad2Id, click – ad3Id, click Evaluation Metric: ROC. Baseline. – Average (Avg). Current competitive method. – Simplified CCM (SCCM). Task. – Implement another advanced click model. – Compare the result with the Avg and SCCM. – Analyzing the reasons of improvement. 66

67. End 67