1. 1 Asim Ansari Carl Mela E-Customization

2. Page 2 Introduction Marketing Targeted Promotions List Segmentation Conjoint Analysis Recommendation Systems Computer Science Collaborative filtering Machine learning u Customization key to managing relationships

3. Page 3 Customization and Electronic Media Electronic media facilitate customization Low production costs Timely data (received) and information (sent) Personalizable Reach

4. Page 4 Customization Benefits Content Providers Increasing site usage via customization can increase advertising revenue Internet Advertising forecast to grow to rapidly E-commerce Increasing sales via customization

5. Page 5 E-Customization Contexts Content providers can customize content (editorial) design (how many links and what order) to increase site visits, advertising revenue and loyalty. E-commerce firms can customize content (products, price, incentives, etc.) and design (how many items and what order) to increase sales and loyalty. The structure of the problem is identical.

6. Page 6 E-Customization Strategies Two customization Strategies Onsite External : e-mails Customizable at low-cost Need not wait for customers to come to site We take an external customization approach

7. Page 7 E-mail Example

8. Page 8 E-mail Marketing: Volume Growth ’99’00’01’02’03’04 Emails (billions) 0 50 100 150 200 250 Email retention services Email acquisition services Source: Forrester Report: Email Marketing Dialog, January 2000

9. Page 9 E-mail Marketing Services: Revenue Growth Revenues (billions) 0 1 2 3 4 5 ’99’00’01’02’03’04 Email retention services Email acquisition services Source: Forrester Report: Email Marketing Dialog, January 2000

10. Page 10 Email Design Problem Sports International News National News Weather Arts Determine the Content and Layout of the e-mail on a one-on-one basis

11. Page 11 Statistical Model Approach E-mail Configuration Click-through Data Optimization New E-mail Configuration Individual level preference coefficients

12. Page 12 Statistical Model Probability of clicking on a link depends upon utility to click Utility of clicking on a link = f( observed e-mail variables (html, # links), observed link variables (content and order of link), unobserved user effect, unobserved e-mail effect, unobserved link effect, error)

13. Page 13 Probit Model Population Component U ijk =  1 +  2 *Text j +  3 *NumItems j +  4 *Position jk +  5 *Content k + i1 + i2 *NumItems j + i3 * Position jk + i4 *Content k +  j  +  j  *Position jk +  j  *Content k +  k1 +e ijk i is person, j is e-mail and k is link. Random across Individuals Random across Emails

14. Page 14 Modeling Heterogeneity Random effects are assumed to come from a population distribution with zero mean i ~ G 1  j ~ G 2  k ~ G 3

15. Page 15 Modeling Heterogeneity Finite Mixtures Continuous Mixtures

16. Page 16 Modeling Heterogeneity: Dirichlet Process Priors Dirichlet Process Priors can be used to model the uncertainty about functional form of the population distribution G Allows semi-parametric estimation of random effects

17. Page 17 Dirichlet Process Priors A Dirichlet Process prior for a distribution G has two parameters A distribution function G 0 (.) and A positive scalar precision parameter  We write where, G 0 represents the expected value of G and  > 0, represents the strength of prior beliefs that sampled distributions G will be close to G 0

18. Page 18 Dirichlet Process Priors Let G be a random distribution from the Dirichlet Process, Let then, q1q1 qiqi qNqN p1p1 pipi pNpN

19. Page 19 Dirichlet Process: Role of  Large  Large number of distinct values from the base distribution Sampled distribution approximates base distribution Small  Sample will have a small number of distinct values Sampled distribution approximates a finite mixture

20. Page 20 Dirichlet Process Priors: Advantages Accommodates non-normality, multi-modality and skewness Provides a semi-parametric alternative to the normal distribution Provides accurate individual-level estimates Allows a synthesis of Finite Mixtures and Normal Heterogeneity

21. Page 21 Modeling Heterogeneity i ~ G 1 ~D(N(0,  ),  1 )  j ~ G 2 ~D(N(0,  ),  2 )  k ~ G 3 ~D(N(0,  ),  3 )

22. Page 22 Inference Bayesian Inference Priors   ~ Multivariate Normal    ~ Wishart   ll  ~ Inverse Gamma   ~ Inverse Gamma   1,  2,  3, ~ Gamma

23. Page 23 Sampling Based Inference Joint Posterior Density is very complex and cannot be summarized in closed form Sampling Based Inference Gibbs Sampling

24. Page 24 Full Conditionals Unknowns include {u}, , { i }, {  j }, {  k }, , , ,       Full conditionals for DP mixed model are very similar to those for normal population distributions

25. Page 25 Full Conditionals for Individual-level parameters: DP model Mixture of distributions And G b is the posterior distribution under the normal base distribution This is akin to collaborative filtering on parameter space

26. Page 26 Application Large content provider with many areas in site One area in the site sends e-mails to registered recipients in an effort to attract them to the area Permission marketing Design targeting issues Number of links, order of links, text or html Content targeting issues Content type (health, financial, etc.)

27. Page 27 Data Three months of e-mails, 1048 users E-mail file: e-mail date, number of links, order of links, link content, html or text User file: when received, by whom (registration data), which links clicked (cookies) Sample: 11,475 observations 7% response rate for links 36% click on more than one link

28. Page 28 Models No heterogeneity Person heterogeneity Person, E-mail and Link heterogeneity (Full Model)

29. Page 29 Predictive Ability Actual Click No Click Click No Click ab c d False Positives Click False Negative Fraction= c/(c+d), False Positive Fraction =b/(a+b) False Negatives Predicted

30. Page 30 Predictive Ability: Link Level ROC Curves False Positive Fraction True Positive Fraction [1-FNF]

31. Page 31 Predictive Ability: Email Level ROC Curves False Positive Fraction True Positive Fraction [1-FNF]

32. Page 32 Results - Parameter Estimates Full Model ParameterValueProb(  <0) Design Variables Intercept (  0 )-1.47(1.0) Person Random Effects ( Std. 0i ) 0.51 E-mail Random Effects (Std.  0j ) 0.45 Link Random Effects (Std.  0k ) 0.21 E-mail Type (  1 ) 0.29 (0.48) Link Order (  2 )-0.37 (1.0) Person Random Effects (Std. 2i ) 0.49 E-mail Random Effects (Std.  2j ) 0.22 Number of Links (  3 )-0.02 (0.55) Person Random Effects (Std. 3i ) 0.18

33. Page 33 Parameter Estimates Dirichlet Process Precision parameters User   = 103 => 61 “clusters” Email   = 114 => 65 “clusters” Links   = 383 => 383 “clusters”

34. Page 34 Link Level Predictions - Calibration Data 12345678910 0 5 15 20 25 30 35 40 45 50 Deciles Click Percentage

35. Page 35 Link Level Predictions - Validation Data 12345678910 0 5 15 20 25 30 35 Deciles Click Percentage

36. Page 36 E-mail Level Prediction - Calibration Data 12345678910 0 20 30 40 50 60 70 80 90 Deciles Click Percentage

37. Page 37 E-mail Level Predictions - Validation Data 12345678910 0 20 30 40 50 60 70 80 Click Percentage Deciles

38. Page 38 Optimization Model Overview Editorial content is fixed on a given day. n links available for k positions, n ¸ k How many links to include, what content to include, and how should it be ordered? Objective Maximize the expected number of click-backs to the site Maximize the likelihood of returning to the site

39. Page 39 Optimization Procedures Alternative 1: Complete Enumeration With many links, computational constraints Alternative 2: Assignment Algorithm

40. Page 40 Optimization: Objective Function Maximize expected number of click-throughs to site Let x ij =1 if link i is in position j Let p ij be the probability of click through if link i is in position j Maximize Objective function Maximize likelihood of at least one click-through Minimize Objective function

41. Page 41 Optimization Model Step 1: Maximize: Obj (x; p(x)|k) Subject to Assignment algorithm provides exact solution Step 2: Maximize over k={1, …, n}.

42. Page 42 Heuristic Approaches Original - No change in content or order Greedy - No change in content, order highest utility first Order - No change in content, optimize order Optimal - Optimize content (#number of links) and order (our procedure)

43. Page 43 Optimization Results

44. Page 44 Optimization Results Objective: At Least One Click Optimal leads to 56% increase in at least one click. Re-ordering gives 52% improvement, content selection is the balance. Optimal improves over Order for 43% of e-mails (those adverse to clutter). Greedy and Order are similar, however for users who have high positive effect for order (scroll to bottom), Greedy does poorly (one user went from 81% to 43%). Objective: Expected Number of Clicks Similar results

45. Page 45 Optimization Results

46. Page 46 Conclusions Modeling link response Varies with content (information) and design (how much, what order) Heterogeneity in persons, links, and e-mails E-targeting Potential to considerable enhance clicks (and presumably advertising revenue and loyalty) Our approach can be applied to both internal and external targeting strategies Our approach can also be applied to e-tailing

47. Page 47 Future Targeting Products and services for purchases Advertising E-grocers (features, displays, prices) How much is a feature worth? Other areas On-line choice processes Agent queries

48. Page 48 Dirchlet Process Moments E[G(B)]=E[G 0 (B)] and Var[G(B)]=G 0 (B)(1-G 0 (B))/ 

49. Page 49 Full Conditionals for Individual Level Model: Normal Heterogeneity Standard Case (Simple Model)

50. Page 50 Dirichlet Process Priors A c.d.f., G on  follows a Dirichlet Process if for any measurable finite partition of (B 1,B 2,.., B m ), of  the joint distribution of the random variables ( G(B 1 ), G(B 2 ), …, G(B m )) is Dirichlet(  G 0 (B 1 ), ….,  G 0 (B m )), where, G 0 is a the base distribution and  is the precision parameter