1. AD Click Prediction a View from the Trenches
Google paper 2013
윤철환

2. Google Ad

3. System Overview

4. FTRL-Proximal Algorithm
Online Gradient Descent(OGD) + Regularized Dual Averaging(RDA)
Gradient
Learning Late ,

5. FTRL-Proximal Algorithm

6. FTRL-Proximal Algorithm

7. Per Coordinate Learning Rates
N : negative events
P: Positive events
p= P / ( N + P )

8. FTRL-Proximal Algorithm

9. Memory saving tech Probabilisitic feature inclusion
Subsampling training data
Encoding values with fewer bits

10. Probabilistic Feature Inclusion
Poisson Inclusion
New feature are inserted with probability p
Bloom Filter Inclusion
Once a feature has occurred more than n times (according to the filter), we add it to the model

11. Subsampling Training Data
Any query for which at least one of the ads was clicked.
A fraction r ∈ (0, 1] of the queries where none of the ads were clicked.
The expected contribution of a randomly chosen event t in the unsampled data to the sub-sampled objective function

12. Encoding Values with Fewer Bits
Naive implementations of the Online Gradient Descent algorithm use 32 or 64 bit floating point encodings.
For their Regularized Logistic Regression models, such encodings waste memory.
Use fixed point (q2.13 % 16bit) encoding instead.
No measurable loss in precision and 75% RAM savings

13. GridViz