1. Chapter 13 – Ensembles and Uplift
Data Mining for Business Analytics
Shmueli, Patel & Bruce

2. Ensembles can improve predictive power
Model 1
Model 2
Error
e1,i
e2,i
Expected Error
E(e1,i) = 0
E(e2,i) = 0
Variance
Var(e1,i)
Var(e2,i)
Ensemble:
E((e1,i+ e2,i)/2) = 0
¼ Var(e1,i) + ¼ Var(e2,i) + Cov(e1,i, e1,i)

3. Methods used Simple averaging Weighted averaging
Voting for classifiers

4. Bagging Bootstrap sampling Aggregating
Generate multiple random samples with replacement
Aggregating
Run modeling algorithm on each sample
Combine the results

5. Boosting
Fit a model
Generate a sample -- oversample misclassified cases
Fit the model to the new sample
Repeat 2 and 3 multiple times
Bagging improves stability
Helps avoid over fitting

6. Advantages and Disadvantages
More precise predictions
Improves stability
Helps avoid over fitting
Disadvantages
Requires more resources
Time
Ensemble model non-interpretable

7. Uplift modeling Collect sample data including current status
Randomly split data into treatment and control group
Apply treatment to treatment group
Measure status change for both groups
Recombine sample and randomly partition into training and validation sets
Develop model to training set with Status change as target and include treatment applied or not as a predictor

8. Uplift modeling …..
Run model to validation data set with treatment set to 1 and calculate propensity, i.e. P(Success/Treatment = 1)
Repeat with treatment set to 0 and calculate propensity, i.e. P(Success/Treatment = 0)
Uplift = P(Success/Treatment = 1) - P(Success/Treatment = 0)

9. Uplift Example

10. Uplift Example

11. Uplift Example

12. Uplift Example