Download presentation
Presentation is loading. Please wait.
1
Bias and Variance of Bagging based on Subsampling with & without Replacement
Andreas Buja Werner Stuetzle * Statistics Department Statistics Department The Wharton School Adjunct Professor, CSE University of Pennsylvania University of Washington * Supported by NSF grant DMS Research performed while on sabbatical at AT&T Labs – Research Research motivated by Friedman & Hall paper ``On Bagging and Nonlinear Estimation" (available on the Web) and a counter-example to one of F & H's claims due to Yoram Gatt. 11/21/2018
2
11/21/2018
3
11/21/2018
4
11/21/2018
5
11/21/2018
6
11/21/2018
7
11/21/2018
8
11/21/2018
9
11/21/2018
10
11/21/2018
11
11/21/2018
12
11/21/2018
13
11/21/2018
14
11/21/2018
15
11/21/2018
16
11/21/2018
17
11/21/2018
18
11/21/2018
19
11/21/2018
20
11/21/2018
21
11/21/2018
22
11/21/2018
23
11/21/2018
24
11/21/2018
25
11/21/2018
26
11/21/2018
27
11/21/2018
28
11/21/2018
29
11/21/2018
30
11/21/2018
31
11/21/2018
32
11/21/2018
33
11/21/2018
34
T(F) T(Fn) F Fn ave = Tbag (Fn) 11/21/2018 Resamples
Space of probability measures F T(F) Fn T(Fn) Resamples ave = Tbag (Fn) 11/21/2018
35
11/21/2018
36
11/21/2018
37
11/21/2018
38
11/21/2018
39
11/21/2018
40
With repl.: g = n/m W/o repl.: g = n/m - 1 Equivalence:
n/mwi = n/mw/o-1 11/21/2018
41
11/21/2018
42
11/21/2018
43
11/21/2018
44
11/21/2018
45
Squared plug-in bias, scenario 2 , n = 800 , black ~ wi, red ~ wo
0.2 0.4 0.6 0.8 1.0 0.000 0.002 0.004 0.006 0.008 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared plug-in bias Squared plug-in bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 11/21/2018
46
Squared estimation bias, scenario 2 , n = 800 , black ~ wi, red ~ wo
0.2 0.4 0.6 0.8 1.0 0.000 0.002 0.004 0.006 0.008 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared estimation bias Squared estimation bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 11/21/2018
47
11/21/2018
48
Squared plug-in bias, scenario 3 , n = 800 , black ~ wi, red ~ wo
0.2 0.4 0.6 0.8 1.0 0.000 0.005 0.010 0.015 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared plug-in bias Squared plug-in bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 11/21/2018
49
Squared estimation bias, scenario 3 , n = 800 , black ~ wi, red ~ wo
0.2 0.4 0.6 0.8 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared estimation bias Squared estimation bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 11/21/2018
50
11/21/2018
51
11/21/2018
Similar presentations
