Bias and Variance of Bagging based on Subsampling with & without Replacement Andreas Buja Werner Stuetzle * Statistics Department Statistics Department The Wharton School Adjunct Professor, CSE University of Pennsylvania University of Washington * Supported by NSF grant DMS-9803226. Research performed while on sabbatical at AT&T Labs – Research Research motivated by Friedman & Hall paper ``On Bagging and Nonlinear Estimation" (available on the Web) and a counter-example to one of F & H's claims due to Yoram Gatt. 11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
T(F) T(Fn) F Fn ave = Tbag (Fn) 11/21/2018 Resamples Space of probability measures F T(F) Fn T(Fn) Resamples ave = Tbag (Fn) 11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
With repl.: g = n/m W/o repl.: g = n/m - 1 Equivalence: n/mwi = n/mw/o-1 11/21/2018
11/21/2018
11/21/2018
11/21/2018
11/21/2018
Squared plug-in bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 0.2 0.4 0.6 0.8 1.0 0.000 0.002 0.004 0.006 0.008 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared plug-in bias Squared plug-in bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 11/21/2018
Squared estimation bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 0.2 0.4 0.6 0.8 1.0 0.000 0.002 0.004 0.006 0.008 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared estimation bias Squared estimation bias, scenario 2 , n = 800 , black ~ wi, red ~ wo 11/21/2018
11/21/2018
Squared plug-in bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 0.2 0.4 0.6 0.8 1.0 0.000 0.005 0.010 0.015 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared plug-in bias Squared plug-in bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 11/21/2018
Squared estimation bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 0.2 0.4 0.6 0.8 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 alpha for sampling wo rep., alpha / (1-alpha) for sampling wi rep. squared estimation bias Squared estimation bias, scenario 3 , n = 800 , black ~ wi, red ~ wo 11/21/2018
11/21/2018
11/21/2018