1 The Bootstrap’s Finite Sample Distribution An Analytical Approach Lawrence C. Marsh Department of Economics and Econometrics University of Notre Dame Midwest Econometrics Group (MEG) October 15 – 16, 2004 Northwestern University

2 This is the first of three papers: (1.) Bootstrap’s Finite Sample Distribution ( today !!! ) (2.) Bootstrapped Asymptotically Pivotal Statistics (3.) Bootstrap Hypothesis Testing and Confidence Intervals

3 traditional approach in econometrics Analytical solution Bootstrap’s Finite Sample Distribution Empirical process  approach used in this paper Analytical problem Analogy principle (Manski) GMM (Hansen) Empirical process 

4 Bootstrap sample of size m: Start with a sample of size n:{X i : i = 1,…,n} {X j * : j = 1,…,m} m n Define M i as the frequency of drawing each X i. bootstrap procedure

5...

6 for i  k

7 = Applied Econometrician: The bootstrap treats the original sample as if it were the population and induces multinomial distributed randomness.

8 = Econometric theorist: what does this buy you? Find out under joint distribution of bootstrap-induced randomness and randomness implied by the original sample data:

9 = Econometric theorist: Applied Econometrician: For example,

10 The Wild Bootstrap Multiply each boostrapped value by plus one or minus one each with a probability of one-half (Rademacher Distribution). Use binomial distribution to impose Rademacher distribution: W i = number of positive ones out of M i which, in turn, is the number of X i ’s drawn in m multinomial draws.

11 The Wild Bootstrap = Econometric Theorist: Applied Econometrician: under zero mean assumption

12...

13 almost surely, where is matrix of second partial derivatives of g. where X is a p x 1 vector. nonlinear function of . Horowitz (2001) approximates the bias of for a smooth nonlinear function gas an estimator of

14

15 Horowitz (2001) uses bootstrap simulations to approximate the first term on the right hand side. Exact finite sample solution: = +

16 Definition: Any bootstrap statistic,, that is a function of the elements of the set {f(X j * ): j = 1,…,m } and satisfies the separability condition where g(M i ) and h( f(X i )) are independent functions and where the expected value E M [g(M i )] exists, is a “directly analyzable” bootstrap statistic. Separability Condition

17 X is an n x 1 vector of original sample values. X * is an m x 1 vector of bootstrapped sample values. X * = HX where the rows of H are all zeros except for a one in the position corresponding to the element of X that was randomly drawn. E H [H] = (1/n) 1 m 1 n ’ where 1 m and 1 n are column vectors of ones.  m * = g(X * ) = g(HX ) Taylor series expansion  m * = g(X o * ) + [G 1 (X o * )]’(X *  X o * ) + (1/2) (X *  X o * )’[G 2 (X o * )](X *  X o * ) + R * Setup for empirical process: X o * = H o X

18  m * = g(X * ) = g(HX ) Taylor series expansion  m * = g(( 1/n )1 m 1 n ’X ) + [G 1 (( 1/n )1 m 1 n ’X )]’(H  ( 1/n )1 m 1 n ’) X + ( 1/2 )X ‘(H  ( 1/n )1 m 1 n ’)’[G 2 (( 1/n )1 m 1 n ’X )](H  ( 1/n )1 m 1 n ’) X + R * Taylor series: Now ready to determine exact finite moments, et cetera. X * = HX where the rows of H are all zeros except for a one in the position corresponding to the element of X that was randomly drawn. Setup for analytical solution: X o * = H o X H o = E H [H] = (1/n) 1 m 1 n ’

19 e = ( I n – X (X’X) -1 X’)  {,,..., } E H [H] = (1/n) 1 n 1 n ’ e * = H e A = ( I n – (1/n)1 n 1 n ’ ) } No restrictions on covariance matrix for errors.

20 Applied Econometrician:. = A = I n or where A1 n 1 n ’ = 01 n 1 n ’A = 0 and A = ( I n – (1/n)1 n 1 n ’ ) so

21 Econometric theorist: + where No restrictions on

22 This is the first of three papers: (1.) Bootstrap’s Finite Sample Distribution ( today !!! ) (2.) Bootstrapped Asymptotically Pivotal Statistics (3.) Bootstrap Hypothesis Testing and Confidence Intervals Thank you ! basically done. almost done.