Nonparametric Bootstrap Inference on the Characterization of a Response Surface Robert Parody Center for Quality and Applied Statistics Rochester Institute.

Slides:



Advertisements
Similar presentations
Rachel T. Johnson Douglas C. Montgomery Bradley Jones
Advertisements

Inference in the Simple Regression Model
Assumptions underlying regression analysis
On Sequential Experimental Design for Empirical Model-Building under Interval Error Sergei Zhilin, Altai State University, Barnaul, Russia.
1 Probabilistic Uncertainty Bounding in Output Error Models with Unmodelled Dynamics 2006 American Control Conference, June 2006, Minneapolis, Minnesota.
“Students” t-test.
Multiple Regression and Model Building
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
Computational Statistics. Basic ideas  Predict values that are hard to measure irl, by using co-variables (other properties from the same measurement.
A.M. Alonso, C. García-Martos, J. Rodríguez, M. J. Sánchez Seasonal dynamic factor model and bootstrap inference: Application to electricity market forecasting.
CHAPTER 2 Building Empirical Model. Basic Statistical Concepts Consider this situation: The tension bond strength of portland cement mortar is an important.
Uncertainty and confidence intervals Statistical estimation methods, Finse Friday , 12.45–14.05 Andreas Lindén.
CmpE 104 SOFTWARE STATISTICAL TOOLS & METHODS MEASURING & ESTIMATING SOFTWARE SIZE AND RESOURCE & SCHEDULE ESTIMATING.
Experimental Design, Response Surface Analysis, and Optimization
6-1 Introduction To Empirical Models 6-1 Introduction To Empirical Models.
11 Simple Linear Regression and Correlation CHAPTER OUTLINE
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 13 Nonlinear and Multiple Regression.
11.1 Introduction to Response Surface Methodology
1 Statistical Tests of Returns to Scale Using DEA Rajiv D. Banker Hsihui Chang Shih-Chi Chang.
© 2010 Pearson Prentice Hall. All rights reserved Least Squares Regression Models.
The Multiple Regression Model Prepared by Vera Tabakova, East Carolina University.
Linear Methods for Regression Dept. Computer Science & Engineering, Shanghai Jiao Tong University.
Resampling techniques Why resampling? Jacknife Cross-validation Bootstrap Examples of application of bootstrap.
1 A Prediction Interval for the Misclassification Rate E.B. Laber & S.A. Murphy.
1 Chapter 3 Multiple Linear Regression Ray-Bing Chen Institute of Statistics National University of Kaohsiung.
2008 Chingchun 1 Bootstrap Chingchun Huang ( 黃敬群 ) Vision Lab, NCTU.
Chapter 11 Multiple Regression.
1 A Prediction Interval for the Misclassification Rate E.B. Laber & S.A. Murphy.
Probability & Statistics for Engineers & Scientists, by Walpole, Myers, Myers & Ye ~ Chapter 11 Notes Class notes for ISE 201 San Jose State University.
Bootstrapping LING 572 Fei Xia 1/31/06.
Basics of regression analysis
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
Simple Linear Regression Analysis
Bootstrapping applied to t-tests
Regression Analysis Regression analysis is a statistical technique that is very useful for exploring the relationships between two or more variables (one.
Chapter 13: Inference in Regression
5-1 Introduction 5-2 Inference on the Means of Two Populations, Variances Known Assumptions.
Copyright © 2013, 2010 and 2007 Pearson Education, Inc. Chapter Inference on the Least-Squares Regression Model and Multiple Regression 14.
Model Building III – Remedial Measures KNNL – Chapter 11.
Essential Statistics in Biology: Getting the Numbers Right
+ Chapter 12: Inference for Regression Inference for Linear Regression.
Non-Linear Models. Non-Linear Growth models many models cannot be transformed into a linear model The Mechanistic Growth Model Equation: or (ignoring.
Using Resampling Techniques to Measure the Effectiveness of Providers in Workers’ Compensation Insurance David Speights Senior Research Statistician HNC.
The Examination of Residuals. Examination of Residuals The fitting of models to data is done using an iterative approach. The first step is to fit a simple.
Various topics Petter Mostad Overview Epidemiology Study types / data types Econometrics Time series data More about sampling –Estimation.
1 10 Statistical Inference for Two Samples 10-1 Inference on the Difference in Means of Two Normal Distributions, Variances Known Hypothesis tests.
Inference for Regression Simple Linear Regression IPS Chapter 10.1 © 2009 W.H. Freeman and Company.
Why Model? Make predictions or forecasts where we don’t have data.
1 11 Simple Linear Regression and Correlation 11-1 Empirical Models 11-2 Simple Linear Regression 11-3 Properties of the Least Squares Estimators 11-4.
ELEC 303 – Random Signals Lecture 18 – Classical Statistical Inference, Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 4, 2010.
Lecture 4: Statistics Review II Date: 9/5/02  Hypothesis tests: power  Estimation: likelihood, moment estimation, least square  Statistical properties.
Limits to Statistical Theory Bootstrap analysis ESM April 2006.
Single-Factor Studies KNNL – Chapter 16. Single-Factor Models Independent Variable can be qualitative or quantitative If Quantitative, we typically assume.
The Simple Linear Regression Model: Specification and Estimation ECON 4550 Econometrics Memorial University of Newfoundland Adapted from Vera Tabakova’s.
Bootstrap Event Study Tests Peter Westfall ISQS Dept. Joint work with Scott Hein, Finance.
Non-Linear Models. Non-Linear Growth models many models cannot be transformed into a linear model The Mechanistic Growth Model Equation: or (ignoring.
Statistical Data Analysis 2011/2012 M. de Gunst Lecture 3.
Regression Analysis1. 2 INTRODUCTION TO EMPIRICAL MODELS LEAST SQUARES ESTIMATION OF THE PARAMETERS PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND ESTIMATION.
Chapter 9 Inferences Based on Two Samples: Confidence Intervals and Tests of Hypothesis.
Estimating standard error using bootstrap
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
(5) Notes on the Least Squares Estimate
Probability Theory and Parameter Estimation I
Statistical Quality Control, 7th Edition by Douglas C. Montgomery.
CHAPTER 29: Multiple Regression*
Chapter 12 Inference on the Least-squares Regression Line; ANOVA
Regression Models - Introduction
The Simple Linear Regression Model: Specification and Estimation
Simple Linear Regression
Fractional-Random-Weight Bootstrap
Presentation transcript:

Nonparametric Bootstrap Inference on the Characterization of a Response Surface Robert Parody Center for Quality and Applied Statistics Rochester Institute of Technology 2009 QPRC June 4, 2009

Presentation Outline Introduction Previous Work New Technique Example Simulation Study Conclusion and Future Research

Introduction Response Surface Methodology (RSM) –Identify the relationship between a set of k- predictor variables and the response variable y –Typically, the goal of the experiment is to optimize E(Y) is transformed into coded x by is transformed into coded x by

The Model A second order model is fit to the data represented by –where: i, ii, and ij are unknown parameters i, ii, and ij are unknown parameters ~ F(0, 2 ) and independent ~ F(0, 2 ) and independent u are other effects such as block effects and covariates, which are not interacting with the x i s u are other effects such as block effects and covariates, which are not interacting with the x i s

Equivalently, in matrix form,

Background Canonical Analysis Rotate the axis system so that the new system lies parallel to the principle axes of the surface P is the matrix of eigenvectors of B where P P = PP = I The rotated variables and parameters: – – w = P x – – = P – – = P BP = diag( i )

If all i 0), the stationary point is a maximizer (minimizer); contours are ellipsoidal. If the i have different signs, the stationary point is a minimax point (complicated hyperbolic contours). Types of Surfaces

Standard Errors for the i Carter Chinchilli and Campbell (1990) –Found standard errors and covariances for i by way of the delta method Bisgaard and Ankenman (1996) –Simplified this with the creation of the Double Linear Regression (DLR) method

Previous Work Edwards and Berry (1987) –Simulated a critical point for a prespecified linear combination of the parameters –The natural pivotal quantity for constructing simultaneous intervals for these linear combinations of the parameters is

Shortcoming The technique on the previous slide is only valid when The technique on the previous slide is only valid when –The errors are i.i.d. normal with constant variance –The set of linear combinations of interest are prespecified

Research Goal Employ a nonparametric bootstrap based on a pivotal quantity to extend the previously mentioned work to include situations where: Employ a nonparametric bootstrap based on a pivotal quantity to extend the previously mentioned work to include situations where: 1. The set of linear combinations of interest are not prespecified 2. Relax the error distribution assumption

12 Bootstrap Idea Resample from the original data – either directly or via a fitted model – to create replicate datasets Use these replicate datasets to create distributions for parameters of interest Consider the nonparametric version by utilizing the empirical distribution

13 Empirical Distribution The empirical distribution is one which equal probability 1/N is given to each sample value y i The corresponding estimate of the cdf F is the empirical distribution function (EDF), which is defined as the sample proportion:

New Technique The pivotal quantity for simultaneous inference on i :

Bootstrap Equivalent Replace the parameter with the estimates and the estimates with the bootstrap estimates to get:

Bootstrap Parameter Estimation Find the model fits Resample from the modified residuals N times with replacement Add these values to the fits and use them as observations Fit the new model and determine the bootstrap parameter estimates

17 An Adjustment We usually at least assume that the errors are iid from a distribution with mean 0 and constant variance 2 The residuals on the other hand come from a common distribution with mean 0 and variance 2 (1-h ii ) So the modified residuals become

Critical Point Procedure Create nonparametric bootstrap estimates for the unknown parameters in Q* Now find Q* by maximizing over the j elements Repeat this process for a large number of bootstrap samples (m) and take the (m+1)(1- ) th order statistic

Bootstrap Simulation Size Edwards and Berry (1987) showed conditional coverage probability of 95% simulation-based bounds will be +/ for 99% of the generations for (m+1)=80000

Example Chemical process experiment with k=5 from Box (1954) Goal: Maximize percentage yield

Parameter Estimates ParameterEstimate

Parameter Estimates ParameterEstimate

Critical Point Using =0.05 and (m+1)=80000, we get

Estimates and Estimates and 95% Simultaneous Confidence Intervals ParameterLCLEstimateUCL

Estimates and Estimates and 95% Simultaneous Confidence Intervals ParameterLCLEstimateUCL

Comparison of critical points –For the example, we would only need ~88% of the sample size for the simulation method as compared to traditional simultaneous methods Computer Time –Approximately 2 minutes on a Intel Core 2 Duo computer Relative Efficiency

Simulation Study 10 critical points were created For each critical point, confidence intervals were created by bootstrapping the residuals This was done 100 times for each point

Simulation Results

Conclusions New technique yields tighter bounds Works for linear combinations not prespecified Relaxes normality assumption on the error terms Simulation study yields adequate coverage

Future Research Relax model assumptions further to include nonhomogeneous error variances Apply to other situations where we are unable to prespecify the combinations, such as ridge analysis