1. On Sequential Experimental Design for Empirical Model-Building under Interval Error Sergei Zhilin, sergei@asu.ru Altai State University, Barnaul, Russia

2. 2 Outline Regression under interval error Experimental design: refining context Classical and “interval” design optimality criteria Sequential experimental design for regression models under interval error Comparative simulation study of classical and “interval” sequential design procedures Conclusions

3. 3 Regression under Interval Error Model structure xTxT + … x1x1 x2x2 xpxp  y  Input variables x = (x 1,…,x p ) T measured without error Output variable y measured with error Linear-parameterized modeling function Model parameters to be estimated Measurement error “Interval” error means “unknown but bounded”:

4. 4 Regression under Interval Error Each row (x j, y j,  j ) of the measurements table constrains possible values of the parameter  with the set Values of the parameter  consistent with all constraints form an uncertainty set

5. 5 Set of feasible models Regression under Interval Error Fitting data with the model y =  1 +  2 x 11 22 x y In (x, y) domain In (  1,  2 ) domain Uncertainty set A is unbounded = not enough data to build the model Uncertainty set A Set of feasible models

6. 6 Regression under Interval Error Problems that may be stated with respect to uncertainty set A Interval estimates of  Point estimates of  22 11 11 11 22 22 Model parameters estimation 11 ^ 22 ^

7. 7 Regression under Interval Error Problems that may be stated with respect to uncertainty set A Point estimate of y Interval estimate of y Prediction of the output variable value for fixed values of input variables x y y(x)y(x) y(x)y(x) x y(x)y(x) ^

8. 8 –Sequential experimental design –Simultaneous experimental design Experimental Design: Refining Context Product or process optimization Model quality optimization Experiment Analysis (Is the model quality satisfactory?) Design for ~1 observation End Begin Experiment Analysis Design for N observations End Begin

9. 9 Experimental Design for Regression under Interval Error –model –design space –design matrix –measurements –error bounds –information matrix –  covariance matrix  –standardized variance function of y(x,  )  Notations 0 1 1 0 1 1

10. 10 Experimental Design for Regression under Interval Error Design optimality criteria –Classical NameMinimizes D -optimality (volume of joint confidence interval) G -optimality (maximal variance of prediction) –Interval ( by M.P. Dyvak ) NameMinimizes I D -optimalitysquared volume of A I E -optimalitysquared maximal diagonal of A I G -optimalitymaximal prediction error D = (X T X) –1 d(x) = x T Dx Depend only on X, hence are applicable for interval error as well I E - and I G - optimality are equivalent for spherical design space and n > p

11. 11 Experimental Design for Regression under Interval Error Motivation –Classical methods of experimental design use only an information which X brings, nor Y, nor E –Interval methods of experimental design developed by Dyvak work for saturated designs ( p=n ) and use X and E, nor Y. –Does using of information, which Y contains, allow to improve the quality of constructed model or to increase the “speed” of sequential experimental design procedure?

12. 12 x next = I E Design(, X, Y, E) Experimental Design for Regression under Interval Error How to use the information which Y brings? 22 11 1.Find out the direction a of maximal spread of A : 2.Next experimental point x next  is selected in such a way that it induces the constraint orthogonal to a has maximal norm (width of constraint ) w Uncertainty set A(X,Y,E)

13. 13 i = 0; repeat x = I E Design(, X i, Y i, E i ); Experimental Design for Regression under Interval Error I E -optimal sequential design ( X 0, Y 0, E 0 ) – initial dataset

14. 14 Experimental Design for Regression under Interval Error I E -optimal sequential design ( X 0, Y 0, E 0 ) – initial dataset y = measurement in x with error  ; i = i + 1; until i > N or IA(X i, Y i, E i ) is small; i = 0; repeat x = I E Design(, X i, Y i, E i );

15. 15 Experimental Design for Regression under Interval Error Simulation study 1. Comparison of I E - and D -optimal sequential designs under zero errors repeat until i > 9 I E -optimal sequential design D -optimal sequential design repeat until i > 9

16. 16 Experimental Design for Regression under Interval Error Simulation study 1. D -optimal sequential design results Variables domainParameters domain Volume(A) = 0.6400  4  2 IA = [0.45, 1.55]  [1.45, 2.55] Volume(IA) = 1.21 3,7 1,5,9 2,6,10 4,8 22

17. 17 Experimental Design for Regression under Interval Error Simulation study 1. I E -optimal sequential design results Variables domainParameters domain Volume(A) = 0.5077   2 IA = [0.59, 1.41]  [1.60, 2.40] Volume(IA) = 0.66 22

18. 18 Experimental Design for Regression under Interval Error Simulation study 2. Comparison of I E - and D -optimal sequential designs under error which follows truncated normal distribution 3  Errors are simulated by – truncated normal distribution { 3 uniformly distributed points from }

19. 19 Experimental Design for Regression under Interval Error for r = 1 to 1500 do until i > N repeat end for { 3 uniformly distributed points from }; { 3 random values from }; random value from ; if then Simulation study 2

20. 20 Experimental Design for Regression under Interval Error Simulation study 2. Results for Number of selected points N Number of winnings k, (1500 – k) 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0510152025 0 250 500 750 1000 1250 1500 I E -Design D

21. 21 Experimental Design for Regression under Interval Error Simulation study 2. Results for Number of selected points N Number of winnings k, (1500 – k) 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0510152025 0 250 500 750 1000 1250 1500 I E -Design D

22. 22 Experimental Design for Regression under Interval Error The “cost” of I E -optimal design –The problem of finding maximal spread direction of A is a concave quadratic programming problem (CQPP) –It is proved that CQPP is NP-hard, i.e. solving time of the problem exponentially depends on its dimension (the number of input variables p ) –To overcome the difficulties we need to use special computational means (such as parallel computers) or we can limit ourself with near-optimal solutions

23. 23 Conclusions Interval model of error allows to use the information about measured values of output variable for effective sequential experimental design The results of the performed simulation study give a cause for careful analytical investigation of properties of I E -optimal sequential design procedures I E -optimal sequential design for high-dimensional models demands for special computational techniques