1. Materials Process Design and Control Laboratory 1 An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations Nicholas Zabaras and Xiang Ma Materials Process Design and Control Laboratory Sibley School of Mechanical and Aerospace Engineering 101 Frank H. T. Rhodes Hall Cornell University Ithaca, NY 14853-3801 Email: zabaras@cornell.edu URL: http://mpdc.mae.cornell.edu/

2. Materials Process Design and Control Laboratory 2 Outline of the presentation  The topic of this presentation is going to address the following two issues: 1.Solve the problems with stochastic discontinuity; 2.Solve the problems of high stochastic dimension (>100). Using the following two techniques: 1.Adaptive sparse grid collocation method (ASGC) 1. 2.High dimensional model representation technique (HDMR) 2. 1. X. Ma, N. Zabaras, A hierarchical adaptive sparse grid collocation method for the solution of stochastic differential equations, Journal of Computational physics, Vol. 228, pp. 3084-3113, 2009. 2. X. Ma, N. Zabaras, A adaptive high dimensional stochastic model representation technique for the solution of stochastic differential equations, Journal of Computational physics, submitted, 2009.

3. Materials Process Design and Control Laboratory 3 Problem definition  Define a complete probability space. We are interested to find a stochastic function such that for P-almost everywhere (a.e.), the following equation hold: where are the coordinates in, L is (linear/nonlinear) differential operator, and B is a boundary operators.  In the most general case, the operator L and B as well as the driving terms f and g, can be assumed random.  In general, we require an infinite number of random variables to completely characterize a stochastic process. This poses a numerical challenge in modeling uncertainty in physical quantities that have spatio-temporal variations, hence necessitating the need for a reduced-order representation.  By using Karhunen-Loève expansion, the random input can be characterized by a set of random variables.

4. Materials Process Design and Control Laboratory 4 The finite-dimensional noise assumption  By using the Doob-Dynkin lemma, the solution of the problem can be described by the same set of random variables, i.e.  So the original problem can be restated as: Find the stochastic function such that  In this work, we assume that are independent random variables with probability density function. Let be the image of. Then is the joint probability density of with support

5. Materials Process Design and Control Laboratory 5 Adaptive Sparse Grid Collocation (ASGC)

6. Materials Process Design and Control Laboratory 666 Stochastic Collocation based framework Function value at any point is simply Stochastic function in 2 dimensions Need to represent this function Sample the function at a finite set of points Use basis function to get a approximate representation Spatial domain is approximated using a FE, FD, or FV discretization. Stochastic domain is approximated using multidimensional interpolating functions

7. Materials Process Design and Control Laboratory 777 Choice of collocation points and nodal basis functions  In the context of incorporating adaptivity, we have used the Newton-Cotes grid using equidistant support nodes and use the linear hat function as the univariate nodal basis.  Furthermore, by using the linear hat function as the univariate nodal basis function, one ensures a local support in contrast to the global support of Lagrange polynomials. This ensures that discontinuities in the stochastic space can be resolved. The piecewise linear basis functions can be defined as

8. Materials Process Design and Control Laboratory 88 LET OUR BASIC 1D INTERPOLATION SCHEME BE SUMMARIZED AS IN MULTIPLE DIMENSIONS, THIS CAN BE WRITTEN AS  Denote the one dimensional interpolation formula as  In higher dimension, a simple case is the tensor product formula Conventional sparse grid collocation (CSGC)  Using the 1D formula, the sparse interpolant, where is the depth of sparse grid interpolation and is the number of stochastic dimensions, is given by the Smolyak algorithm as  Here we define the hierarchical surplus as:

9. Materials Process Design and Control Laboratory 9 Nodal basis vs. hierarchical basis Nodal basis Hierarchical basis

10. Materials Process Design and Control Laboratory 1010 Hierarchical Integration  The mean of the random solution can be evaluated as follows:  Denoting we rewrite the mean as  To obtain the variance of the solution, we need to first obtain an approximate expression for

11. Materials Process Design and Control Laboratory 111111 Adaptive sparse grid collocation (ASGC) 1 Let us first revisit the 1D hierarchical interpolation  For smooth functions, the hierarchical surpluses tend to zero as the interpolation level increases.  On the other hand, for non-smooth function, steep gradients/finite discontinuities are indicated by the magnitude of the hierarchical surplus.  The bigger the magnitude is, the stronger the underlying discontinuity is.  Therefore, the hierarchical surplus is a natural candidate for error control and implementation of adaptivity. If the hierarchical surplus is larger than a pre-defined value (threshold), we simply add the 2N neighbor points of the current point. 1.X. Ma, N. Zabaras, An hierarchical adaptive sparse grid collocation method for the solution of stochastic differential equations, JCP, 228 (2009) 3084-3113.

12. Materials Process Design and Control Laboratory 12 Adaptive sparse grid collocation  As we mentioned before, by using the equidistant nodes, it is easy to refine the grid locally around the non-smooth region.  It is easy to see this if we consider the 1D equidistant points of the sparse grid as a tree-like data structure  Then we can consider the interpolation level of a grid point Y as the depth of the tree D(Y). For example, the level of point 0.25 is 3.  Denote the father of a grid point as F(Y), where the father of the root 0.5 is itself, i.e. F(0.5) = 0.5.  Thus the conventional sparse grid in N- dimensional random space can be reconsidered as

13. Materials Process Design and Control Laboratory 13 Adaptive sparse grid collocation  Thus, we denote the sons of a grid point by  From this definition, it is noted that, in general, for each grid point there are two sons in each dimension, therefore, for a grid point in a N- dimensional stochastic space, there are 2N sons.  The sons are also the neighbor points of the father, which are just the support nodes of the hierarchical basis functions in the next interpolation level.  By adding the neighbor points, we actually add the support nodes from the next interpolation level, i.e. we perform interpolation from level |i| to level |i|+1.  Therefore, in this way, we refine the grid locally while not violating the developments of the Smolyak algorithm

14. Materials Process Design and Control Laboratory 14 NUMERICAL EXAMPLES

15. Materials Process Design and Control Laboratory 15 Adaptive sparse grid interpolation Ability to detect and reconstruct steep gradients

16. Materials Process Design and Control Laboratory 1616  The discontinuity then occurs at the planes and.  We first study the stochastic response subject to the following random 1D initial input: where. Since the random initial data can cross the plane we know that gPC will fail for this case. (X.Wan, G. Karniadakis, 2005)2005  The governing equations are: The Kraichnan-Orszag three-mode problem

17. Materials Process Design and Control Laboratory 17 One-dimensional random input: variance  gPC fails after t = 6s while the ASGC method converges even with a large threshold. The solid line is the result from MC-SOBOL sequence with 10 6 iterations.

18. Materials Process Design and Control Laboratory 18 One-dimensional random input: adaptive sparse grid  From the adaptive sparse grid, it is seen that most of the refinement after level 8 occurs around the discontinuity point Y = 0.0, which is consistent with previous discussion.  The refinement stops at level 16, which corresponds to 425 number of points, while the conventional sparse grid requires 65537 points. Adaptive sparse grid with

19. Materials Process Design and Control Laboratory 19 One-dimensional random input The ‘exact’ solution is taken as the results given by MC- SOBOL The error level is defined as

20. Materials Process Design and Control Laboratory 20 Two-dimensional random input  Next, we study the K-O problem with 2D input:  Now, instead of a point, the discontinuity region becomes a line.

21. Materials Process Design and Control Laboratory 21 Two-dimensional random input  It can be seen that even though the gPC fails at a larger time, the adaptive collocation method converges to the reference solution given by MC-SOBOL with 10 6 iterations.

22. Materials Process Design and Control Laboratory 22 Two-dimensional random input The error level is defined as The number of points of the conventional grid with interpolation 20 is 12582913.

23. Materials Process Design and Control Laboratory 23 Stochastic elliptic problem  Here, we adopt the model problem with the physical domain. To avoid introducing errors of any significance from the domain discretization, we take a deterministic smooth load with homogeneous boundary conditions.  The deterministic problem is solved using the finite element method with 900 bilinear quadrilateral elements. Furthermore, in order to eliminate the errors associated with a numerical K-L expansion solver and to keep the random diffusivity strictly positive, we construct the random diffusion coefficient with 1D spatial dependence as where are independent uniformly distributed random variables in the interval

24. Materials Process Design and Control Laboratory 2424 Stochastic elliptic problem  In the earlier expansion, and  The parameter can be taken as and the parameter L is

25. Materials Process Design and Control Laboratory 25 Stochastic elliptic problem  This expansion is similar to a K-L expansion of a 1D random field with stationary covariance  Small values of the correlation L correspond to a slow decay, i.e. each stochastic dimension weighs almost equally. On the other hand, large values of L results in fast decay rates, i.e., the first several stochastic dimensions corresponds to large eigenvalues weigh relatively more.  By using this expansion, it is assumed that we are given an analytic stochastic input. Thus, there is no truncation error. This is different from the discretization of a random filed using the K-L expansion, where for different correlation lengths we keep different terms accordingly.  In this example, we fix N and change L to adjust the importance of each stochastic dimension. In this way, we investigate the effects of L on the ability of the ASGC method to detect the important dimensions.

26. Materials Process Design and Control Laboratory 26 Stochastic elliptic problem: N = 11

27. Materials Process Design and Control Laboratory 27 Stochastic elliptic problem: N = 11  We estimate the approximation error for the mean and variance. Specifically, to estimate the computation error in the q-th level, we fix N and compare the results at two consecutive levels, e.g. the error for the mean is  The previous figures shows results for different correlation lengths at N=11.  For small L, the convergence rates for the CSGC and ASGC are nearly the same. On the other hand, for large L, the ASGC method requires much less number of collocation points than the CSGC for the same accuracy.  This is because more points are placed only along the important dimensions which are associated with large eigenvalues.  Next, we study some higher-dimensional cases. Due to the rapid increase in the number of collocation points, we focus on moderate high correlation lengths so that the ASGC is effective.

28. Materials Process Design and Control Laboratory 28 Stochastic elliptic problem: higher-dimensions #points:20271 Comparison: CSGC: 68074001

29. Materials Process Design and Control Laboratory 29 Stochastic elliptic problem: higher-dimensions  In order to further verify our results, we compare the mean and the variance when N = 75 using the AGSC method with with the ‘exact’ solution obtained by MC-SOBOL simulation with 10 6 iterations. Computational time: ASGC : 0.5 hour MC: 2 hour

30. Materials Process Design and Control Laboratory 30 Stochastic elliptic problem: N=25  Correlation length affects the convergence of ASGC. If every dimension is equally important, the ASGC is not effective. It is even worse than standard MC method for a high dimensional problem as a result of the weak dependence on the dimensionality in the logarithmic term of the error bound.

31. Materials Process Design and Control Laboratory 31 High Dimensional Model Representation (HDMR)

32. Materials Process Design and Control Laboratory 3232 High dimensional model representation (HDMR) 1  Let be a real-value multivariate stochastic function:, which depends on a N-dimensional random vector. A HDMR of can be described by where the interior sum is over all sets of integers, that satisfy.This relation means that can be viewed as a finite hierarchical correlated function expansion in terms of the input random variables with increasing dimensions.  For most well-defined physical systems, the first- and second-order expansion terms are expected to have most of the impact upon the output and the contribution of higher-order terms would be insignificant. 1. O. F. Alis, H. Rabitz, General foundations of high dimensional representations, Journal of Mathematical Chemistry, 25 (1999) 127-142.

33. Materials Process Design and Control Laboratory 3333 High dimensional model representation (HDMR) In this expansion:  denotes the zeroth-order effect which is a constant.  The component function gives the effect of the variable acting independently of the other input variables.  The component function describes the interactive effects of the variables and. Higher-order terms reflect the cooperative effects of increasing numbers of variables acting together to impact upon.  The last term gives any residual dependence of all the variables locked together in a cooperative way to influence the output.  Depending on the method to represent component functions, there are two types of HDMR: ANOVA-HDMR and CUT-HDMR.

34. Materials Process Design and Control Laboratory 34 HDMR: Compact notation  This equation is often written in a more compact notation: for a given set where denotes the set of coordinate indices and. Here, denotes the - dimensional vector containing those components of whose indices belong to the set, where is the cardinality of the corresponding set, i.e..  The component functions can be explicitly given as where the projection operator is defined as where induced by the measure

35. Materials Process Design and Control Laboratory 35 HDMR ANOVA-HDMRCUT-HDMR Lebesgue measure Dirac measure dimensional integration dimensional function at a reference point Computational expensiveComputational efficient

36. Materials Process Design and Control Laboratory 36 CUT-HDMR  Within the framework of CUT-HDMR, we can write where the notation means that the components of other than those indices that belong to the set equal to those of the reference point.  Therefore, the -dimensional stochastic problem is transformed to several lower-order -dimensional problems which can easily solved by ASGC:  If the HDMR is a converged expansion, the choice of this point does not affect the approximation. In this work, the mean of the random input vector is chosen as the reference point.

37. Materials Process Design and Control Laboratory 37 CUT-HDMR  Let us denote as the mean of the component function. Then the mean of the HDMR expansion is simply.  It is common to refer to the terms collectively as the “order- terms”. Then the expansion order is the maximum of. The number of collocation points in this expansion is defined as the sum of the number of points for each sub-problem, i.e.  In practice, we always truncate the expansion by taking only a subset of all indices. We can define an interpolation formula for the approximation of as  However, the number of order- component function is, which increases quickly with the number of dimensions. Therefore, we developed an adaptive version of HDMR.

38. Materials Process Design and Control Laboratory 38 Adaptive HDMR  First, we try to find the important dimensions. To this end, we always construct the zeroth- and first-order HDMR expansion. We define a weight: Then we define the important dimensions as those whose weights are larger than a predefined error threshold. Only higher order terms which consist of only these important dimensions are considered.  However, not all the possible terms are computed. For higher-order term, a weight is also defined as We also define the important terms in a similar way. We put all the important dimensions and higher-order terms in to a set. When adaptively constructing HDMR for each new order, we then only calculate the term whose indices satisfy the admissibility relation

39. Materials Process Design and Control Laboratory 39 Adaptive HDMR  In this way, we try to find those terms which may have significant contributions to the overall expansion while ignoring other trivial terms in order to reduce the computational cost for extremely high-dimensional problems.  Let us denote the order of expansion as Furthermore, we also define a relative error of the integral value between two consecutive expansion orders and as  If is smaller than another predefined error threshold, the HDMR is regarded as converged and the construction stops.

40. Materials Process Design and Control Laboratory 4040 Numerical example: Flow through random media Basic equation for pressure and velocity in a domain where denotes the source/sink term. A mixed finite element method is utilized to solve the forward problem.  To impose the non-negativity of the permeability, we will treat the permeability as a log- normal random field obtained from the K-L expansion injection well production well where is a zero mean random field with a covariance function where is the correlation length and is the standard deviation.

41. Materials Process Design and Control Laboratory 41 Numerical example: K-L Expansion  The eigenvalues and their corresponding eigenfunctions can be determined analytically. The are assumed as i.i.d uniform random variables on [-1,1].  According to the decay rate of eigenvalues, the number of stochastic dimensions is and, respectively for and. Series of eigenvalues and their finite sums for three different correlation lengths at  Monte Carlo simulations are conducted for the purpose of comparison. For each case, the reference solution is taken from samples and all errors are defined as normalized errors. In all cases,.

42. Materials Process Design and Control Laboratory 42 Standard deviation for different correlation lengths Standard deviation of the velocity-component along the cross section for different correlation lengths

43. Materials Process Design and Control Laboratory 43 PDF at (0,0.5) for different correlation lengths PDF of the velocity-component at point for different correlation lengths

44. Materials Process Design and Control Laboratory 44 Convergence of the normalized errors Convergence of the normalized errors of the standard deviation of the velocity-component for different correlation lengths

45. Materials Process Design and Control Laboratory 45 Standard deviations for different with Standard deviation of the velocity-component along the cross section for different.

46. Materials Process Design and Control Laboratory 46 PDF at (0,0.5) for different with PDF of the velocity-component at point for different

47. Materials Process Design and Control Laboratory 47 Convergence of the normalized errors with Convergence of the normalized errors of the standard deviation of the velocity-component for different

48. Materials Process Design and Control Laboratory 48 Conclusions  An adaptive hierarchical sparse grid collocation method is developed based on the error control of local hierarchical surpluses.  Besides the detection of important and unimportant dimensions as dimension-adaptive methods can do, additional singularity and local behavior of the solution can also be revealed.  By combining HDMR and ASGC, extremely high dimensional stochastic problem can be solve accurately and efficiently.  The numerical examples show that the number of component functions needed in the HDMR for a fixed stochastic dimension depends more on the input variability.  This method to our knowledge is the first approach which can solve high-dimensional stochastic problems by reducing dimensions from truncation of HDMR and resolve low-regularity by local adaptivity through ASGC.