Materials Process Design and Control Laboratory 1 An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential.

Slides:



Advertisements
Similar presentations
Isoparametric Elements Element Stiffness Matrices
Advertisements

Fast Algorithms For Hierarchical Range Histogram Constructions
Point-wise Discretization Errors in Boundary Element Method for Elasticity Problem Bart F. Zalewski Case Western Reserve University Robert L. Mullen Case.
Budapest May 27, 2008 Unifying mixed linear models and the MASH algorithm for breakpoint detection and correction Anders Grimvall, Sackmone Sirisack, Agne.
The Finite Element Method Defined
EARS1160 – Numerical Methods notes by G. Houseman
Visual Recognition Tutorial
ECIV 720 A Advanced Structural Mechanics and Analysis
ECIV 201 Computational Methods for Civil Engineers Richard P. Ray, Ph.D., P.E. Error Analysis.
Lecture 2: Numerical Differentiation. Derivative as a gradient
Finite Element Method Introduction General Principle
SYSTEMS Identification
2-1 Sample Spaces and Events Conducting an experiment, in day-to-day repetitions of the measurement the results can differ slightly because of small.
Boyce/DiPrima 9th ed, Ch 11.2: Sturm-Liouville Boundary Value Problems Elementary Differential Equations and Boundary Value Problems, 9th edition, by.
MCE 561 Computational Methods in Solid Mechanics
Lecture II-2: Probability Review
1 10. Joint Moments and Joint Characteristic Functions Following section 6, in this section we shall introduce various parameters to compactly represent.
Radial Basis Function Networks
Normalised Least Mean-Square Adaptive Filtering
Reduced-order modeling of stochastic transport processes Materials Process Design and Control Laboratory Swagato Acharjee and Nicholas Zabaras Materials.
Component Reliability Analysis
Ch 8.1 Numerical Methods: The Euler or Tangent Line Method
1 CE 530 Molecular Simulation Lecture 7 David A. Kofke Department of Chemical Engineering SUNY Buffalo
1 Statistical Mechanics and Multi- Scale Simulation Methods ChBE Prof. C. Heath Turner Lecture 11 Some materials adapted from Prof. Keith E. Gubbins:
An efficient Bayesian inference approach to inverse problems based on an adaptive sparse grid collocation method Xiang Ma and Nicholas Zabaras Materials.
01/24/05© 2005 University of Wisconsin Last Time Raytracing and PBRT Structure Radiometric quantities.
Development of An ERROR ESTIMATE P M V Subbarao Professor Mechanical Engineering Department A Tolerance to Error Generates New Information….
Materials Process Design and Control Laboratory THE STEFAN PROBLEM: A STOCHASTIC ANALYSIS USING THE EXTENDED FINITE ELEMENT METHOD Baskar Ganapathysubramanian,
Kalman filtering techniques for parameter estimation Jared Barber Department of Mathematics, University of Pittsburgh Work with Ivan Yotov and Mark Tronzo.
Extrapolation Models for Convergence Acceleration and Function ’ s Extension David Levin Tel-Aviv University MAIA Erice 2013.
Stochastic Linear Programming by Series of Monte-Carlo Estimators Leonidas SAKALAUSKAS Institute of Mathematics&Informatics Vilnius, Lithuania
Module 1: Statistical Issues in Micro simulation Paul Sousa.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.
Materials Process Design and Control Laboratory 1 High-dimensional model representation technique for the solution of stochastic PDEs Nicholas Zabaras.
1 EEE 431 Computational Methods in Electrodynamics Lecture 4 By Dr. Rasime Uyguroglu
Non-Intrusive Stochastic Uncertainty Quantification Methods Don Zhang University of Southern California Uncertainty Quantification Workshop.
Materials Process Design and Control Laboratory 1 Stochastic Modeling in High-Dimensional Spaces Nicholas Zabaras Materials Process Design and Control.
CHAPTER 4 Adaptive Tapped-delay-line Filters Using the Least Squares Adaptive Filtering.
Tarek A. El-Moselhy and Luca Daniel
Materials Process Design and Control Laboratory Finite Element Modeling of the Deformation of 3D Polycrystals Including the Effect of Grain Size Wei Li.
6. Introduction to Spectral method. Finite difference method – approximate a function locally using lower order interpolating polynomials. Spectral method.
Uncertainty quantification in multiscale deformation processes Babak Kouchmeshky Nicholas Zabaras Materials Process Design and Control Laboratory Sibley.
The Dirichlet Labeling Process for Functional Data Analysis XuanLong Nguyen & Alan E. Gelfand Duke University Machine Learning Group Presented by Lu Ren.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
1 Complex Images k’k’ k”k” k0k0 -k0-k0 branch cut   k 0 pole C1C1 C0C0 from the Sommerfeld identity, the complex exponentials must be a function.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: ML and Simple Regression Bias of the ML Estimate Variance of the ML Estimate.
Materials Process Design and Control Laboratory 1 An efficient Bayesian inference approach to inverse problems based on an adaptive sparse grid collocation.
Spatial Analysis & Geostatistics Methods of Interpolation Linear interpolation using an equation to compute z at any point on a triangle.
Materials Process Design and Control Laboratory 1 An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential.
ECE-7000: Nonlinear Dynamical Systems Overfitting and model costs Overfitting  The more free parameters a model has, the better it can be adapted.
HEAT TRANSFER FINITE ELEMENT FORMULATION
Review of Probability. Important Topics 1 Random Variables and Probability Distributions 2 Expected Values, Mean, and Variance 3 Two Random Variables.
Discretization Methods Chapter 2. Training Manual May 15, 2001 Inventory # Discretization Methods Topics Equations and The Goal Brief overview.
Advanced Engineering Mathematics, 7 th Edition Peter V. O’Neil © 2012 Cengage Learning Engineering. All Rights Reserved. CHAPTER 4 Series Solutions.
Variational and Weighted Residual Methods
Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering National Taiwan University 1/45 GEOSTATISTICS INTRODUCTION.
In Chapters 6 and 8, we will see how to use the integral to solve problems concerning:  Volumes  Lengths of curves  Population predictions  Cardiac.
Joint Moments and Joint Characteristic Functions.
Materials Process Design and Control Laboratory Sibley School of Mechanical and Aerospace Engineering 169 Frank H. T. Rhodes Hall Cornell University Ithaca,
A Brief Maximum Entropy Tutorial Presenter: Davidson Date: 2009/02/04 Original Author: Adam Berger, 1996/07/05
Boyce/DiPrima 9 th ed, Ch 11.3: Non- Homogeneous Boundary Value Problems Elementary Differential Equations and Boundary Value Problems, 9 th edition, by.
Materials Process Design and Control Laboratory 1 A stochastic dimension reduction for stochastic PDEs Nicholas Zabaras Materials Process Design and Control.
CPH Dr. Charnigo Chap. 11 Notes Figure 11.2 provides a diagram which shows, at a glance, what a neural network does. Inputs X 1, X 2,.., X P are.
Fundamentals of Data Analysis Lecture 11 Methods of parametric estimation.
Chapter 4 Dynamical Behavior of Processes Homework 6 Construct an s-Function model of the interacting tank-in-series system and compare its simulation.
Chapter 7. Classification and Prediction
Chapter 4 Dynamical Behavior of Processes Homework 6 Construct an s-Function model of the interacting tank-in-series system and compare its simulation.
Finite Volume Method for Unsteady Flows
Filtering and State Estimation: Basic Concepts
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
Presentation transcript:

Materials Process Design and Control Laboratory 1 An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations Nicholas Zabaras and Xiang Ma Materials Process Design and Control Laboratory Sibley School of Mechanical and Aerospace Engineering 101 Frank H. T. Rhodes Hall Cornell University Ithaca, NY URL:

Materials Process Design and Control Laboratory 2 Outline of the presentation  The topic of this presentation is going to address the following two issues: 1.Solve the problems with stochastic discontinuity; 2.Solve the problems of high stochastic dimension (>100). Using the following two techniques: 1.Adaptive sparse grid collocation method (ASGC) 1. 2.High dimensional model representation technique (HDMR) X. Ma, N. Zabaras, A hierarchical adaptive sparse grid collocation method for the solution of stochastic differential equations, Journal of Computational physics, Vol. 228, pp , X. Ma, N. Zabaras, A adaptive high dimensional stochastic model representation technique for the solution of stochastic differential equations, Journal of Computational physics, submitted, 2009.

Materials Process Design and Control Laboratory 3 Problem definition  Define a complete probability space. We are interested to find a stochastic function such that for P-almost everywhere (a.e.), the following equation hold: where are the coordinates in, L is (linear/nonlinear) differential operator, and B is a boundary operators.  In the most general case, the operator L and B as well as the driving terms f and g, can be assumed random.  In general, we require an infinite number of random variables to completely characterize a stochastic process. This poses a numerical challenge in modeling uncertainty in physical quantities that have spatio-temporal variations, hence necessitating the need for a reduced-order representation.  By using Karhunen-Loève expansion, the random input can be characterized by a set of random variables.

Materials Process Design and Control Laboratory 4 The finite-dimensional noise assumption  By using the Doob-Dynkin lemma, the solution of the problem can be described by the same set of random variables, i.e.  So the original problem can be restated as: Find the stochastic function such that  In this work, we assume that are independent random variables with probability density function. Let be the image of. Then is the joint probability density of with support

Materials Process Design and Control Laboratory 5 Adaptive Sparse Grid Collocation (ASGC)

Materials Process Design and Control Laboratory 666 Stochastic Collocation based framework Function value at any point is simply Stochastic function in 2 dimensions Need to represent this function Sample the function at a finite set of points Use basis function to get a approximate representation Spatial domain is approximated using a FE, FD, or FV discretization. Stochastic domain is approximated using multidimensional interpolating functions

Materials Process Design and Control Laboratory 777 Choice of collocation points and nodal basis functions  In the context of incorporating adaptivity, we have used the Newton-Cotes grid using equidistant support nodes and use the linear hat function as the univariate nodal basis.  Furthermore, by using the linear hat function as the univariate nodal basis function, one ensures a local support in contrast to the global support of Lagrange polynomials. This ensures that discontinuities in the stochastic space can be resolved. The piecewise linear basis functions can be defined as

Materials Process Design and Control Laboratory 88 LET OUR BASIC 1D INTERPOLATION SCHEME BE SUMMARIZED AS IN MULTIPLE DIMENSIONS, THIS CAN BE WRITTEN AS  Denote the one dimensional interpolation formula as  In higher dimension, a simple case is the tensor product formula Conventional sparse grid collocation (CSGC)  Using the 1D formula, the sparse interpolant, where is the depth of sparse grid interpolation and is the number of stochastic dimensions, is given by the Smolyak algorithm as  Here we define the hierarchical surplus as:

Materials Process Design and Control Laboratory 9 Nodal basis vs. hierarchical basis Nodal basis Hierarchical basis

Materials Process Design and Control Laboratory 1010 Hierarchical Integration  The mean of the random solution can be evaluated as follows:  Denoting we rewrite the mean as  To obtain the variance of the solution, we need to first obtain an approximate expression for

Materials Process Design and Control Laboratory Adaptive sparse grid collocation (ASGC) 1 Let us first revisit the 1D hierarchical interpolation  For smooth functions, the hierarchical surpluses tend to zero as the interpolation level increases.  On the other hand, for non-smooth function, steep gradients/finite discontinuities are indicated by the magnitude of the hierarchical surplus.  The bigger the magnitude is, the stronger the underlying discontinuity is.  Therefore, the hierarchical surplus is a natural candidate for error control and implementation of adaptivity. If the hierarchical surplus is larger than a pre-defined value (threshold), we simply add the 2N neighbor points of the current point. 1.X. Ma, N. Zabaras, An hierarchical adaptive sparse grid collocation method for the solution of stochastic differential equations, JCP, 228 (2009)

Materials Process Design and Control Laboratory 12 Adaptive sparse grid collocation  As we mentioned before, by using the equidistant nodes, it is easy to refine the grid locally around the non-smooth region.  It is easy to see this if we consider the 1D equidistant points of the sparse grid as a tree-like data structure  Then we can consider the interpolation level of a grid point Y as the depth of the tree D(Y). For example, the level of point 0.25 is 3.  Denote the father of a grid point as F(Y), where the father of the root 0.5 is itself, i.e. F(0.5) = 0.5.  Thus the conventional sparse grid in N- dimensional random space can be reconsidered as

Materials Process Design and Control Laboratory 13 Adaptive sparse grid collocation  Thus, we denote the sons of a grid point by  From this definition, it is noted that, in general, for each grid point there are two sons in each dimension, therefore, for a grid point in a N- dimensional stochastic space, there are 2N sons.  The sons are also the neighbor points of the father, which are just the support nodes of the hierarchical basis functions in the next interpolation level.  By adding the neighbor points, we actually add the support nodes from the next interpolation level, i.e. we perform interpolation from level |i| to level |i|+1.  Therefore, in this way, we refine the grid locally while not violating the developments of the Smolyak algorithm

Materials Process Design and Control Laboratory 14 NUMERICAL EXAMPLES

Materials Process Design and Control Laboratory 15 Adaptive sparse grid interpolation Ability to detect and reconstruct steep gradients

Materials Process Design and Control Laboratory 1616  The discontinuity then occurs at the planes and.  We first study the stochastic response subject to the following random 1D initial input: where. Since the random initial data can cross the plane we know that gPC will fail for this case. (X.Wan, G. Karniadakis, 2005)2005  The governing equations are: The Kraichnan-Orszag three-mode problem

Materials Process Design and Control Laboratory 17 One-dimensional random input: variance  gPC fails after t = 6s while the ASGC method converges even with a large threshold. The solid line is the result from MC-SOBOL sequence with 10 6 iterations.

Materials Process Design and Control Laboratory 18 One-dimensional random input: adaptive sparse grid  From the adaptive sparse grid, it is seen that most of the refinement after level 8 occurs around the discontinuity point Y = 0.0, which is consistent with previous discussion.  The refinement stops at level 16, which corresponds to 425 number of points, while the conventional sparse grid requires points. Adaptive sparse grid with

Materials Process Design and Control Laboratory 19 One-dimensional random input The ‘exact’ solution is taken as the results given by MC- SOBOL The error level is defined as

Materials Process Design and Control Laboratory 20 Two-dimensional random input  Next, we study the K-O problem with 2D input:  Now, instead of a point, the discontinuity region becomes a line.

Materials Process Design and Control Laboratory 21 Two-dimensional random input  It can be seen that even though the gPC fails at a larger time, the adaptive collocation method converges to the reference solution given by MC-SOBOL with 10 6 iterations.

Materials Process Design and Control Laboratory 22 Two-dimensional random input The error level is defined as The number of points of the conventional grid with interpolation 20 is

Materials Process Design and Control Laboratory 23 Stochastic elliptic problem  Here, we adopt the model problem with the physical domain. To avoid introducing errors of any significance from the domain discretization, we take a deterministic smooth load with homogeneous boundary conditions.  The deterministic problem is solved using the finite element method with 900 bilinear quadrilateral elements. Furthermore, in order to eliminate the errors associated with a numerical K-L expansion solver and to keep the random diffusivity strictly positive, we construct the random diffusion coefficient with 1D spatial dependence as where are independent uniformly distributed random variables in the interval

Materials Process Design and Control Laboratory 2424 Stochastic elliptic problem  In the earlier expansion, and  The parameter can be taken as and the parameter L is

Materials Process Design and Control Laboratory 25 Stochastic elliptic problem  This expansion is similar to a K-L expansion of a 1D random field with stationary covariance  Small values of the correlation L correspond to a slow decay, i.e. each stochastic dimension weighs almost equally. On the other hand, large values of L results in fast decay rates, i.e., the first several stochastic dimensions corresponds to large eigenvalues weigh relatively more.  By using this expansion, it is assumed that we are given an analytic stochastic input. Thus, there is no truncation error. This is different from the discretization of a random filed using the K-L expansion, where for different correlation lengths we keep different terms accordingly.  In this example, we fix N and change L to adjust the importance of each stochastic dimension. In this way, we investigate the effects of L on the ability of the ASGC method to detect the important dimensions.

Materials Process Design and Control Laboratory 26 Stochastic elliptic problem: N = 11

Materials Process Design and Control Laboratory 27 Stochastic elliptic problem: N = 11  We estimate the approximation error for the mean and variance. Specifically, to estimate the computation error in the q-th level, we fix N and compare the results at two consecutive levels, e.g. the error for the mean is  The previous figures shows results for different correlation lengths at N=11.  For small L, the convergence rates for the CSGC and ASGC are nearly the same. On the other hand, for large L, the ASGC method requires much less number of collocation points than the CSGC for the same accuracy.  This is because more points are placed only along the important dimensions which are associated with large eigenvalues.  Next, we study some higher-dimensional cases. Due to the rapid increase in the number of collocation points, we focus on moderate high correlation lengths so that the ASGC is effective.

Materials Process Design and Control Laboratory 28 Stochastic elliptic problem: higher-dimensions #points:20271 Comparison: CSGC:

Materials Process Design and Control Laboratory 29 Stochastic elliptic problem: higher-dimensions  In order to further verify our results, we compare the mean and the variance when N = 75 using the AGSC method with with the ‘exact’ solution obtained by MC-SOBOL simulation with 10 6 iterations. Computational time: ASGC : 0.5 hour MC: 2 hour

Materials Process Design and Control Laboratory 30 Stochastic elliptic problem: N=25  Correlation length affects the convergence of ASGC. If every dimension is equally important, the ASGC is not effective. It is even worse than standard MC method for a high dimensional problem as a result of the weak dependence on the dimensionality in the logarithmic term of the error bound.

Materials Process Design and Control Laboratory 31 High Dimensional Model Representation (HDMR)

Materials Process Design and Control Laboratory 3232 High dimensional model representation (HDMR) 1  Let be a real-value multivariate stochastic function:, which depends on a N-dimensional random vector. A HDMR of can be described by where the interior sum is over all sets of integers, that satisfy.This relation means that can be viewed as a finite hierarchical correlated function expansion in terms of the input random variables with increasing dimensions.  For most well-defined physical systems, the first- and second-order expansion terms are expected to have most of the impact upon the output and the contribution of higher-order terms would be insignificant. 1. O. F. Alis, H. Rabitz, General foundations of high dimensional representations, Journal of Mathematical Chemistry, 25 (1999)

Materials Process Design and Control Laboratory 3333 High dimensional model representation (HDMR) In this expansion:  denotes the zeroth-order effect which is a constant.  The component function gives the effect of the variable acting independently of the other input variables.  The component function describes the interactive effects of the variables and. Higher-order terms reflect the cooperative effects of increasing numbers of variables acting together to impact upon.  The last term gives any residual dependence of all the variables locked together in a cooperative way to influence the output.  Depending on the method to represent component functions, there are two types of HDMR: ANOVA-HDMR and CUT-HDMR.

Materials Process Design and Control Laboratory 34 HDMR: Compact notation  This equation is often written in a more compact notation: for a given set where denotes the set of coordinate indices and. Here, denotes the - dimensional vector containing those components of whose indices belong to the set, where is the cardinality of the corresponding set, i.e..  The component functions can be explicitly given as where the projection operator is defined as where induced by the measure

Materials Process Design and Control Laboratory 35 HDMR ANOVA-HDMRCUT-HDMR Lebesgue measure Dirac measure dimensional integration dimensional function at a reference point Computational expensiveComputational efficient

Materials Process Design and Control Laboratory 36 CUT-HDMR  Within the framework of CUT-HDMR, we can write where the notation means that the components of other than those indices that belong to the set equal to those of the reference point.  Therefore, the -dimensional stochastic problem is transformed to several lower-order -dimensional problems which can easily solved by ASGC:  If the HDMR is a converged expansion, the choice of this point does not affect the approximation. In this work, the mean of the random input vector is chosen as the reference point.

Materials Process Design and Control Laboratory 37 CUT-HDMR  Let us denote as the mean of the component function. Then the mean of the HDMR expansion is simply.  It is common to refer to the terms collectively as the “order- terms”. Then the expansion order is the maximum of. The number of collocation points in this expansion is defined as the sum of the number of points for each sub-problem, i.e.  In practice, we always truncate the expansion by taking only a subset of all indices. We can define an interpolation formula for the approximation of as  However, the number of order- component function is, which increases quickly with the number of dimensions. Therefore, we developed an adaptive version of HDMR.

Materials Process Design and Control Laboratory 38 Adaptive HDMR  First, we try to find the important dimensions. To this end, we always construct the zeroth- and first-order HDMR expansion. We define a weight: Then we define the important dimensions as those whose weights are larger than a predefined error threshold. Only higher order terms which consist of only these important dimensions are considered.  However, not all the possible terms are computed. For higher-order term, a weight is also defined as We also define the important terms in a similar way. We put all the important dimensions and higher-order terms in to a set. When adaptively constructing HDMR for each new order, we then only calculate the term whose indices satisfy the admissibility relation

Materials Process Design and Control Laboratory 39 Adaptive HDMR  In this way, we try to find those terms which may have significant contributions to the overall expansion while ignoring other trivial terms in order to reduce the computational cost for extremely high-dimensional problems.  Let us denote the order of expansion as Furthermore, we also define a relative error of the integral value between two consecutive expansion orders and as  If is smaller than another predefined error threshold, the HDMR is regarded as converged and the construction stops.

Materials Process Design and Control Laboratory 4040 Numerical example: Flow through random media Basic equation for pressure and velocity in a domain where denotes the source/sink term. A mixed finite element method is utilized to solve the forward problem.  To impose the non-negativity of the permeability, we will treat the permeability as a log- normal random field obtained from the K-L expansion injection well production well where is a zero mean random field with a covariance function where is the correlation length and is the standard deviation.

Materials Process Design and Control Laboratory 41 Numerical example: K-L Expansion  The eigenvalues and their corresponding eigenfunctions can be determined analytically. The are assumed as i.i.d uniform random variables on [-1,1].  According to the decay rate of eigenvalues, the number of stochastic dimensions is and, respectively for and. Series of eigenvalues and their finite sums for three different correlation lengths at  Monte Carlo simulations are conducted for the purpose of comparison. For each case, the reference solution is taken from samples and all errors are defined as normalized errors. In all cases,.

Materials Process Design and Control Laboratory 42 Standard deviation for different correlation lengths Standard deviation of the velocity-component along the cross section for different correlation lengths

Materials Process Design and Control Laboratory 43 PDF at (0,0.5) for different correlation lengths PDF of the velocity-component at point for different correlation lengths

Materials Process Design and Control Laboratory 44 Convergence of the normalized errors Convergence of the normalized errors of the standard deviation of the velocity-component for different correlation lengths

Materials Process Design and Control Laboratory 45 Standard deviations for different with Standard deviation of the velocity-component along the cross section for different.

Materials Process Design and Control Laboratory 46 PDF at (0,0.5) for different with PDF of the velocity-component at point for different

Materials Process Design and Control Laboratory 47 Convergence of the normalized errors with Convergence of the normalized errors of the standard deviation of the velocity-component for different

Materials Process Design and Control Laboratory 48 Conclusions  An adaptive hierarchical sparse grid collocation method is developed based on the error control of local hierarchical surpluses.  Besides the detection of important and unimportant dimensions as dimension-adaptive methods can do, additional singularity and local behavior of the solution can also be revealed.  By combining HDMR and ASGC, extremely high dimensional stochastic problem can be solve accurately and efficiently.  The numerical examples show that the number of component functions needed in the HDMR for a fixed stochastic dimension depends more on the input variability.  This method to our knowledge is the first approach which can solve high-dimensional stochastic problems by reducing dimensions from truncation of HDMR and resolve low-regularity by local adaptivity through ASGC.