Variance reduction and Brownian Simulation Methods Yossi Shamai Raz Kupferman The Hebrew University

All (incompressible) fluids are governed by mass- momentum conservation equations u(x,t) = velocity  (x,t) = polymeric stress  Dumbbell models

 (q,x,t) = pdf. The polymers are modeled by two beads connected by a spring (dumbbell). The conformation is modeled by an end-to-end vector q. less affect more affect q

The (random) conformations are distributed according to a density function  (q,x,t), which satisfies an evolution equation advectiondeformationdiffusion The stress is an ensemble average of polymeric conformations, q g(q) = q  F(q)

The stress Conservation laws (macroscopic dynamics) Polymeric density distribution (microscopic dynamics) Problem: high dimensionality Assumption: 1-D

Closable systems In certain cases, a PDE for  (x,t) can be derived, yielding a closed-form system for u(x,t),  (x,t). Closable systems can be solved by standard methods. Brownian simulations can be used for non-closable systems. Example (Semi-linear systems): if g(q) = q 2, and b(q,u) = b(u) q then  (x,t) satisfies the PDE

Outline 1. Brownian simulation methods 2. Some mathematical preliminaries on spatial correlations 3. A variance reduction mechanism in Brownian simulations 4. Examples

Brownian simulations The average stress  (x,t) is an expectation with respect to a stochastic process q(x,t) with PDF  (q,x,t) q(x,t) is simulated by a collection of realizations q i (x,t). The stress is approximated by an empirical mean PDESPDE

A reminder: real-valued Brownian motion 1.B(t) is a random function of time. 2.Almost surely continues. 3.Independent increments. 4.B(t)-B(s) ~ N(0,t-s). L 2 -valued Brownian motion 1.B(x,t) is a random function of time and space. 2.For fixed x, B(x,t) is a real-valued Brownian motion. 3.Finite norm

Spatial correlations B(x,t) is characterized by the spatial correlation function 1.Symmetry: c(x,y) = c(y,x). 2.c(x,x) = 1. 3.L 2 - function:

Spatial correlations (cont.) An L 2 - function is a correlation function iff a. c(x,x) = 1. b. It has a “square root” in L 2 Oscillatory Discretization Uniform Piecewise constant uncorrelated

No spatially uncorrelated L 2 -valued Brownian motion. Spatially uncorrelated noise has meaning only in a discrete setting. It is a sequence of piecewise constant standard Brownian motions, uncorrelated at any two distinct steps, that converges to 0. Spatial correlations (cont.)

Spatial correlations can be alternatively described by Correlation operators C is nonnegative, symmetric and trace class. For any f,g in L 2 No Id-correlated Brownian motion (trace Id = ∞ ).

SDEs versus SPDEs SDEs (Stochastic Differential Equations) F,G are operators SPDEs (Stochastic Partial Differential Equations) Ito’s integral

q(x,t) has spatial correlation. PDE (Fokker- Plank) SDE SDEs PDE (Fokker- Plank) SPDE SPDEs SDEs versus SPDEs

Brownian simulations unifying approach Equivalence class insensitive to spatial correlations.  Consistency: for every x, q(x,0) ~  (q,x,0).  Lemma: Let (u, ,q) be a solution for the stochastic system  on some time interval [0,T]. Let  (q,x,t) be the PDF corresponding to q(x,t). Then (u, ,  ) is a solution for the deterministic system  on [0,T].

Brownian simulation methods The stochastic process q is simulated by n “realizations” driven by i.i.d Brownian motions. Expectation is approximated by an empirical mean with respect to the realizations:  Advantages: 1.No Fokker-plank equation. 2.Easy to simulate. Disadvantages: 1. No error analysis. 2. Variance is O(n - 1 ).

Brownian simulation methods The approximation The system CONNFFESSIT (Calculations of Non Newtonian Fluids Finite Elements and Stochastic Simulation Techniques) - Piecewise constant uncorrelated noise (Ottinger et al. 1993) BCF - Spatially uniform noise (Hulsen et al. 1997)   Correlation affects approximation but not the exact solution Error reduction ?

1. Prove that e(n,t)  Reduce the error by choosing the spatial correlation of the Brownian noise: Step 1. Express e(n,t) as a function F(c). Step 2. Minimize F(c). The error of the Brownian simulations is Goals The idea of adapting correlation to minimize variance first proposed by Jourdain et al. (2004) in the context of shear flow with a specific FEM scheme.

An “integral-type” system The Brownian simulation is Example

Results: Brownian simulation The stress n = 2000 with spatially uniform noise ( c(x,y) = 1 ). The (normalized) error as a function of time Large error (1.47) “smooth” simulation The Brownian simulation at t=20 (dotted curve) Brownian simulation Stress

Results: The Brownian simulation at t=20 (dotted curve) “noisy” simulations n = 2000 with piecewise constant uncorrelated noise. The (normalized) error as a function of time reduced error (1.06)

Error analysis We want to analyze the error of the Brownian simulations Lets demonstrate the analysis for semi-linear system…

Closable systems In certain cases, a PDE for  (x,t) can be derived, yielding a closed-form system for u(x,t),  (x,t). Closable systems can be solved by standard methods. Brownian simulations can be used for non-closable systems. Example (Semi-linear systems): if g(q) = q 2, and b(q,u) = b(u) q then  (x,t) satisfies the PDE

Error analysis for Semi-linear systems Linearize (properly) In semi-linear systems, the stress field  (x,t) satisfies a PDE We want to estimate the error of the Brownian simulations An analogous evolution equation for T(x,t) is derived

 Linearized system and

Theorem 1. To leading order: and k is a kernel function determined by the parameters. where The function F can be also expressed in terms of the correlation operator C,

F is convex In principle, the analysis is the same Proof is restricted to closable systems Theorem 1. To leading order in n, Error analysis for Closable systems

The optimization problem Minimize F(c) over the domain: S = {c(x,y) : c has a root in L 2, c(x,x) = 1} Difficulties: A. Infinite dimensional optimization problem. B. S is not compact. In general, there is no minimizer Find a sequence of correlations c n  S such that F(c n ) converge to

Finite dimensional approximations 1. Set a natural k. 2. Discretize the problem to a k-point mesh Theorem 2. The sequence of errors converges (as k  ∞ ) to the optimal error

The F-D optimization problem The F-D optimization problem is: Minimize F(A), A is k-by-k symmetric PSD Subject to A ii = 1, i=1,…,k We want to minimize F(c k ) over S k. c k (x,y) is indexed by k 2 mesh points (x i,x j ) (matrix). Symmetric Positive-Semi-Definite. c k (x i,x i ) = 1. F is convex  SDP algorithms (Semi-Definite Programming)

So what did we do?  Developed a unifying approach for a variance reduction mechanism in Brownian simulations.  Formulated an optimization problem (in infinite dimensions).  Showed that it is amenable to a standard algorithm (SDP).

Example 1 A linear advection-dissipation equation in [0,1]. In stochastic formulation, The Brownian simulation is

The error is Variance independent of correlations (no reduction) Insights: the dynamics (advection and dissipation) do not mix different points in space. Thus, the error only ‘sees’ diagonal elements of the correlations, which are fixed by the constraints.

An “integral-type” system (x  [0,1]): Closable: Example 2

Results: Brownian simulation The stress n = 2000 with spatially uniform noise ( c(x,y) = 1 ). (BCF) The (normalized) error as a function of time Large error (1.47) “smooth” simulation The Brownian simulation at t=20 (dotted curve) Brownian simulation Stress

Results: The Brownian simulation at t=20 (dotted curve) “noisy” simulations n = 2000 with piecewise constant uncorrelated noise (CONNFFESSIT). The (normalized) error as a function of time optimal error (1.06)

Why?…  the optimal error is obtained by taking c  0 (CONNFESSIT). g(x,y,t) is singular on the diagonal (x=y), and a smooth positive function off the diagonal. c(x,x) = 1 The error is

Example 3: 1-D planar Shear flow model. (Jourdain et al. 2004) The system: Closable: set

To leading order, the error of the Brownian simulations is C - the spatial correlation operator. K(t) - a nonnegative bounded operator. Any sequence c k  0 yields the optimal error (e.g, spatially constant uncorrelated)

So is CONFFESSIT always optimal? No! We can construct a problem for which e(n,t) = n -1 (const + Tr[K(t)C]) for K(t) bounded and not PSD. Theorem. If the semi-groups are Hilbert-Schmidt (they have L 2 -kernels) then CONNFFESSIT is optimal.

Some further thoughts… The spatial correlation of the initial data q(x,0) may also be considered. Non-closable systems? Gain insights about the optimal correlation by understand relations between type of equation and optimal correlation.