1. MULTISCALE COMPUTATION: From Fast Solvers To Systematic Upscaling A. Brandt The Weizmann Institute of Science UCLA www.wisdom.weizmann.ac.il/~achi

2. Major scaling bottlenecks: computing Elementary particles (QCD) Schrödinger equation molecules condensed matter Molecular dynamics protein folding, fluids, materials Turbulence, weather, combustion,… Inverse problems da, control, medical imaging Vision, recognition

3. Scale-born obstacles: Many variables n gridpoints / particles / pixels / … Interacting with each other O(n 2 ) Slowness Slow Monte Carlo / Small time steps / … Slowly converging iterations / due to 1.Localness of processing

4. small step Moving one particle at a time fast local ordering slow global move

5. Solving PDE : Influence of pointwise relaxation on the error Error of initial guess Error after 5 relaxation sweeps Error after 10 relaxations Error after 15 relaxations Fast error smoothing slow solution

6. Scale-born obstacles: Many variables n gridpoints / particles / pixels / … Interacting with each other O(n 2 ) Slowness Slow Monte Carlo / Small time steps / … Slowly converging iterations / due to 1.Localness of processing 2. Attraction basins

8. Fluids Gas/Liquid 1.Positional clustering Lennard-Jones r0r0 2.Electrostatic clustering Dipoles Water: 1& 2

9. r E(r) Optimization min E(r) multi-scale attraction basins

10. Scale-born obstacles: Many variables n gridpoints / particles / pixels / … Interacting with each other O(n 2 ) Slowness Slow Monte Carlo / Small time steps / … Slowly converging iterations / due to 1.Localness of processing 2. Attraction basins Removed by multiscale processing

11. Solving PDE : Influence of pointwise relaxation on the error Error of initial guess Error after 5 relaxation sweeps Error after 10 relaxations Error after 15 relaxations Fast error smoothing slow solution

12. LU = F h 2h 4h L h U h = F h L 2h U 2h = F 2h L 4h V 4h = R 4h L 2h V 2h = R 2h

13. interpolation (order l+p) to a new grid interpolation (order m) of corrections relaxation sweeps algebraic error < truncation error residual transfer enough sweeps or direct solver * *** Full MultiGrid (FMG) algorithm... * h0h0 h 0 /2 h 0 /4 2h h

14. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)

15. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971) Nonlinear FAS (1975)

16. LU = F h 2h 4h L h U h = F h L 4h U 4h = F 4h Fine-to-coarse defect correction L 2h V 2h = R 2h U 2h = U h,approximate +V 2h L 2h U 2h = F 2h

17. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

18. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

19. Same fast solver Local patches of finer grids Each level correct the equations of the next coarser level Each patch may use different coordinate system and anisotropic grid “Quasicontiuum” method [B., 1992] Each patch may use different coordinate system and anisotropic grid and different physics; e.g. Atomistic and differet physics; e.g. atomistic

21. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

23. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

24. ALGEBRAIC MULTIGRID (AMG) 1982

25. Classical ALGEBRAIC MULTIGRID (AMG) 1982 Ax = b, "M matrix" a ii a ij a ij = a ji  0 (i = 1,…,n) Relaxation  approximation 1 2 3 4 5 a 23 a 34 a 35

26. Classical ALGEBRAIC MULTIGRID (AMG) 1982 1 2 3 4 5 a 23 a 34 a 35 Ax = b, "M matrix" a ii a ij a ij = a ji  0 (i = 1,…,n) Coarse variables - a subset ( x c ) 3 = ( a 23 x 2 + a 34 x 4 )/( a 23 + a 34 ) AMG Cycle Relaxation  approximation Solve A c x c = T (b - A ) (recursion)  + x c

27. ALGEBRAIC MULTIGRID (AMG) 1982 Coarse variables - a subset 1. “General” linear systems 2. Variety of graph problems

28. Graph problems Partition: min cut Clustering bioinformatics Image segmentation VLSI placement Routing Linear arrangement: bandwidth, cutwidth Graph drawing low dimension embedding Coarsening: weighted aggregation Recursion: inherited couplings (like AMG) Modified by properties of coarse aggregates General principle: Multilevel objectives

29. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

30. 2h h 2  wavelength Non-local components: e i  x,  ≈ ±k Slow to converge in local processing The error after relaxation v(x) = A 1 (x) e ikx + A 2 (x) e -ikx A 1 (x), A 2 (x) smooth A r (x) are represented on coarser grids: A 1  + 2 i k A 1 ′ = f 1 = r h (x) e -ikx 1D Wave Equation: u”+k 2 u=f

31.  k   8,  8 )  1,  1 )  2,  2 )  3,  3 )  4,  4 )  5,  5 )  6,  6 )  7,  7 ) O(H) 2D Wave Equation: Du+k 2 u=f Non-local: e i(   x +  2 y)    +    ≈ k 2 On coarser grid (meshsize H):  Fully efficient multigrid solver  Tends to Geometrical Optics  Radiation Boundary Conditions: directly on coarsest level

32. Σ r = 1 m A r (x) φ r (x) Generally: LU=F Non-local part of U has the form L φ r ≈ 0 A r (x) smooth {φ r } found by local processing A r represented on a coarser grid

33. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

34. N eigenfunctions Electronic structures (Kohn-Sham eq): i = 1, …, N = # electrons O (N) gridpoints per  i O (N 2 ) storage Orthogonalization O (N 3 ) operations O (N log N) storage & operations Multiscale eigenbase 1D: Livne V = V nuclear + V(  ) One shot solver

35. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Full matrix Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

36. Integro-differential Equation differential, dense Multigrid solver Distributive relaxation: 1 st order 2 nd order Solution cost ≈ one fast transform (one fast evaluation of the discretized integral transform)

37. Integral Transforms G(x,  Transform Fourier Laplace Gauss Potential Complexity  n logn)  n) G(x,  Exp(ik  Waves  n logn)

38. G local G(x,y) G smooth s|x-y| G(x,y) = G smooth (x,y) + G local (x,y) s ~ next coarser scale ~ 1 / | x – y | O(n) not static!

39. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Monte-Carlo Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

40. Discretization Lattice for accuracy Monte Carlo cost ~ “volume factor” “critical slowing down” Multiscale ~ Multigrid moves Many sampling cycles at coarse levels

41. Multigrid solvers Cost: 25-100 operations per unknown Linear scalar elliptic equation (~1971)* Nonlinear Grid adaptation General boundaries, BCs* Discontinuous coefficients Disordered: coefficients, grid (FE) AMG Several coupled PDEs* (1980) Non-elliptic: high-Reynolds flow Highly indefinite: waves Many eigenfunctions (N) Near zero modes Gauge topology: Dirac eq. Inverse problems Optimal design Integral equations Statistical mechanics Massive parallel processing *Rigorous quantitative analysis (1986) FAS (1975) Within one solver (1977,1982)

42. Same fast solver Local patches of finer grids Each level correct the equations of the next coarser level Each patch may use different coordinate system and anisotropic grid “Quasicontiuum” method [B., 1992] Each patch may use different coordinate system and anisotropic grid and different physics; e.g. Atomistic and differet physics; e.g. atomistic

43. Repetitive systems e.g., same equations everywhere UPSCALING: Derivation of coarse equations in small windows

44. Scale-born obstacles: Many variables n gridpoints / particles / pixels / … Interacting with each other O(n 2 ) Slowness Slow Monte Carlo / Small time steps / … Slowly converging iterations / due to 1.Localness of processing 2.Attraction basins Removed by multiscale processing

45. A solution value is NOT generally determined just by few local equations A coarse equation IS generally determined just by few local equations  O (N) operations The coarse equation can be derived ONCE for all similar neighborhoods  # operations << N

46. Systematic Upscaling 1.Choosing coarse variables 2.Constructing coarse-level operational rules equations Hamiltonian

50. Macromolecule ~ 10 -15 second steps

51. Systematic Upscaling 1.Choosing coarse variables Criterion: Fast convergence of “compatible relaxation”

52. Systematic Upscaling 1.Choosing coarse variables Criterion: Fast equilibration of “compatible Monte Carlo” OR: Fast convergence of “compatible relaxation” Local dependence on coarse variables 2.Constructing coarse-level operational rules Done locally In representative “windows” fast

53. Systematic Upscaling 1.Choosing coarse variables Criterion: Fast equilibration of “compatible Monte Carlo” Local dependence on coarse variables 2.Constructing coarse-level operational rules Done locally In representative “windows” fast

54. Macromolecule

55. Potential Energy Lennard-Jones Electrostatic Bond length strain Bond angle strain torsion hydrogen bond rkrk E  ijkl riri rjrj rlrl r ij

56. Macromolecule Two orders of magnitude faster simulation

57. Macromolecule      + Lennard-Jones ~10 4 Monte Carlo passes for one T G i transition G1G1 G2G2 T Dihedral potential + Electrostatic

59. Fluids £ Total mass £ Total momentum £ Total dipole moment £ average location

60. 1 1 2

61. Hierarchy of Coarser levels Total mass at scale s at point x: summed recursively densities at all scales Summing recursively density variations at any scale averaged to all higher scales s s s - 1

62. Windows Coarser level Larger density fluctuations Still coarser level

63. Fluids Total mass: Summing

64. Lower Temperature T Summing also Still lower T: More precise crystal direction and periods determined at coarser spatial levels Heisenberg uncertainty principle : Better orientational resolution at larger spatial scales

65. Optimization by Multiscale annealing Identifying increasingly larger-scale degrees of freedom at progressively lower temperatures Handling multiscale attraction basins E(r) r

66. Systematic Upscaling Rigorous computational methodology to derive from physical laws at microscopic (e.g., atomistic) level governing equations at increasingly larger scales. Scales are increased gradually (e.g., doubled at each level) with interscale feedbacks, yielding: Inexpensive computation : needed only in some small “windows” at each scale. No need to sum long-range interactions Applicable to fluids, solids, macromolecules, electronic structures, elementary particles, turbulence, … Efficient transitions between meta-stable configurations.

67. Upscaling Projects QCD (elementary particles): Renormalization multigrid Ron BAMG solver of Dirac eqs. Livne, Livshits Fast update of, det Rozantsev (3n +1) dimensional Schrödinger eq. Filinov Real-time Feynmann path integrals Zlochin multiscale electronic-density functional DFT electronic structures Livne, Livshits molecular dynamics Molecular dynamics: Fluids Ilyin, Suwain, Makedonska Polymers, proteins Bai, Klug Micromechanical structures Ghoniem defects, dislocations, grains Navier Stokes Turbulence McWilliams Dinar, Diskin

68. THANK YOU

70. Aggregating Regions Adaptively e.g., by similarity of densities astrophysics heights epitaxial growth color image segmentation color variances at all scales elongation continuation deblurring shapes recognition …

71. Cure: Multiscale Computation  Define a Coarser system  Derive equations (or probabilistic rules) governing the coarse system  Move similarly to a still-coarser system; etc.  Small computational volumes at each scale  No need to sum far interactions  No slowness  Leading to macroscopic “equations” (or tabled rules) of the material

72. Exact Quantum Mechanics n masses m 1, …, m n located at r 1, …, r n r j =(x j, y j, z j ) Forcespotential V(r 1, …, r n ) Classical: r(t) Probability amplitude  (r 1, …, r n, t) Approximations: Born-Oppenheimer; Hartree-Fock; Local density; perturbations Direct Numerical Real-time path integrals 3n+1 dimensional PDE (Schrodinger): roro

73. F cycle h0h0 h 0 /2 h 0 /4 2h h ***... * interpolation (order l+p) to a new grid interpolation (order m) of corrections relaxation sweeps algebraic error < truncation error residual transfer enough sweeps or direct solver * residual transfer no relaxation