1. 2013 SIAM Great Lakes Section From PDEs to Information Science and Back Russel Caflisch IPAM Mathematics Department, UCLA 1

2. 2013 SIAM Great Lakes Section Collaborators & Support UCLA Stan Osher Hayden Schaeffer Oak Ridge National Labs Cory Hauck 2

3. Themes Over the last 20 years, there has been a lively influx of ideas and techniques from analysis and PDEs into information science –e.g., image processing Rapid progress in information science has produced wonderful mathematics –e.g., compressed sensing Ideas and techniques from info science are starting to be used for PDEs 2013 SIAM Great Lakes Section 3

4. Information Science From Wikipedia Information science (or information studies) is an interdisciplinary field primarily concerned with the analysis, collection, classification, manipulation, storage, retrieval, movement, and dissemination of information. This talk will focus on the analysis and manipulation of data in the form of images and signals. 2013 SIAM Great Lakes Section 4

5. Image Denoising 2013 SIAM Great Lakes Section 5

6. Image Denoising Removing noise from an image Example –From “Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods” by T. Chan and J. Shen (2005) 2013 SIAM Great Lakes Section 6

7. Denoising by Weiner Filter Noise is random or has rapid oscillations –So it can be canceled by local averaging Describe the original noisy image as a function –x is position and u is gray scale. The Wiener filter transforms u to 2013 SIAM Great Lakes Section 7

8. Weiner Filter as a PDE The Wiener filter transforms u to is the fundamental solution for the heat equation, i.e., Therefore in which 2013 SIAM Great Lakes Section 8

9. Denoising by Rudin-Osher PDE Variational principle for noise removal –L. Rudin, S. Osher and E. Fatemi (1992) –For noisy image u 0, the denoised image u minimizes Gradient descent is the nonlinear parabolic PDE 2013 SIAM Great Lakes Section 9

10. Significance of Rudin-Osher The Rudin-Osher variational principle –λ is a Lagrange multiplier –u minimizes, for constant value of measures total variation (TV) of u –TV used for nonlinear hyperbolic PDEs –Promotes steep gradients, as in shock waves and edges –Edges are dominant feature of images 2013 SIAM Great Lakes Section 10

11. Comparison of Rudin-Osher to Wiener Rudin-Osher variational principle –L 2 alternative leads to heat equation (with lower order terms), almost the same as the PDE for Wiener filtering Rather than promoting edges like Rudin-Osher, –Wiener filtering smooths gradients 2013 SIAM Great Lakes Section 11

12. Results for Rudin-Osher vs. Wiener 2013 SIAM Great Lakes Section 12 Rudin, Osher, Fatemi (1992)

13. Extensions of the Variational Approach to Imaging Applications 2013 SIAM Great Lakes Section 13 Segmentation Inpainting Texture

14. Image Segmentation Find boundaries Γ of objects in image region Ω –Active contour model: given image u, Γ minimizes – = average of u inside each component of Γ –T. Chan and L. Vese (2001) Earlier variational principle of Mumford-Shah 2013 SIAM Great Lakes Section 14

15. Image segmentation by Active Contour Model 2013 SIAM Great Lakes Section 15 Chan, Vese (2001)

16. Image Inpainting Extend image to region where info is missing –TV inpainting model: given image u 0 and region D, inpainted image u minimizes –Information in D found by continuing in from boundary Γ –T. Chan and J. Shen (2002) 2013 SIAM Great Lakes Section 16

17. TV Impainting 2013 SIAM Great Lakes Section 17 Chan, Shen (2005)

18. Texture Texture is regular, oscillatory features in image –Y. Meyer (2001), texture should average to 0, so that it belongs in the dual of BV –Image model: –With u = regular component, including contours v = oscillatory texture component w = oscillatory, unattractive noise component –Variational principle: For image u 0, chose u, g to minimize 2013 SIAM Great Lakes Section 18

19. Example of Texture 2013 SIAM Great Lakes Section 19 Bertalmio, Vese, Sapiro, Osher (2003)

20. Inpainting of Texture 2013 SIAM Great Lakes Section 20 Bertalmio, Vese, Sapiro, Osher (2003)

21. New Methods in Information Science 2013 SIAM Great Lakes Section 21 Wavelets Sparsity and Compressed Sensing

22. Wavelets –An orthonormal basis –Indexed by position and scale (~ wavenumber) –Based on translation and scaling of a single function –Easy forward and inverse transforms –Localized in both x and k Invention –Wavelet transform developed: Morlet 1981 –Nontrivial wavelet basis:Yves Meyer 1986 –Compact and smooth wavelets: Daubechies 1988 2013 SIAM Great Lakes Section 22

23. Sparsity Sparsity in datasets (e.g., sensor signals) –Signal which is “m-sparse”, with –i.e., x has at most m non-zero components –n measurements of x, corresponds to Objectives –How many measurements are required? What is the value of n? –How hard is it to compute x? Tractable or intractable? 2013 SIAM Great Lakes Section 23

24. Compressed Sensing Compressed sensing 2006 –David Donoho –Emmanuel Candes, Justin Romberg & Terry Tao 2013 SIAM Great Lakes Section 24

25. Compressed Sensing Problem statement –Find x that is m-sparse and solves Ax = f –Assuming that an m-sparse solution exists Standard methods min subject to constraint Ax = f –note Compressed sensing min subject to constraint Ax = f – note 2013 SIAM Great Lakes Section 25

26. How many measurements are required? For m << N, find m-sparse solution of Standard methods require: n = N –#(equations)=#(unknowns) Compressed sensing: n = m (log N) –n << N. Many fewer equations than unknowns! –Solution is exact with high probability! Reduced isometry property (RIP) –convex programming 2013 SIAM Great Lakes Section 26

27. How hard is it to compute x? Standard methods: NP hard = intractable Compressed sensing: tractable and fast convex programming 2013 SIAM Great Lakes Section 27

28. Why Does L 1 Promote Sparsity? Compressed sensing min subject to constraint Ax = f Two simplified problems: Find x in R 2 solving –1. min subject to constraint –2. min for given y in R 2 2013 SIAM Great Lakes Section 28

29. Version 1: Geometric solution Find x on line with smallest –For all but 45° lines, L 1 norm is smallest at a vertex. Vertices are sparse points, since a component is 0. Works in higher dimension 2013 SIAM Great Lakes Section 29

30. Version 2: Analytic Solution Given y, find x that minimizes –Minimum x has each component x i minimizing –Exercise: Show that the minimum is Operator S λ is “soft-thresholding” 2013 SIAM Great Lakes Section 30

31. Soft Thresholding 2013 SIAM Great Lakes Section 31 y x=S λ y

32. Applications of Information Science to PDEs 2013 SIAM Great Lakes Section 32 Wavelets for turbulence Sparsity for PDEs

33. Wavelets for Turbulence Turbulent solutions of the incompressible Navier-Stokes equations –Marie Farge and co-workers –Transformed velocity into wavelet basis –Deleted wavelet components with small coefficients to get “coherent part”. In 2D 256 2 computation, 0.7% of wavelet coefficients retain 99.2% of energy and 94% of enstrophy. Farge, et al. 1999 In 3D 256 3 computation, 3% of wavelet coefficients retain 99% of energy and 75% of enstrophy. Okamoto, et al. 2007. 2013 SIAM Great Lakes Section 33

34. Vorticity in 2D Turbulence 2013 SIAM Great Lakes Section 34 Total field Coherent partIncoherent part Farge, Schneider & Kevlahan 1999

35. Vorticity in 3D Turbulence 2013 SIAM Great Lakes Section 35 Okamoto, Yoshimatsu, Schneider, Farge, Kaneda 2007

36. Sparsity for PDEs PDE –Schaeffer, Osher, Caflisch & Hauck, 2013 –Apply soft-thresholding S λ to Fourier coefficients λ = c Δt 2 Alternatives to Fourier (e.g., framelets) now being used –Promotes sparsity How should soft-thresholding be used? Discretization in time, u n = u(t n = n Δt) 2013 SIAM Great Lakes Section 36 F= Fourier transform

37. Examples for Sparse PDEs Solver Schaeffer, Osher, Caflisch & Hauck, 2013 Examples –Convection eqtn with rapidly varying coefficients –Parabolic eqtn with rapidly varying coefficients –Viscous Burgers eqtn with rapidly varying convection term –2D Navier-Stokes vorticity equation, with rapidly oscillatory forcing f is rapidly oscillating in x, constant in t 2013 SIAM Great Lakes Section 37

38. Sparse Solution of 2D Navier Stokes for Interacting Vortices with Oscillatory Forcing 2013 SIAM Great Lakes Section 38 Schaeffer, Osher, Caflisch & Hauck, 2013

39. 2013 SIAM Great Lakes Section Possible Future Directions Texture in solutions of PDEs –Proposed model for “incoherent component of vorticity” by Farge Combination of network-based models, data- driven models and continuum models. Empirical mode decomposition (EMD) –Norden Huang, Tom Hou, Nathan Kutz Machine learning for many applications –Klaus Muller for materials Many possibilities! 39