1. Collective Sensing: a Fixed-Point Approach in the Metric Space 1 Xin Li LDCSEE, WVU 1 This work is partially supported by NSF ECCS-0968730

2. Unreasonable Effectiveness of Mathematics in Engineering “Unreasonable effectiveness of mathematics in natural sciences” Wigner’1960 “Unreasonable effectiveness of mathematics in natural sciences” Wigner’1960 To understand how nature works, you need to grasp the tool of mathematics first To understand how nature works, you need to grasp the tool of mathematics first The tension between mathematicians and engineers The tension between mathematicians and engineers Wavelets vs. filter banks Wavelets vs. filter banks “the hype that would arise around wavelets caused surprise and some understandable resentment in the subband filtering community” in Where do wavelets come from? I. Daubechies’1996 “the hype that would arise around wavelets caused surprise and some understandable resentment in the subband filtering community” in Where do wavelets come from? I. Daubechies’1996 Compressed sensing is another example of how mathematicians have stolen the show from engineers Compressed sensing is another example of how mathematicians have stolen the show from engineers

3. Mathematical Structures are Double-Bladed Swords Hilbert-space: a complete Inner-product space Quantum mechanics Fourier/wavelet analysis Learning theory PDE(e.g., Total-Variation) Mathematical formalism (Hilbert, Ackermann, Von Neumann …) Metric space: a set with a notion of distance General relativity Fixed-point theorems Game theory Dynamic systems Mathematical constructivism (Poincare, Brouwer, Weyl …)

4. Criticism of Compressed Sensing Where does sparsity come from? Where does sparsity come from? Nonlinear processing of wavelet coefficients Nonlinear processing of wavelet coefficients Nonlinear diffusion minimizing TV Nonlinear diffusion minimizing TV What is wrong? What is wrong? Over-emphasize the role of locality (it does not hold in complex systems) Over-emphasize the role of locality (it does not hold in complex systems) Inner-product is an artificial structure (it carries little insight about how patterns form in nature) Inner-product is an artificial structure (it carries little insight about how patterns form in nature) basis functions approximation of l 0 signal of interest

5. A Physical View of Sparsity How nature works? (e.g., variational principle) How nature works? (e.g., variational principle) Reaction-diffusion systems A. Turing’1952 Reaction-diffusion systems A. Turing’1952 “More is Different.” P.W. Anderson’1972 “More is Different.” P.W. Anderson’1972 Self-organizing systems I. Prigogine’1977 Self-organizing systems I. Prigogine’1977 Fractals and Chaos Mandelbrot’1977 Fractals and Chaos Mandelbrot’1977 Complex networks 1990s- Complex networks 1990s- Implications into image processing Implications into image processing Hilbert space might not be a proper mathematical framework for characterizing the complexity of natural images? Hilbert space might not be a proper mathematical framework for characterizing the complexity of natural images?

6. From Hilbert-space to Metric-space Images are viewed as the fixed-points in the metric space f=Pf Images are viewed as the fixed-points in the metric space f=Pf Nonlinear mapping P characterizes the organizational principle underlying images Nonlinear mapping P characterizes the organizational principle underlying images Example (nonlocal filter): Example (nonlocal filter): Non-expansiveness of P NLF guarantees the existence of fixed-points Bilateral, nonlocal mean and BM3D filters are special cases of P NLF

7. “ Phase Space ” of Image Signals SA-DCTTV BM3DNonlocal-TV Local filters Nonlocal filters

8. Nonlocal Regularization Magic BM3D Nonlocal-TV Key Observation: As the temperature (regularization) parameter varies, nonlocal models can traverse different phases corresponding to coarse/fine structures

9. From Compressed Sensing to Collective Sensing Key messages: 1.From local to nonlocal regularization thanks to the fixed-point formulation in the metric space (P NLF depends on the clustering result or similarity matrix) 2.From convex to nonconvex optimization: deterministic annealing (also-called graduated nonconvexity) is the ``black magic” behind

10. Variational Interpretations TV: Nonlocal TV: BM3D:

11. Application (I): Collective Sensing l 1 -magic PSNR=68.53dB Ours PSNR=84.47dB l 1 -magic PSNR=19.53dB Ours PSNR=40.97dB

12. Application (II): Lossy Compression House (256×256) Barbara (512×512) JPEG-decoded NL-enhanced SPIHT-decoded MATLAB codes accompanying this work are available at my homepage: http://www.csee.wvu.edu/~xinl/http://www.csee.wvu.edu/~xinl/ or Google “Xin Li WVU”

13. Image Comparison Results JPEG-decoded at rate of 0.32bpp (PSNR=32.07dB) NL-enhanced at rate of 0.32bpp (PSNR=33.22dB) SPIHT-decoded at rate of 0.20bpp (PSNR=26.18dB) NL-enhanced at rate of 0.20bpp (PSNR=27.33dB) Maximum-Likelihood (ML) Decoding Maximum a Posterior (MAP) Decoding

14. Application (III): Image Deblurring ISNR(dB) comparison among competing deblurring schemes for cameraman image: uniform 9×9 blurring kernel and noise level of BSNR=40dB

15. Image Comparison Results originaldegradedTVMM Ours ISTSADCT

16. Unexpected Connections Spectral clustering Spectral clustering Eigenvectors of graph Laplacian determine a provably optimal embedding Eigenvectors of graph Laplacian determine a provably optimal embedding Nonlinear dynamical systems Nonlinear dynamical systems Regularization implemented by the joint force of excitation and inhibition in a neuron network Regularization implemented by the joint force of excitation and inhibition in a neuron network Statistical physics Statistical physics Variational principle underlying Ising model, spin glass and Hopfield network Variational principle underlying Ising model, spin glass and Hopfield network

17. Summary and Conclusions One way of competing with mathematicians is to think like physicists One way of competing with mathematicians is to think like physicists Basis construction/pursuit is only one (local and suboptimal) way of understanding sparsity Basis construction/pursuit is only one (local and suboptimal) way of understanding sparsity Nonlocal regularization can more effectively handle complexity of natural images Nonlocal regularization can more effectively handle complexity of natural images The distinction between signals and systems is artificial and a holistic (collective) view is preferred The distinction between signals and systems is artificial and a holistic (collective) view is preferred

18. Ongoing Works Duality between similarity and dissimilarity Duality between similarity and dissimilarity The implication of sensory inhibition into image processing The implication of sensory inhibition into image processing From graphical models to complex networks From graphical models to complex networks The role of complex network topology The role of complex network topology Unification of signal reconstruction and object recognition Unification of signal reconstruction and object recognition To remove the artificial boundary between low-level and high-level vision To remove the artificial boundary between low-level and high-level vision