1. Week 11 – Spectral TV and Convex analysis Guy Gilboa Course 049064

2. Topics: Some basic definitions in convex analysis. New research on Spectral TV. Guy Gilboa, Technion2

3. Classical Fourier filtering example Guy Gilboa, Technion3

4. Classical Signal Processing (Fourier) Positive features: Decomposition (transform) into a better representation, orthonormal basis. Filtering in the transform space – simple amplification or attenuation of coefficients. Spectrum plot – visualization of active frequencies, L2 energy is preserved – Perseval identity. Linearity – forward and inverse transforms are linear. A well established mathematical theory and fast computational methods. Known drawbacks : Does not handle well discontinuities and spatially local features, not an adequate basis for images.. Guy Gilboa, Technion4

5. Variational Spectral Processing Decomposition (transform) into a better representation, orthonormal basis. Filtering in the transform space – simple amplification or attenuation of coefficients functions. Spectrum plot – visualization of active “generalized- frequencies”, Perseval-type rule. Linearity – forward and inverse transform are is linear. A well established mathematical theory and fast computational methods. Not yet.. Guy Gilboa, Technion5

6. TV spectral representation [G., SIAM-IS, 2014] Let u(t) be the TV-flow solution at time t with u(0)=f. TV- flow f t S(t)f... ϕ t H(t) 1 0 S(t) Guy Gilboa, Technion6

7. Nonlinear eigenvalue problem Linear problem (L linear operator) General operator T: A convex functional J(u) induces an operator p(u) by its subgradient: Guy Gilboa, Technion7

8. Understanding a regularizer is knowing its eigenfunctions [Alter-Caselles-Chambolle-2003]. My view : What are the TV eigenfunctions? Guy Gilboa, Technion8

9. Why do we get a delta in time for eigenfunctions ? Guy Gilboa, Technion9

10. Ideal low-pass-filter (LPF) eigenvalue Guy Gilboa, Technion10

11. Standard possible filters (borrowing the names from classical signal processing) Guy Gilboa, Technion11

12. 1D Decomposition Example Guy Gilboa, Technion12

13. Application Guy Gilboa, Technion13

14. TV Band-Pass and Band-Stop filters fS(t) TV Band-stopTV Band-pass

15. Old man

16. Old man – close up, original

17. Old man – 2 modes attenuated

18. Wavelet vs. Spectral-TV decomposition f Haar WaveletsSpectral TV Guy Gilboa, Technion18

19. Haar Wavelets vs. Spectral TV WaveletSpectral TV Guy Gilboa, Technion19

20. Texture analysis and processing in the spectral TV domain with Dikla Horesh Guy Gilboa, Technion20

21. Spatially varying texture Perspective Lighting Combination Goal– decompose textures which are gradually varying in scale, contrast or lighting. Scale change Perspective Lighting

22. Spatially varying contrast and scale Guy Gilboa, Technion22 Input f(x) T(x)

23. What happens for a natural image? Guy Gilboa, Technion23

24. Proposed result How can we use it to separate? Classical TV-G separation at some cutoff scale Guy Gilboa, Technion24 Time of maximal value of phi(t;x) for each pixel x Input

25. Algorithm Input image Spectral decomposition Max response time Surface Fitting Take Max Φ in each pixel Separation Bands Separated image layers Values in percentiles 85-95 are taken for surface fitting

26. Separation surface Band width taken for separationSeparation bands Input image Example 1

27. Decomposition example (perspective) Maximal phi response Proposed Input Rolling-Guidance-Filter [*] [*] Zhang et al, ECCV-2014 Guy Gilboa, Technion27

28. Desired textureStructure Input Texture enhancement

29. Application: Texture enhancement enhanced Input Attenuated

30. Application: Texture enhancement (2) Desired textureStructure Input

31. Enhancement 2 enhanced Input Attenuated Guy Gilboa, Technion31

32. Michael Moeller’s texture transfer Guy Gilboa, Technion32