LSM with Sparsity Constraints

Slides:



Advertisements
Similar presentations
CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng.
Advertisements

Instabilities of SVD Small eigenvalues -> m+ sensitive to small amounts of noise Small eigenvalues maybe indistinguishable from 0 Possible to remove small.
Optimization.
Multisource Full Waveform Inversion of Marine Streamer Data with Frequency Selection Multisource Full Waveform Inversion of Marine Streamer Data with Frequency.
Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.
Computational Challenges for Finding Big Oil by Seismic Inversion.
Multi-source Least Squares Migration and Waveform Inversion
Jonathan Richard Shewchuk Reading Group Presention By David Cline
Lecture 9 Inexact Theories. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture 03Probability and.
Performance Optimization
Numerical Optimization
Newton’s Method applied to a scalar function Newton’s method for minimizing f(x): Twice differentiable function f(x), initial solution x 0. Generate a.
Methods For Nonlinear Least-Square Problems
Unconstrained Optimization Problem
Advanced Topics in Optimization
Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.
CSE 245: Computer Aided Circuit Simulation and Verification
Why Function Optimization ?
An Introduction to Optimization Theory. Outline Introduction Unconstrained optimization problem Constrained optimization problem.
Regularized meshless method for solving the Cauchy problem Speaker: Kuo-Lun Wu Coworker : Kue-Hong Chen 、 Jeng-Tzong Chen and Jeng-Hong Kao 以正規化無網格法求解柯西問題.
Desingularized meshless method for solving the Cauchy problem Speaker: Kuo-Lun Wu Coworker : Kue-Hong Chen 、 Jeng-Tzong Chen and Jeng-Hong Kao 以去奇異無網格法求解柯西問題.
9 1 Performance Optimization. 9 2 Basic Optimization Algorithm p k - Search Direction  k - Learning Rate or.
UNCONSTRAINED MULTIVARIABLE
MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 9. Optimization problems.
ENCI 303 Lecture PS-19 Optimization 2
Making the Most from the Least (Squares Migration) G. Dutta, Y. Huang, W. Dai, X. Wang, and Gerard Schuster G. Dutta, Y. Huang, W. Dai, X. Wang, and Gerard.
Application of Differential Applied Optimization Problems.
Nonlinear least squares Given m data points (t i, y i ) i=1,2,…m, we wish to find a vector x of n parameters that gives a best fit in the least squares.
LTSI (1) Faculty of Mech. & Elec. Engineering, University AL-Baath, Syria Ahmad Karfoul (1), Julie Coloigner (2,3), Laurent Albera (2,3), Pierre Comon.
CS Statistical Machine learning Lecture 10 Yuan (Alan) Qi Purdue CS Sept
Computer Animation Rick Parent Computer Animation Algorithms and Techniques Optimization & Constraints Add mention of global techiques Add mention of calculus.
Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION ASEN 5070 LECTURE 11 9/16,18/09.
CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods I Chung-Kuan Cheng.
Quasi-Newton Methods of Optimization Lecture 2. General Algorithm n A Baseline Scenario Algorithm U (Model algorithm for n- dimensional unconstrained.
CHAPTER 10 Widrow-Hoff Learning Ming-Feng Yeh.
+ Quadratic Programming and Duality Sivaraman Balakrishnan.
Chapter 2-OPTIMIZATION G.Anuradha. Contents Derivative-based Optimization –Descent Methods –The Method of Steepest Descent –Classical Newton’s Method.
INTRO TO OPTIMIZATION MATH-415 Numerical Analysis 1.
Fast Least Squares Migration with a Deblurring Filter Naoshi Aoki Feb. 5,
Migration Velocity Analysis of Multi-source Data Xin Wang January 7,
Numerical Analysis – Data Fitting Hanyang University Jong-Il Park.
Regularized Least-Squares and Convex Optimization.
Numerical Methods for Inverse Kinematics Kris Hauser ECE 383 / ME 442.
Fast 3D Least-squares Migration with a Deblurring Filter Wei Dai.
Optimal Control.
Bounded Nonlinear Optimization to Fit a Model of Acoustic Foams
LSM Theory: Overdetermined vs Underdetermined
Pipelined Adaptive Filters
Widrow-Hoff Learning (LMS Algorithm).
Probabilistic Models for Linear Regression
CSE 245: Computer Aided Circuit Simulation and Verification
Iterative Non-Linear Optimization Methods
Steepest Descent Optimization
Non-linear Least-Squares
CSE 245: Computer Aided Circuit Simulation and Verification
Chapter 10. Numerical Solutions of Nonlinear Systems of Equations
Migration Intuitive Least Squares Migration Green’s Theorem.
Overview of Multisource and Multiscale Seismic Inversion
Structure from Motion with Non-linear Least Squares
Overview of Multisource and Multiscale Seismic Inversion
~ Least Squares example
Stationary Point Notes
~ Least Squares example
X y y = x2 - 3x Solutions of y = x2 - 3x y x –1 5 –2 –3 6 y = x2-3x.
A Numerical Analysis Approach to Convex Optimization
Performance Optimization
Steepest Descent Optimization
Structure from Motion with Non-linear Least Squares
CS5321 Numerical Optimization
Presentation transcript:

LSM with Sparsity Constraints Additive noisy data, minimize influence of wild Outlier noise by choosing p=1 1 < p Minimize S S |Lijmj – di| + l P(m) p i j or Minimize S |mj | subject to Lm=d p Undetermined problems, where sparsest soln Is desired so choose something close to p=0. Choose A domain where model signal and noise separate

Motivation Problem: Noise (inconsistent physics) in model Correct velocity e=||Lm-d||2 e=||Lm-d||2 + l||m||2 e=||Lm-d||2 + l||𝛻2m||2 Solution: Sparsity Constraint

Motivation Problem: Noise (inconsistent physics) in model Correct velocity e=||Lm-d||2 e=||Lm-d||2 + l||m||2 e=||Lm-d||2 + l||𝛻2m||2 Solution: Sparsity Constraint e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2

Motivation Problem: Noise (inconsistent physics) in model Incorrect velocity (5.5 km/s instead of 5.0 km/s) e=||Lm-d||2 e=||Lm-d||2 + l||m||2 e=||Lm-d||2 + l||𝛻2m||2 Solution: Sparsity Constraint e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2

Motivation Problem: Noise (inconsistent physics) in model e=||Lm-d||2 + l||m||2 Solution: Sparsity Constraint e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2

Motivation Problem: Noise (inconsistent physics) in model e=||Lm-d||2 + l||m||2 Solution: Sparsity Constraint e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2

Entropy Regularization e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2 Entropy: Property: Entropy minimum when Si clumped Spike Example (s_1=1): Uniform Example:

Entropy Regularization dS/dsi= -[s’i + 1]ds’i/dsi e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2

Entropy Regularization e = ||Lm-d||2 + l||𝑒𝑛𝑡𝑟𝑜𝑝𝑦(𝑚)||2 Use previous migration so ds’_i/ds_j=0 Step 1: Step 2: dS/dsi Will this lead to a symmetric SPD Hessian?

Outline LSM with Cauchy Constraint: Part Admundsen Reweighted Least Squares LSM with Entropy Regularization

Newton -> Steepest Descent Method Given: (1) Find: stationary point x* s.t. F(x*)=0 D Soln: Newton’s Method

Gradients of L2 vs Cauchy Norms Frechet derivative: dPi/dsj

Gradients of L2 vs Cauchy Norms Key Benefit: Large Residual DP are Downweighted. l is like standard dev.

Adjust l to insure SPD diagonal

Adjust l to insure SPD diagonal

Numerical Tests

Simulated Data

Outline LSM with Cauchy Constraint: Saachi Reweighted Least Squares LSM with Entropy Regularization

LSM with Sparsity Constraint Given: Solve: m subject to Iterative Rewighted Least Squares: Note: Large values of Dm downweighted

Multichannel Decon

Standard LSM vs LSM with Sparsity Migration CIG Precon. LSM CIG Sparsity :LSM CIG

Standard LSM vs LSM with Sparsity Migration Precon. LSM Sparsity :LSM

References

Outline LSM with Cauchy Constraint Reweighted Least Squares LSM with Entropy Regularization

Solve: m subject to (your Choice) LSM with Sparsity Constraint Given: Solve: m subject to (your Choice) Iterative Rewighted Least Squares: Rii = (|ri|) p-2 For small r replace with cutoff or waterr level

Solve: m subject to (your Choice) LSM with Sparsity Constraint Given: Solve: m subject to (your Choice) Iterative Rewighted Least Squares: Rii = (|ri|) p-2 For small r replace with cutoff or water level

VSP Example Model Src-Rec & Rays

Outline LSM with Cauchy Constraint Reweighted Least Squares LSM with Entropy Regularization

Solve: m subject to (your Choice) LSM with Maximum Entropy Given: Solve: m subject to (your Choice) Entropy Reg.

Solve: m subject to (your Choice) LSM with Maximum Entropy Given: Solve: m subject to (your Choice) Entropy Reg.

LSM with Maximum Entropy Entropy Reg.

Outline

Minimum Entropy Inversion: Given: Solve: Minimum Entropy Inversion: Normal eqs.