Introduction to Scientific Computing II

Slides:



Advertisements
Similar presentations
Scientific Computing Lab Results Worksheet 3 Dr. Miriam Mehl Institut für Informatik Scientific Computing in Computer Science.
Advertisements

1 Numerical Solvers for BVPs By Dong Xu State Key Lab of CAD&CG, ZJU.
SOLVING THE DISCRETE POISSON EQUATION USING MULTIGRID ROY SROR ELIRAN COHEN.
Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system.
Modern iterative methods For basic iterative methods, converge linearly Modern iterative methods, converge faster –Krylov subspace method Steepest descent.
Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.
Jonathan Richard Shewchuk Reading Group Presention By David Cline
Lecture 2 Linear Variational Problems (Part II). Conjugate Gradient Algorithms for Linear Variational Problems in Hilbert Spaces 1.Introduction. Synopsis.
12 1 Variations on Backpropagation Variations Heuristic Modifications –Momentum –Variable Learning Rate Standard Numerical Optimization –Conjugate.
September 23, 2010Neural Networks Lecture 6: Perceptron Learning 1 Refresher: Perceptron Training Algorithm Algorithm Perceptron; Start with a randomly.
Advanced Topics in Optimization
CSE 245: Computer Aided Circuit Simulation and Verification
Why Function Optimization ?
Implementation of Nonlinear Conjugate Gradient Method for MLP Matt Peterson ECE 539 December 10, 2001.
9 1 Performance Optimization. 9 2 Basic Optimization Algorithm p k - Search Direction  k - Learning Rate or.
ECE 530 – Analysis Techniques for Large-Scale Electrical Systems Prof. Hao Zhu Dept. of Electrical and Computer Engineering University of Illinois at Urbana-Champaign.
Computational Optimization
CS B553: A LGORITHMS FOR O PTIMIZATION AND L EARNING Gradient descent.
Introduction to Adaptive Digital Filters Algorithms
Introduction to Scientific Computing II From Gaussian Elimination to Multigrid – A Recapitulation Dr. Miriam Mehl.
CS 478 – Tools for Machine Learning and Data Mining Backpropagation.
1 Unconstrained Optimization Objective: Find minimum of F(X) where X is a vector of design variables We may know lower and upper bounds for optimum No.
CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods I Chung-Kuan Cheng.
Data Modeling Patrice Koehl Department of Biological Sciences National University of Singapore
Introduction to Scientific Computing II Multigrid Dr. Miriam Mehl Institut für Informatik Scientific Computing In Computer Science.
Introduction to Scientific Computing II Multigrid Dr. Miriam Mehl.
Introduction to Scientific Computing II
Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.
Gradient Methods In Optimization
Variations on Backpropagation.
Survey of unconstrained optimization gradient based algorithms
Machine learning optimization Usman Roshan. Machine learning Two components: – Modeling – Optimization Modeling – Generative: we assume a probabilistic.
9 Nov B - Introduction to Scientific Computing1 Sparse Systems and Iterative Methods Paul Heckbert Computer Science Department Carnegie Mellon.
Numerical Analysis – Data Fitting Hanyang University Jong-Il Park.
ECE 530 – Analysis Techniques for Large-Scale Electrical Systems
Fast 3D Least-squares Migration with a Deblurring Filter Wei Dai.
Fall 2004 Backpropagation CS478 - Machine Learning.
Non-linear Minimization
Scientific Computing Lab
CSE 245: Computer Aided Circuit Simulation and Verification
Introduction to Scientific Computing II
Pressure Poisson Equation
Non-linear Least-Squares
Outline Single neuron case: Nonlinear error correcting learning
CSE 245: Computer Aided Circuit Simulation and Verification
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Conjugate Gradient Method
Structure from Motion with Non-linear Least Squares
CS5321 Numerical Optimization
Instructor :Dr. Aamer Iqbal Bhatti
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Iterative Algorithm g = Af g : Vector of projection data
Numerical Linear Algebra
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Programming assignment # 3 Numerical Methods for PDEs Spring 2007
Performance Optimization
Nonlinear Conjugate Gradient Method for Supervised Training of MLP
Structure from Motion with Non-linear Least Squares
First-Order Methods.
Presentation transcript:

Introduction to Scientific Computing II Conjugate Gradients Dr. Miriam Mehl

Steepest Descent – Basic Idea solution of SLE minimization iterative one-dimensional minima direction of steepest descent?

Steepest Descent – Algorithm

Steepest Descent – Algorithm II

Steepest Descent – Example initial error after 1 iteration after 10 iterations

Steepest Descent – Example 1/128 1/64 1/32 1/16 h 48,629 11,576 2,744 646 iterations

Steepest Descent – Convergence Poisson with 5-point-stencil like Jacobi

Steepest Descent – Convergence

Conjugate Gradients – Basic Idea solution of SLE minimization iterative one-dimensional minima no repeating search directions

Steepest Descent – Principle

Conjugate Gradients – Principle

CG – Algorithm

Steepest Descent – Example initial error after 1 iteration after 10 iterations

Conjugate Gradients – Example initial error after 1 iteration after 10 iterations

Conjugate Gradients – Example 322 157 76 35 iterations cg 1/128 1/64 1/32 1/16 h 48,629 11,576 2,744 646 iterations sd 16,129 3,969 961 225 #unknowns

CG – Convergence Poisson with 5-point-stencil like SOR no parameter adjustment

PCG – Idea convergence rate cg: Solve system M-1Ax=M-1b better condition number k M-1 easy to apply

PCG – Algorithm

PCG – Algorithm