1. Algorithms and Convergence
Sec:1.3
Algorithms and Convergence

2. Sec:1.3 Algorithms and Convergence
An algorithm is a procedure that describes, in an unambiguous manner, a finite sequence of steps to be performed in a specified order. The object of the algorithm is to implement a procedure to solve a problem or approximate a solution to the problem.
We use a pseudocode to describe the algorithms. This pseudocode specifies the form of the input to be supplied and the form of the desired output.

3. Sec:1.3 Algorithms and Convergence
Looping techniques
counter-controlled
x=1:5;
vsum = 0;
for i=1:5
vsum = vsum + x(i);
end
vsum
conditional execution
x=1:5;
vsum = 0;
for i=1:5
vsum = vsum + x(i);
if vsum > 5;
break;
end
vsum
condition-controlled
x=1:5;
vsum = 0; i=1;
while i < 3
vsum = vsum + x(i);
i = i + 1;
end
vsum
Indentation

4. Sec:1.3 Algorithms and Convergence
𝑷 𝑵 𝒙 = 𝒙−𝟏 𝟏 − 𝒙−𝟏 𝟐 𝟐 𝒙−𝟏 𝟑 𝟑 −…+ −𝟏 𝑵+𝟏 𝒙−𝟏 𝑵 𝟑
Calculate: 𝑷 𝟗 𝟏.𝟓
clear; clc
n = 9;
x=1.5;
s=+1; pw = x-1; pn = s*pw;
for i = 2:n
s = -s; pw=pw*(x-1);
term = s*pw/i;
pn = pn + term;
end pn
clear; clc
n = 9;
x=1.5;
pn = 0;
for i = 1:n
term = (-1)^(i+1)*(x-1)^i/i;
pn = pn + term;
end pn

5. Sec:1.3 Algorithms and Convergence
Construct an algorithm to determine the minimal value of N required for
𝑷 𝑵 𝒙 = 𝒙−𝟏 𝟏 − 𝒙−𝟏 𝟐 𝟐 𝒙−𝟏 𝟑 𝟑 −…+ −𝟏 𝑵+𝟏 𝒙−𝟏 𝑵 𝟑
| 𝒍𝒏⁡𝟏.𝟓 − 𝑷 𝑵 (𝟏.𝟓)| < 𝟏𝟎 −𝟓 ,
𝑺 𝑵 = 𝟎.𝟓 𝟏 − 𝟎.𝟓 𝟐 𝟐 𝟎.𝟓 𝟑 𝟑 −…+ −𝟏 𝑵+𝟏 𝟎.𝟓 𝑵 𝟑
clear; clc
n = 13;
x=1.5;
pn = 0;
for i = 1:n
term = (-1)^(i+1)*(x-1)^i/i;
pn = pn + term;
end pn
From calculus we know that
|𝑆 − 𝑆 𝑁 | ≤ | 𝑡𝑒𝑟𝑚 𝑁+1 |.
if abs(term) < 1e-5; N=i; break; end

6. Sec:1.3 Algorithms and Convergence
Algorithm is stable
small changes in the initial data produce correspondingly small changes in the final results. otherwise it is unstable.
Some algorithms are stable only for certain choices of initial data, and are called conditionally stable.
Example:
How small is
𝝅 𝑥 𝑥−22=0
𝑥 1
𝑥 2
𝑥 1 − 𝑥 1
(𝟑.𝟏𝟒𝟏𝟓) 𝑥 𝑥−22=0
𝑥 1
𝑥 2
𝑥 2 − 𝑥 2
|𝝅−𝟑.𝟏𝟒𝟏𝟓|
𝑥 1 − 𝑥 1
𝑥 2 − 𝑥 2
small changes in the initial data
produce small changes

7. Sec:1.3 Algorithms and Convergence
Example
Rates of Convergence
Consider the following two series
sequence:
{αn}  α 𝒏
𝜶 𝒏
𝜸 𝒏
{ 𝜶 𝒏 −𝜶}  0 1 2 3 4 5 6 7
then we say that {αn} converges to α with rate (order) of convergence O( ( 𝟏 𝒏 ) 𝒑 ).
“big oh of”
If a positive constant K exists with
|αn − α| ≤ K 1 𝑛 𝒑
for large n,
𝜶 𝒏 = 𝒏+𝟏 𝒏 𝟐
𝜸 𝒏 = 𝒏+𝟑 𝒏 𝟑
Then we write:
Which one is faster?
αn = α + O( ( 1 𝑛 ) 𝑝 ).
Rate of convergence
Remark: Comparsion test and Limit comparison test

8. Sec:1.3 Algorithms and Convergence
Example
Rates of Convergence
Consider the following two series
sequence:
{αn}  α
𝜶 𝒏 = 𝒏+𝟏 𝒏 𝟐
𝜸 𝒏 = 𝒏+𝟑 𝒏 𝟑
{ 𝜶 𝒏 −𝜶}  0
then we say that {αn} converges to α with rate (order) of convergence O( ( 𝟏 𝒏 ) 𝒑 ).
𝜶 𝒏 = 𝒏+𝟏 𝒏 𝟐 ≤𝟐 ( 𝟏 𝒏 ) 𝟏
“big oh of”
𝒑=𝟏
If a positive constant K exists with
𝜸 𝒏 = 𝒏+𝟑 𝒏 𝟑 ≤𝟒 ( 𝟏 𝒏 ) 𝟐
𝒑=𝟐
|αn − α| ≤ K 1 𝑛 𝒑
for large n,
Then we write:
αn = α + O( ( 1 𝑛 ) 𝑝 ).
Remark: Comparsion test and Limit comparison test

9. Sec:1.3 Algorithms and Convergence
Rates of Convergence
Suppose {βn} is a sequence known to converge to zero, and {αn} converges to a number α. If a positive constant K exists with
|αn − α| ≤ K|βn|, for large n,
then we say that {αn} converges to α with rate (order) of convergence O(βn).
(This expression is read “big oh of βn”.)
Rates of Convergence
|αn − α| ≤ K 1 𝑛 𝑝
for large n,
Two sequences:
{αn}  α
{ 𝛽 𝑛 = ( 1 𝑛 ) 𝑝 }  0
We are generally interested in the largest value of p with
αn = α + O( ( 1 𝑛 ) 𝑝 ).

10. Root-finding problem 𝑓 𝑥 = 0
The root-finding problem is a process involves finding a root, or solution, of an equation of the form
𝑓 𝑥 = 0
for a given function 𝑓 . A root of this equation is also called a zero of the function 𝑓 .
In graph, the root (or zero) of a function is the x-intercept
Three numerical methods for
root-finding
Sec(2.1): The Bisection Method
Sec(2.2): Fixed point Iteration
root
Sec(2.3): The Newton-Raphson
Method

11. 𝒏 𝒙 𝒏 Newton’s Method 𝑓 𝑥 =0
THE NEWTON-RAPHSON METHOD is a method for finding successively better approximations to the roots (or zeroes) of a function.
Example
Algorithm
Use the Newton-Raphson method to estimate the root of f (x) = 𝒆 −𝒙 −𝒙, employing an initial guess of x1 = 0.
To approximate the roots of
𝑓 𝑥 =0
Given initial guess 𝑥 1
f (x) = 𝒆 −𝒙 −𝒙
𝒇 ′ 𝒙 =−𝒆 −𝒙 −𝟏
f (0) =𝟏
𝒇 ′ 𝟎 = −𝟐
𝑥 𝑛+1 = 𝑥 𝑛 − 𝑓( 𝑥 𝑛 ) 𝑓′( 𝑥 𝑛 )
𝑥 1 =0
𝑥 2 = 𝑥 1 − 𝑓( 𝑥 1 ) 𝑓′( 𝑥 1 )
𝑥 2 =0− 𝑓(0) 𝑓′(0) 𝒏
𝒙 𝒏
=0.5
𝑥 3 =0.5− 𝑓(0.5 ) 𝑓′(0.5 ) =
The true value of the root: Thus, the approach rapidly converges on the true root.

12. Newton’s Method
THE NEWTON-RAPHSON METHOD is a method for finding successively better approximations to the roots (or zeroes) of a function.
Example
clear
f exp(-t) - t;
df - exp(-t) - 1;
x(1) = 0;
for i=1:4
x(i+1) = x(i) - f( x(i) )/df( x(i) );
end x'
Use the Newton-Raphson method to estimate the root of f (x) = 𝒆 −𝒙 −𝒙, employing an initial guess of x1 = 0. 𝒏
𝒙 𝒏

13. 𝒏 𝒙 𝒏 (Newton) Newton’s Method Example Approximate a root of
f (x) = 𝒄𝒐𝒔𝒙 − 𝒙 using
Newton’s Method
employing an initial guess of 𝒙𝟏 = 𝝅 𝟒
clear
f exp(-t) - t;
df - exp(-t) - 1;
x(1) = 0;
for i=1:4
x(i+1) = x(i) - f( x(i) )/df( x(i) );
end x' 𝒏
𝒙 𝒏 (Newton) 1 2 3 4 5 6 7 8