1. Numerical methods Themes Solution of equations
Numerical Differentiation
Numerical Integration
Calculation of curve length
Optimization

2. Theme 1. Solution of Equations
Numerical methods
Theme 1.
Solution of Equations

3. ? Basic concept Task: solve the equation Type of solution:
analytical (exact, formulary)
approximate (inexact)
How? ?
Numerical methods
Graphic approach
initial approximation
at N  

4. Numerical methods
Idea: successive refinement of solution by the instrumentality some algorithm
Field of use: when it’s impossible or extremely difficult to find the exact solution.
Some solution can be find
In many cases the error can be estimated (i.e. solution with given accuracy can be find)
The exact solution can’t be find
It’s impossible to investigate the solution at the parametric variation
Computationally intensive (high volume of calculation)
Sometimes it’s difficult to estimate the error
No universal methods

5. ! Whether the solution exists on [a, b]? x y x* a b x y x* a b x y x*
Solution doesn’t exist
Solution doesn’t exist x y x* a b x y x* a b x y x* a b
If continuous function f (x) has different signs on the end of the interval [a, b], then in some point inside [a, b] we have f (x) = 0! !

6. Dichotomy method (bisection)x y x* a b с
To find the bisecting point of a segment [a,b]: c = (a + b) / 2;
If f(c)*f(a)<0, to move the right boundary of the interval b = c;
If f(c)*f(a)≥ 0, to move the left boundary of the interval a = c;
To repeat steps 1-3, until b – a ≤ .

7. Dichotomy method (bisection)
Simplicity
Solution with given accuracy can be obtained (in the range of computer calculation accuracy)
must to know the interval [a, b]
on the interval [a, b] must be only one solution
large number of steps to reach the high accuracy
only for a function with one variable

8. Segment bisection method//
// BinSolve finds solution on [a,b] by the
// segment bisection method
// Input: a, b – interval boundaries, a < b
// eps - solution accuracy
// Output: x – solution of equation f(x)=0
float BinSolve ( float a, float b, float eps ) {
float c;
while ( b - a > eps )
c = (a + b) / 2;
if ( f(a)*f(c) < 0 )
b = c;
else a = c; }
return (a + b) / 2;
float f ( float x ) {
return x*x – 5; }

9. Value of variable changes inside of function
How to calculate a number of steps?
float BinSolve ( float a, float b,
float eps, int &n ) {
float c;
n = 0;
while ( b - a > eps )
c = (a + b) / 2;
if ( f(a)*f(c) < 0 )
b = c;
else a = c;
n ++; }
return (a + b) / 2;
int &n
Value of variable changes inside of function
n = 0;
Call in main program:
float x;
int N;
...
x = BinSolve ( 2, 3, , N );
printf(“Answer: x = %7.3f", x);
printf(“Number of steps: %d", N);
n ++;

10. Method of iterations (repetition)
Tasks:
equivalent transformations :
has a solution at
Idea of solution:
– initial approximation (for example)
Questions:
how to choose ?
Whether a solution always can be find?

11. Iterative convergence
Convergent iterative process:
Sequence approaches (converge) exact solution.
One-sided convergence
Double-sided convergence

12. Divergence of iteration
Divergent iterative process:
Sequence indefinitely increase or decrease, doesn’t approach to solution.
One-sided divergence
Double-sided divergence

13. On what depends convergence?
diverge
Conclusions:
convergence of iteration depends on derivative
Iteration converge at and diverge at
convergence is defined by choosing parameter b

14. ? How to choose b? by guesswork, trying different versions
for initial approximation x0
Recalculate on each step, forexample:
What kind of problems might be? ?

15. Iteration method (program code)//
// Iter solution of equation by the iteration method
// Input: x – initial approximation
// b - parameter
// eps - accuracy of solution
// Output: solution of equation f(x)=0
// n – number of steps
////
float Iter ( float x, float b, float eps, int &n) {
int n = 0;
float dx;
while ( 1 ) {
dx = b*f(x);
x = x + dx;
if ( fabs(dx) < eps ) break;
n ++;
if ( n > 100 ) break; }
return x;
rated output
abend exit

16. ? Newton's method (tangent method)
What’s the connection with iteration method? ?

17. Newton's method (program code)//
// Newton solution of equation by the Newton’s method
// Input: x – initial approximation
// eps - exact solution
// Output: solution of equation f(x)=0
// n – number of steps
////
float Newton ( float x, float eps, int &n) {
int n = 0;
float dx;
while ( 1 ) {
dx = f(x) / df(x);
x = x - dx;
if ( fabs(dx) < eps ) break;
n ++;
if ( n > 100 ) break; }
return x;
float f ( float x ) {
return 3*x*x*x+2*x+5; }
float df ( float x ) {
return 9*x*x + 2;

18. Newton's method
rapid (quadratic) convergence – error on k-th step inversely proportional to k2
No need to know interval, only initial approximation
It’s applicable for a function with many variables
Have to be able to calculate derivatives (by formula or numerically)
Derivative mustn’t be equal to zero
Might be cycled

19. Тheme 2. Numerical Differentiation
Numerical methods
Тheme 2.
Numerical Differentiation

20. 1. The Taylor series and approximation
Taylor’s theorem states that any smooth function can be approximated as a polynomial
f (xi+1)
f (xi) xi
xi+1
f (x)
f (xi-1) x
xi-1

21. 1. The Taylor series and approximation
Taylor’s theorem states that any smooth function can be approximated as a polynomial
f (xi+1) xi
xi+1
f (x)
f (xi) x

22. The first-order finite-difference of first derivative So for a finite
The finite-difference formulas
From the Taylor’s series a various finite-difference formulas can be obtained
The first-order finite-difference of first derivative So for a finite

23. Тheme 2. Numerical Integration
Numerical methods
Тheme 2.
Numerical Integration

24. Area of curvilinear trapezoidx y b a
y = f (x) x y b a
y = f1 (x)
y = f2 (x)

25. ? ? Method of (left) rectangles y = f2 (x) f1 (x) y = f1 (x) y f2 (x)Si x
x+h
f1 (x)
f2 (x)
y = f1 (x) x y S2 S3 S4
xс2
xс1 S1 h
float Area() {
float x, S = 0, h=0.001;
for ( x = xc1; x < xc2; x += h)
S += h*(f1(x) – f2(x));
return S; }
Why not x <= xc2? ?
for ( x = xc1; x < xc2; x += h )
S += f1(x) – f2(x);
S *= h;
How to improve the solution? ?

26. Method of (right) rectangles
y = f2 (x) Si x
x+h
f1 (x)
f2 (x)
y = f1 (x) y S1 S2 S3 S4 h
xс1
xс2 x
float Area() {
float x, S = 0, h=0.001;
for ( x = xc1; x < xc2; x += h)
S += h*(f1(x+h) – f2(x+h));
return S; }
for ( x = xc1; x < xc2; x += h )
S += f1(x+h) – f2(x+h);
S *= h;

27. ? Method of (average) rectangles y = f2 (x) y = f1 (x) f1 (x) y f2 (x)Si
x+h
y = f1 (x) y S2 S3 S1 S4 h
xс1
xс2 x
float Area() {
float x, S = 0, h=0.001;
for ( x = xc1; x < xc2; x += h)
S += h*(f1(x+h) – f2(x+h));
return S; }
which method is more
accurate? ?
for ( x = xc1; x < xc2; x += h )
S += f1(x+h/2) – f2(x+h/2);
S *= h;
left (right):
middle

28. ? Trapezium method y = f2 (x) f1 (x) y = f1 (x) y f2 (x) xс1 xс2 x h xSi
y = f1 (x) y S1 S2 S3 S4 h
xс1
xс2 x
for ( x = xc1; x < xc2; x += h )
S += f1(x) – f2(x) +
f1(x+h) – f2(x+h);
S *= h/2;
how to improve? ?
S =( f1(xc1) - f2(xc1)
+ f1(xc2) - f2(xc2) )/2.;
for ( x = xc1+h; x < xc2; x += h )
S += f1(x) – f2(x);
S *= h;
Error

29. Monte-Carlo method
Application: calculation of polygram areas (difficult to use other methods).
Requirements: it needs to fairly simple define, if the point (x, y) is fallen into the figure.
Example: Given 100 circles (coordinates of center , radius), which are may intersect. Find the square of area which is blocked by circles.
how to find S? ?

30. Метод Монте-Карло
Polygram is inscribed into other figure, which square is easily calculated (rectangle, circle, …).
uniformly N points with random coordinate inside of rectangle.
Counting up quantity of points, which are fallen onto the figure: M.
4. Square is calculated:
On figure M points
Totally N points !
Method is approximate .
Distribution must be uniform.
The more points, the more accuracy.
Accuracy is limited by detector of random number.

31. Тheme 3. Calculation of curve length
Numerical methods
Тheme 3.
Calculation of curve length

32. Curve length x y b a y = f (x) L f (xi+h) f (xi) Li Exact solution:LN
Needs formula for derivative
Difficult to take integral
Approximate solution:
f (xi+h) xi
xi+h
f (xi) Li

33. Curve length //----------------------------------------------
// CurveLen calculation of curve length
// Input: a, b – boundary of integral
// Output: length of curve y = f(x) on interval[a,b]
float CurveLen ( float a, float b ) {
float x, dy, h = , h2 = h*h, L = 0;
for ( x = a; x < b; x += h ) {
dy = f(x+h) - f(x);
L += sqrt(h2 + dy*dy); }
return L;

34. Numerical methods
Тheme 4.
Optimization

35. basic concepts Optimization – search of optimal solution.
Aim: to define the value of unknown parameters, at which given function reaches to minimum (expenses) or maximum (profit).
Constraints – condition, which makes problem sensible. or
Find x, at which or at given constrains.

36. Local and global minimums
Tasks: to find global minimum
Reality:
Many of known algorithms find only local minimum near the initial point
Algorithm of search a global minimum in general case unknown
y = f (x)
Local minimum
Global minimum
What to do?:
Initial point for a function with one variable is defined by graphics
Random search of initial point
launch of algorithm of search from many different point and choice of the best result

37. ? Minimum of function of single variable y = f (x)
Given: on interval [a,b] function is continuous and has unique minimum.
Find: x*
Principle of interval contraction :
how to choose c and d to the best advantage ? ?

38. Minimum of function of single variable
contraction constant in both cases:
y = f (x)
Compression coefficient :
Very fast compression:
must be c  d at
Method of “almost half” division:
– small number
It needs to search two values of function on each step

39. Ratio of “golden section”
Idea: choose c and d that, in order to on each step calculate only one new value of function.
Equation for definition g:
Ratio of «golden section»:

40. ? Ratio of “golden section”//
// Gold search of function minimum(«golden section»)
// Input: a, b – interval boundary
// eps – accuracy
// Output: x, at wich f(x) has minimum // on the interval[a,b]
float Gold (float a, float b, float eps ) {
float x1, x2, g = , R = g*(b - a);
while ( fabs(b-a) > eps ) {
x1 = b - R; x2 = a + R;
if ( f(x1) > f(x2) ) a = x1;
else b = x2;
R *= g; }
return (a + b) /2.;
how to calculate only one value on each step? ?

41. Function of several variables
Find , for which at given constraints.
Problems:
No universal algorithms to search the global minimum
It’s not clear, how to choose an initial approximation (depends on problem and intuition)
Approaches:
Methods of local optimization (the result depends on choice of initial approximation)
random search (without warrants)
Method of global optimization (for special classes of function)

42. Initial approximation
Alternating-variable descent method
Idea:
Initial point is chooses
Only x1 will be change and other variables are fixed, minimum is defined by x1
then only x2 will be change and other variables are fixed, …
minimum
Initial approximation
simplicity, reduced to several problems with one variable
It’s possible to move to the minimum faster
high volume of calculation
Solution for composite function may not be find

43. Initial approximation
Gradient method
Gradient – it’s a vector wish shows the direction of
quick increase function .
Idea:
Initial point is chooses
On each step we’ll move in direction
of anti gradient
gradient
minimum
rapid convergence
Initial approximation
It needs to calculate derivatives (by formula or numerically)

44. Initial approximation
Random search method
Idea:
initial point is choosed
we’ll try to take a step in random direction
if the value of function is decreased, step is successful (is memorized)
minimum
Initial approximation
simplicity of realization
no need to calculate the derivatives
applicable for function with many local minimums
Very high volume of calculation