1. NumPy, SciPy, Mpi4Py Shepelenko Olha

2. History of NumPy Originally, Python was not developed as a language for numerical computing. However, due to its simplicity it attracted attention of scientists. - Numeric matrix package was designed in 1995. It is slow for large arrays (still operating, but outdated); -Numarray was designed as a replacement for Numeric package. It is fast for large arrays, but slow for small arrays. (also outdated); -Numpy is a combination of Numeric and Numarray was released in 2006 (Numpy 1.0).

3. NumPy package NumPy is the fundamental package for scientific computing with Python. It contains among other things: a powerful N-dimensional array object sophisticated (broadcasting) functions tools for integrating C/C++ and Fortran code useful linear algebra, Fourier transform, and random number capabilities Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases. from www.numpy.org/

4. NumPy arrays The main feature of NumPy is an array object. All array elements have to be the same type (usually float or integer); Array elements can be accessed, sliced, and manipulated in the same way as the lists; Arrays can be N-dimensional; The number of elements in the array is fixed; Shape of the array can be changed.

5. History of SciPy SciPy is a package for scientific computing that provides a standard collection of common numerical operations on top of the Numeric array data structure. SciPy is a product of merging three pieces of codes (based on Numeric) that were developed by Travis Oliphant, Eric Jones and Pearu Peterson in one package. SciPy was released in 2001. https://en.wikipedia.org/wiki/SciPy#History_of_SciPy

6. SciPy as a scientific tool Modules for: – statistics, optimization; – integration, interpolation; – linear algebra, solving non-linear equations; – Fourier transforms; – ODE solvers, special functions; – signal and image processing.

7. Library overview The SciPy package of key algorithms and functions core to Python's scientific computing capabilities. Available sub- packages include:  constants : physical constants and conversion factors (since version 0.7.0)  cluster : hierarchical clustering, vector quantization, K-means  fftpack : Discrete Fourier Transform algorithms  integrate : numerical integration routines  interpolate : interpolation tools  io : data input and output  lib : Python wrappers to external libraries from https://en.wikipedia.org/wiki/SciPy#The_SciPy_Library.2FPackage

8. Library overview  linalg : linear algebra routines  misc : miscellaneous utilities (e.g. image reading/writing)  ndimage : various functions for multi-dimensional image processing  optimize : optimization algorithms including linear programming  signal : signal processing tools  sparse : sparse matrix and related algorithms  spatial : KD-trees, nearest neighbors, distance functions  special : special functions  stats : statistical functions  weave : tool for writing C/C++ code as Python multiline strings

9. Linear algebra Python’s mathematical libraries, NumPy and SciPy, have extensive tools for numerically solving problems in linear algebra. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

10. Basic computations in linear algebra SciPy has a number of routines for performing basic operations with matrices. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

11. Solving systems of linear equations Solving systems of equations is nearly as simple as constructing a coefficient matrix and a column vector. Suppose you have the following system of linear equations to solve: The first task is to recast this set of equations as a matrix equation of the form Ax=b. In this case, we have: from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

12. Next we construct the array A and vector b as NumPy arrays and find vector x:

13. Eigenvalue problems One of the most common problems in science and engineering is the eigenvalue problem, which in matrix form is written as Ax=λx, where A is a square matrix, x is a column vector, and λ is a scalar (number). Given the matrix A, the problem is to find the set of eigenvectors x and their corresponding eigenvalues λ that solve this equation. We can solve eigenvalue equations like this using scipy.linalg.eig. the outputs of this function is an array whose entries are the eigenvalues and a matrix whose rows are the eigenvectors. Let’s return to the matrix we were using previously and find its eigenvalues and eigenvectors. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

15. Numerical integration When a function cannot be integrated analytically, or is very difficult to integrate analytically, one generally turns to numerical integration methods. SciPy has a number of routines for performing numerical integration. Most of them are found in the same scipy.integrate library. List some of them: quad - single integration dblquad - double integration tplquad - triple integration fixed_quad - Gaussian quadrature, order n trapz - trapezoidal rule polyint - analytical polynomial integration (NumPy) from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

16. Single integrals The function quad is the workhorse of SciPy’s integration functions. Numerical integration is sometimes called quadrature, hence the name. It is normally the default choice for performing single integrals of a function f(x) over a given fixed range from a to b. The general form of quad is scipy.integrate.quad(f, a, b), where f is the name of the function to be integrated and a and b are the lower and upper limits, respectively. The routine uses adaptive quadrature methods to numerically evaluate integrals, meaning it successively refines the subintervals (makes them smaller) until a desired level of numerical precision is achieved. For the quad routine, this is about 10^{- 8}, although it usually does even better. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

17. As an example, let’s integrate a Gaussian function over the range from 0 to 1 We first need to define the function which we do using a lambda expression, and then we call the function quad to perform the integration. The function call scipy.integrate.quad(f, 0, 1) returns two numbers. The first is 0.7468..., which is the value of the integral, and the second is 8.29...e-15, which is an estimate of the absolute error in the value of the integral, which we see is quite small compared to 0.7468 from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

18. Double integrals The scipy.integrate function dblquad can be used to numerically evaluate double integrals of the form. The general form of dblquad is scipy.integrate.dblquad(func, a, b, gfun, hfun), where func if the name of the function to be integrated, a and b are the lower and upper limits of the x variable, respectively, and gfun and hfun are the names of the functions that define the lower and upper limits of the y variable. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

19. As an example, let’s perform the double integral We define the functions f, g, and h, using lambda expressions. Note that even if g, and h are constants, as they may be in many cases, they must be defined as functions, as we have done here for the lower limit. Once again, there are two outputs: the first is the value of the integral (0.5) and the second is its absolute uncertainty (5.551…e-15). from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

20. Solving ODEs The scipy.integrate library has two powerful routines, ode and odeint, for numerically solving systems of coupled first order ordinary differential equations (ODEs). While ode is more versatile, odeint (ODE integrator) has a simpler Python interface works very well for most problems. A typical problem is to solve a second or higher order ODE for a given set of initial conditions. Here we illustrate solving the equation for a driven damped pendulum using odeint. The equation of motion for the angle Θ that the pendulum makes with the vertical is given by from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

21. where t is time, Q is the quality factor, d is the forcing amplitude, and Ω is the driving frequency of the forcing. Reduced variables have been used such that the natural (angular) frequency of oscillation is 1. The ODE is nonlinear owing to the sin Θ term. Of course, it’s precisely because there are no general methods for solving nonlinear ODEs that one employs numerical techniques, so it seems appropriate that we illustrate the method with a nonlinear ODE. The first step is always to transform any nth-order ODE into a system of n first order ODEs of the form: from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

22. We also need n initial conditions, one for each variable yi. Here we have a second order ODE so we will have two coupled ODEs and two initial conditions. We start by transforming our second order ODE into two coupled first order ODEs. The transformation is easily accomplished by defining a new variable ω=dΘ/dt. With this definition, we can rewrite our second order ODE as two coupled first order ODEs: In this case the functions on the right hand side of the equations are from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

23. Note that there are no explicit derivatives on the right hand side of the functions fi; they are all functions of t and the various yi, in this case Θ and ω. The initial conditions specify the values of Θ and ω at t=0. SciPy’s ODE solver scipy.integrate.odeint has three required arguments and many optional keyword arguments, of which we only need one, args, for this example. So in this case, odeint has the form odeint(func, y0, t, args=()) The first argument func is the name of a Python function that returns a list of values of the n functions fi(t, y1,..., yn) at a given time t. The second argument y0 is an array (or list) of the values of the initial conditions of y1,..., yn). The third argument is the array of times at which you want odeint to return the values of y1,..., yn). The keyword argument args is a tuple that is used to pass parameters (besides y0 and t) that are needed to evaluate func. Our example should make all of this clear. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

24. After having written the nth-order ODE as a system of n first- order ODEs, the next task is to write the function func. The function func should have three arguments: (1) the list (or array) of current y values, the current time t, and a list of any other parameters params needed to evaluate func. The function func returns the values of the derivatives dyi/dt = fi(t, y1,..., yn) in a list (or array). Lines 5-11 illustrate how to write func for our example of a driven damped pendulum. The only other tasks remaining are to define the parameters needed in the function, bundling them into a list (see line 22 below), and to define the initial conditions, and bundling them into another list (see line 25 below). After defining the time array in lines 28-30, the only remaining task is to call odeint with the appropriate arguments and a variable, psoln in this case to store output. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

25. The output psoln is an n element array where each element is itself an array corresponding the values of yi for each time in the time t array that was an argument of odeint. For this example, the first element psoln[:,0] is the y0 or theta array, and the second element psoln[:,1] is the y1 or omega array. The remainder of the code simply plots out the results in different formats. The resulting plots are shown in the figure Pendulum trajectory after the code. from http://www.physics.nyu.edu/pine/pymanual/html/chap9/chap9_scipy.html

28. The plots above reveal that for the particular set of input parameters chosen, Q = 2.0, d = 1.5, and Omega = 0.65, the pendulum trajectories are chaotic.

29. Parallel programming with MPI MPI - Message passing interface. MPI is a library that used for running programs in parallel. To use MPI library in Python, we need to install MPI4py module. It provides standard functions, for instance, to get the rank of processors, send/receive messages from various nodes in the clusters. It allows the program to be executed in parallel while messages passing between nodes.

30. Simple examples Hello World program for multiple processes: 5 python processes will be executed (as we specified 5), they can all communicate with each other. When each program runs, it will print ‘hello world from process’, and display its rank:

31. MPI.COMM_WORLD is a static reference to a Comm object, and comm is just a reference to it for our convenience. A communicator is a logical unit that defines which processes are allowed to send and receive messages A group of independent processes called ranks share a communicator. http://materials.jeremybejarano.com/MPIwithPython/introMPI.html

32. When an MPI program is run, each process receives the same code. However, each process is assigned a different rank. This allows us to embed a seperate code for each process into one file. In the following code, all processes are given the same two numbers. However, though there is only one file, 3 processes are given completely different instructions for what to do with them. Process 0 sums them, process 1 multiplies them, and process 2 takes the maximum of them: http://materials.jeremybejarano.com/MPIwithPython/introMPI.html

33. Note the difference between upper/lower case!  send/recv: general Python objects, slow  Send/Recv: continuous arrays, fast

34. Computing Pi https://mpi4py.scipy.org/docs/usrman/tutorial.html

35. Thank you