Results Matter. Trust NAG. Numerical Algorithms Group Mathematics and technology for optimized performance Optimisation and the NAG Toolbox for MATLAB University of Warwick Mick Pont, John Holden - NAG

University of Warwick - January 31 st 20082The NAG Toolbox for MATLAB Agenda…  NAG licences at Warwick  NAG Toolbox for MATLAB  Optimization and other topics

University of Warwick - January 31 st 20083The NAG Toolbox for MATLAB Licences at the University of Warwick Site Licence for eight implementations: Site contact:  NAG’s Serial Libraries (Fortran 77, 90, C & DMC)  REVIEW OF LICENCES UNDERWAY – PROBABLE CHANGES:  Windows (Fortran and C DLLs)  32-bit Linux (Fortran-g77 and C-gcc)  64-bit Linux (Fortran and C) NAG Toolbox for MATLAB  Licence upgrade “in progress”  Maple-NAG Connector  collaborative licence opportunities – contact NAG direct

University of Warwick - January 31 st 20084The NAG Toolbox for MATLAB What does the University licence cover?  Unlimited use for the licensed implementations  As long as for academic or research purposes  Installation may be on any university, staff or student machine  Full access to NAG Support  Need to use address  Our software:  Includes online documentation - also  Supplied with extensive example programs  data and results

University of Warwick - January 31 st 20085The NAG Toolbox for MATLAB Technical Agenda  The NAG Engine  Wrapper functionality  Algorithmic contents  Navigating around the NAG toolbox in MATLAB  NAG Optimization Chapters  Random numbers

University of Warwick - January 31 st 20086The NAG Toolbox for MATLAB The NAG Engine NAG Engine (algorithmic repository) NAG C Library NAG Fortran Library Other NAG Software NAG SMP Library User-callable library routines are thin wrappers NAG software is based on NAG Engine technology NAG Toolbox for MATLAB

University of Warwick - January 31 st 20087The NAG Toolbox for MATLAB NAG Libraries Ease of Integration  C++ (various)  C# /.NET  Visual Basic  Java  Borland Delphi  Python  …  and more  Excel  MATLAB  Maple  LabVIEW  R and S-Plus  SAS  Simfit  …  and more

University of Warwick - January 31 st 20088The NAG Toolbox for MATLAB NAG Toolbox for MATLAB  Built as MATLAB mex files  Auto-generated from XML documentation  Contains essentially all NAG functionality  not a subset  Currently runs under Windows (32-bit) or Linux (32/64-bit)  Installed under the usual MATLAB toolbox directory  Makes use of a DLL or shared version of the NAG Library

University of Warwick - January 31 st 20089The NAG Toolbox for MATLAB NAG Library Contents  Root Finding  Summation of Series  Quadrature  Ordinary Differential Equations  Partial Differential Equations  Numerical Differentiation  Integral Equations  Mesh Generation  Interpolation  Curve and Surface Fitting  Optimization  Approximations of Special Functions  Dense Linear Algebra  Sparse Linear Algebra  Correlation and Regression Analysis  Multivariate Analysis of Variance  Random Number Generators  Univariate Estimation  Nonparametric Statistics  Smoothing in Statistics  Contingency Table Analysis  Survival Analysis  Time Series Analysis  Operations Research

University of Warwick - January 31 st The NAG Toolbox for MATLAB NAG naming scheme  Based on ACM modified SHARE classification  John Bolstad - A proposed classification scheme for computer program libraries. ACM SIGNUM Newsletter, November 1975  References: Nottingham (England) Algorithms Group, First Annual Report, June,  Routines separated into chapters  Each routine has a 5-letter base name for ease of classification, e.g. (MATLAB/Fortran / C)  c06eb / C06EBF / c06ebc  nag_fft_hermitian(long name for readability)  Each routine is independently documented, with complete example program

University of Warwick - January 31 st The NAG Toolbox for MATLAB Chapter e04 – Minimization / Maximization Problem: minimize F(x 1, x 2, …, x n ) possibly subject to constraints The function F(x) is called the objective function. We wish to determine x, the n-vector of variables. May have:  No constraints  Bound constraints: l i <= x i <= u i  Linear or nonlinear constraints: l <= G(x) <= u

University of Warwick - January 31 st The NAG Toolbox for MATLAB Unconstrained optimization

University of Warwick - January 31 st The NAG Toolbox for MATLAB Linearly constrained optimization

University of Warwick - January 31 st The NAG Toolbox for MATLAB Nonlinear constraints

University of Warwick - January 31 st The NAG Toolbox for MATLAB Chapter e04 Problems categorized according to properties of objective function:  nonlinear  sum of squares of nonlinear functions  quadratic  sum of squares of nonlinear functions  linear Example – nonlinear objective and constraints: Minimize f(x,y) = (1 − x) (y − x 2 ) 2 subject to x 2 + y 2 <= 2 -2 <= x <= 2

University of Warwick - January 31 st The NAG Toolbox for MATLAB E04WD  Sequential quadratic programming (SQP) algorithm  obtains search directions from a sequence of QP subproblems.  designed for problems with many variables and constraints  P. Gill (San Diego), W. Murray (Stanford) and M. Saunders (Stanford)

University of Warwick - January 31 st The NAG Toolbox for MATLAB Chapter e04 It is important to choose a method appropriate to your problem type, for efficiency and the best chance of success. NAG documentation is comprehensive – for advice see the Chapter Introduction for e04:

University of Warwick - January 31 st The NAG Toolbox for MATLAB Some routines available in Chapter e04  e04ab: minimize a function of one variable  e04dg: minimization using conjugate gradients  e04mf: linear programming  e04nc: linear least-squares  e04nf: quadratic programming  e04nq: LP or QP (for sparse problems)  e04un: nonlinear least-squares  e04vh: general sparse constrained nonlinear  e04wd: general nonlinear all-purpose  etc.

University of Warwick - January 31 st The NAG Toolbox for MATLAB New optimization coming at next Mark Currently many optimization routines in NAG, but these have all been for local optimization. No guarantee about which minimum (or maximum) is returned.

University of Warwick - January 31 st The NAG Toolbox for MATLAB Local optimization

University of Warwick - January 31 st The NAG Toolbox for MATLAB Global requirements Users often ask for global optimization methods. In next releases of NAG Libraries we will have software based on 'multilevel coordinate search' (MCS) method - Huyer and Neumaier: Search space is recursively split into sub-boxes, looking for child boxes where gain in objective is expected. Boxes swept through in turn, perhaps being split, until a box with maximum level exists. Then a local search is performed. Already in NAG Engine - new Chapter e05 Beta available now on request

University of Warwick - January 31 st The NAG Toolbox for MATLAB New NAG Chapter – E05  Main routine named E05JB  Plus initialization and option setting routines  Currently handles only bound constraints: Minimize f(x 1, x 2 … x n ) Subject to l i <= x i <= u i

University of Warwick - January 31 st The NAG Toolbox for MATLAB Chapter g05 – Random Numbers Random numbers – used to model real-life processes. Humans are bad at choosing them.  e.g. faking a random sequence of coin tosses is very difficult: HTHHTHTTHTHHTTTHTTHHTTHTHHTHTH …  Most people would choose a sequence easily proved not to be random  (Do you play the lottery?) Therefore good algorithms are required.

University of Warwick - January 31 st The NAG Toolbox for MATLAB Chapter g05 – Random Numbers Random numbers from non-uniform distributions  Generated from uniform numbers  Transformation methods  Rejection methods  Table search methods All the likely suspects are there:  Normal (Gaussian) distribution  Student’s t distribution  Beta and exponential  Chi-squared and binomial  Geometric, poisson, F, Gamma, …

University of Warwick - January 31 st The NAG Toolbox for MATLAB Some routines available in Chapter g05  g05ca: pseudo-random from uniform distribution  g05cb / g05cc: initialize generator to repeatable or non-repeatable sequence respectively  g05db: exponential distribution  g05dd: normal distribution  g05fe: beta distribution  g05ff: gamma distribution  g05ya: quasi-random numbers  Other distributions, plus GARCH, copula, etc.

University of Warwick - January 31 st The NAG Toolbox for MATLAB Pseudo- versus Quasi-Random Numbers Pseudo-random numbers:  generated systematically  e.g. multiplicative congruential n i = a n i-1 mod m  properties close to true random numbers  (assuming that a and m are chosen wisely)  negligible correlation between consecutive numbers Quasi-random numbers:  not statistically independent  give more even distribution in space (“looks more random”)  useful for Monte Carlo integration

University of Warwick - January 31 st The NAG Toolbox for MATLAB Other NAG software We’d welcome contact on our other software:  Maple-NAG Connector  NAG's Excel related products  Fortran Builder (NAG’s Windows Fortran compiler)  New library functionality  NAG’s High Performance libraries ..

University of Warwick - January 31 st The NAG Toolbox for MATLAB NAG Toolbox for MATLAB - summary  Runs under Windows and Linux  Currently on beta test  Downloadable from NAG web site  Can be used with MATLAB compiler  Release version coming soon Questions? or