Computational Methods in Physics PHYS 3437 Dr Rob Thacker Dept of Astronomy & Physics (MM-301C)

Today’s Lecture Use MC techniques to simulate a random walk Use MC techniques to simulate a random walk A few slides on random number generators A few slides on random number generators

Mobility of molecules Consider a single molecule suspended in air, on average how far will it travel in 1s? Consider a single molecule suspended in air, on average how far will it travel in 1s? Naïve calculation: mv 2 /2=(5/2)kT for a diatomic gas, hence Naïve calculation: mv 2 /2=(5/2)kT for a diatomic gas, hence k=1.38x J s -1 and take T=300 K, take mass to be that of N 2 ~28m p =4.68x kg k=1.38x J s -1 and take T=300 K, take mass to be that of N 2 ~28m p =4.68x kg Implies a velocity of v=665 m s -1 Implies a velocity of v=665 m s -1

Distance travelled So does a molecule move 2/3 km on average in 1s? So does a molecule move 2/3 km on average in 1s? No – collisions ensure that the motion changes direction an enormous number of times per second No – collisions ensure that the motion changes direction an enormous number of times per second Result – average molecule moves < 1 m in a second Result – average molecule moves < 1 m in a second The path of a particle frequently interrupted by collisions is called a random walk The path of a particle frequently interrupted by collisions is called a random walk

Random walks & diffusion Gas diffusion can be approximated on the atomic scale by modelling many thousands of particles undergoing random walks Gas diffusion can be approximated on the atomic scale by modelling many thousands of particles undergoing random walks We do not need to know any specific details about the collisions We do not need to know any specific details about the collisions just that they occur randomly just that they occur randomly After each collision the new direction is random After each collision the new direction is random We’ll just look at the properties of one random walk in this lecture though We’ll just look at the properties of one random walk in this lecture though

Scattering angle – random points on a sphere After the collision the incident particle i is scattered in a random direction After the collision the incident particle i is scattered in a random direction The scattering directions should be equally distributed on the surface of a sphere The scattering directions should be equally distributed on the surface of a sphere This is not the same as picking  and  randomly! This is not the same as picking  and  randomly!   i t

Picking points randomly on a sphere Left result is from randomly choosing  [0,2  ] and  [0,  ] Left result is from randomly choosing  [0,2  ] and  [0,  ] Notice that points are more clustered around the pole Notice that points are more clustered around the pole

Spherical geometry If one uses spherical polar coordinates to describe the surface, the area element on the sphere is d  =sin  d  d  If one uses spherical polar coordinates to describe the surface, the area element on the sphere is d  =sin  d  d  Think about the geometry near the poles – when  =0 the  coordinate is completely degenerate Think about the geometry near the poles – when  =0 the  coordinate is completely degenerate Since d  contains a factor of sin  we cannot take a uniform distribution of  &  Since d  contains a factor of sin  we cannot take a uniform distribution of  &  However since –d(cos  )=sin  d  we can write |d  |=d(cos  d  However since –d(cos  )=sin  d  we can write |d  |=d(cos  d  This means we take a uniform distribution of  where  [0,2  ] This means we take a uniform distribution of  where  [0,2  ] We also take a uniform distribution of cos  [-1,1] We also take a uniform distribution of cos  [-1,1] sin  d   dd d  =sin  d  d  sin 

Back of the envelope calculation for mean free path Let  be the cross-sectional area of each particle Let  be the cross-sectional area of each particle Let n=number of particles per unit volume of side length l Let n=number of particles per unit volume of side length l What distance is required so that the accumulated cross section of particles is ½l 2 ? What distance is required so that the accumulated cross section of particles is ½l 2 ? i.e. half of the time the incident particle will collide with another particle after having travelled i.e. half of the time the incident particle will collide with another particle after having travelled l

Average free streaming length Given the distance and cross sectional area l 2 the total number of particles in the volume l 2 is n l 2 Given the distance and cross sectional area l 2 the total number of particles in the volume l 2 is n l 2 Assuming the particles are distributed on a “lattice” that has no overlaps the total accumulated cross section of all these particles is just  n l 2 Assuming the particles are distributed on a “lattice” that has no overlaps the total accumulated cross section of all these particles is just  n l 2 Need to find such that  n l 2 =l 2 /2 Need to find such that  n l 2 =l 2 /2 Simple rearrangement gives =1/(2  n) Simple rearrangement gives =1/(2  n) Note that the factor of ½ is a result of us assuming that there are no overlaps Note that the factor of ½ is a result of us assuming that there are no overlaps

Lattices are not realistic in this case If we think about particles being arranged on a lattice then there are a number of possible paths with an infinite path length between collisions If we think about particles being arranged on a lattice then there are a number of possible paths with an infinite path length between collisions Of course this is improbably unlikely to ever occur in a gas – the particles will be distributed randomly Of course this is improbably unlikely to ever occur in a gas – the particles will be distributed randomly The random distribution will have a number of overlaps which means the factor of proportionality is 1/√2 rather than ½ The random distribution will have a number of overlaps which means the factor of proportionality is 1/√2 rather than ½ Hence Hence

Estimate of the free streaming length Given Given We plug in  =(10 -9 ) 2 m 2 We plug in  =(10 -9 ) 2 m 2 n~N A /(22 l) = 6.02×10 23 / m 3 ~ 2.7×10 25 m -3 n~N A /(22 l) = 6.02×10 23 / m 3 ~ 2.7×10 25 m -3 Find that ~2.6×10 -8 m Find that ~2.6×10 -8 m Diameter of N 2 molecule ~ 3.1× m Diameter of N 2 molecule ~ 3.1× m This is roughly 83 molecular diameters – so not too large! This is roughly 83 molecular diameters – so not too large! Given this information we can now put together a Monte Carlo random walk routine Given this information we can now put together a Monte Carlo random walk routine

MC random walk algorithm 1) Choose  and cos  at random 2) Let the particle move, go to 1 Need to keep track of particles original position (i.e. (0,0,0)) current position, and the number of collisions N Implementation hint: leaves distances in units of. That way you can scale results after the simulation is completed for different.

MC Experiment We perform m experiments, and allow up to N=1000 collisions (for example) We perform m experiments, and allow up to N=1000 collisions (for example) Typically we take m~500 or more, for example Typically we take m~500 or more, for example For each N we calculate the average distance RMS / that the particle has migrated from the origin For each N we calculate the average distance RMS / that the particle has migrated from the origin The average is taken over the m paths The average is taken over the m paths RMS is given by RMS is given by Plot up RMS / and look at results Plot up RMS / and look at results

The underlying law is very obvious here Simulating that many collisions is incredibly difficult right now, although the US has computers capable of almost operations per second

Simple explanation of √N scaling in 1d Consider a random walk in 1d, you can go forwards or back Consider a random walk in 1d, you can go forwards or back Each step in building up the path is given by a random variable x i, x i is either +1 or -1 Each step in building up the path is given by a random variable x i, x i is either +1 or -1 Clearly the average value of x i is zero: =(1+ -1)/2=0 Clearly the average value of x i is zero: =(1+ -1)/2=0 The average value of x i 2 is 1: =(1+1)/2=1 The average value of x i 2 is 1: =(1+1)/2=1 Path endpoint d is given by Path endpoint d is given by The mean of the square of number of paths is then The mean of the square of number of paths is then The cross terms i≠j average to zero: -1×1+-1×-1+1×1+1×-1=0 The cross terms i≠j average to zero: -1×1+-1×-1+1×1+1×-1=0 Hence the RMS distance is Hence the RMS distance is

Different classifications of random- “ness” Truly Random Truly Random Exhibiting true randomness – values cannot be predicted in any way Exhibiting true randomness – values cannot be predicted in any way Pseudorandom Pseudorandom Appearance of randomness but having a deterministic pattern Appearance of randomness but having a deterministic pattern Period of pattern can be exceptionally long Period of pattern can be exceptionally long Must produce “good” approximation to a random deviate Must produce “good” approximation to a random deviate Quasi(sub)-random Quasi(sub)-random Having a set of non-random numbers in a randomized order Having a set of non-random numbers in a randomized order Useful when we want a series of points to have certain properties Useful when we want a series of points to have certain properties e.g. optimal space filling, like Sobol, Halton sequences e.g. optimal space filling, like Sobol, Halton sequences

Hardware generators Linux operating system example: “/dev/random” Linux operating system example: “/dev/random” Gathers information from hardware within the computer through the operating system – “noise” Gathers information from hardware within the computer through the operating system – “noise” creates an “entropy pool” that is used to create the random numbers creates an “entropy pool” that is used to create the random numbers Avoids tracking things such as network traffic that can be manipulated by outsiders Avoids tracking things such as network traffic that can be manipulated by outsiders Was designed to be a true random generator but recent research (March 2006) has shown it has a few weaknesses Was designed to be a true random generator but recent research (March 2006) has shown it has a few weaknesses Other inputs for random information: Other inputs for random information: Readings from a Geiger counter – based on a quantum process, completely unpredictable in theory Readings from a Geiger counter – based on a quantum process, completely unpredictable in theory Detected noise from a radio receiver Detected noise from a radio receiver Thermal or quantum-mechanical noise, amplified to provide a random voltage source. Thermal or quantum-mechanical noise, amplified to provide a random voltage source.

Humour: LavaRnd Lava Lamps! Lava Lamps! SGI created a random generator based upon random bits that were extracted from images of the erupting blobs inside six Lava Lite lamps SGI created a random generator based upon random bits that were extracted from images of the erupting blobs inside six Lava Lite lamps Project is no longer functioning Project is no longer functioning A new project uses the same idea but relies on noise in the ccd of digital cameras A new project uses the same idea but relies on noise in the ccd of digital cameras

Commercial RNGs Operate from USB or Serial connections Operate from USB or Serial connections Standard mode is to deliver one byte of data at a time Standard mode is to deliver one byte of data at a time Current models pass “DIEHARD” battery of tests Current models pass “DIEHARD” battery of tests Very fast generation Very fast generation Prices range anywhere from ~$100 to more than $1,000 per unit Prices range anywhere from ~$100 to more than $1,000 per unit

Types of pseudo random generator Linear Congruential Generators Linear Congruential Generators Based on a simple iterative process Based on a simple iterative process Lagged Fibonnaci Generators Lagged Fibonnaci Generators Utilize a similar concept to the Fibonacci sequence Utilize a similar concept to the Fibonacci sequence Shift Register Generators Shift Register Generators Similar in principle to LFGs Similar in principle to LFGs Combined Generators Combined Generators Can be designed to offer best of combined methods Can be designed to offer best of combined methods

Linear Congruential Generators (LCG) A Linear Congruential Generator (LCG) is defined as A Linear Congruential Generator (LCG) is defined as I j+1 = (I j  a + c) mod m Where – I j – current number [I 0 – seed] Ij +1 – next number a - multiplier c - increment m – modulus The period is clearly (at most) the value of the modulus, m. The period is clearly (at most) the value of the modulus, m. mod m=remainder after dividing by m

Park and Miller “minimal” LCG Really Lewis, Goodman & Miller (1968) – algorithm was extensively reviewed in a later paper by Park and Miller Really Lewis, Goodman & Miller (1968) – algorithm was extensively reviewed in a later paper by Park and Miller By choosing the period and multiplier carefully LCGs of the form I j+1 = I j  a mod m can be made to behave very well By choosing the period and multiplier carefully LCGs of the form I j+1 = I j  a mod m can be made to behave very well This type of form cannot use 0 as the initial seed This type of form cannot use 0 as the initial seed LG & M suggest choosing LG & M suggest choosing a=7 5 and m= a=7 5 and m= Implementing it in a code is straightforward apart from one very important issue Implementing it in a code is straightforward apart from one very important issue The multiplication of I j by a will frequently produce an overflow The multiplication of I j by a will frequently produce an overflow Rather than using a 64bit multiplication we can use “Schrage’s method” to avoid overflow (see slide after summary) Rather than using a 64bit multiplication we can use “Schrage’s method” to avoid overflow (see slide after summary)

RANDU RANDU is an infamous LCG that was widespread on IBM mainframes in 1960s And adopted elsewhere! Definition: I j+1 =(65539I j ) mod 2 31 Straightforward algebra shows that I j+2 =6I j+1 -9I j Describes a series of planes in 3d (15) Many scientific papers were invalidated due to the poor properties of RANDU

Quick & dirty estimates Sometimes you just need a variable to “slightly” randomize things on a very quick timescale Sometimes you just need a variable to “slightly” randomize things on a very quick timescale LCG generators provide a good way of doing this provided we choose m,a,c appropriately LCG generators provide a good way of doing this provided we choose m,a,c appropriately But we want to avoid overflow, which constrains variable values But we want to avoid overflow, which constrains variable values NR gives a list of suitable values for m,a,c NR gives a list of suitable values for m,a,c Example FORTRAN code for a [0:1] deviate: Example FORTRAN code for a [0:1] deviate:j=mod(j*ia+ic,im)ran=float(j)/float(im) Even faster generator are possible if we first check that overflow is a well behaved process Even faster generator are possible if we first check that overflow is a well behaved process low order bits only survive low order bits only survive Then can use something like i= *i Then can use something like i= *i

Lagged Fibonnaci Generators LFGs are based on the concept of the Fibonnaci sequence & are closely related to MRGs LFGs are based on the concept of the Fibonnaci sequence & are closely related to MRGs I n =I n-1 +I n-2 I n =I n-1 +I n-2 1, 1, 2, 3, 5, 8, … 1, 1, 2, 3, 5, 8, … The idea can be generalized to any operator (as opposed to addition) and to any pair of previous values n-j, n-k as opposed to n-1 and n-2 The idea can be generalized to any operator (as opposed to addition) and to any pair of previous values n-j, n-k as opposed to n-1 and n-2 Currently popular LFG ’ s include Additive LFG ’ s based upon the following formula Currently popular LFG ’ s include Additive LFG ’ s based upon the following formula I n =I n-j +I n-k (mod 2 m ) where 0<j<k I n =I n-j +I n-k (mod 2 m ) where 0<j<k k and l are called “ lags ” – the current value is determined by values from l and k places ago k and l are called “ lags ” – the current value is determined by values from l and k places ago

Survey of Numerical Recipes routines You may well come across codes containing these routines, so it is worthwhile knowing what they are You may well come across codes containing these routines, so it is worthwhile knowing what they are Ran0 – Park-Miller LCG Ran0 – Park-Miller LCG Minimal generator Minimal generator Fails statistical tests for N>few 10 7, much less than the period Fails statistical tests for N>few 10 7, much less than the period Ran1 – Park-Miller plus a Bays-Durham shuffle routine Ran1 – Park-Miller plus a Bays-Durham shuffle routine Tries to avoid “low-order” serial correlations Tries to avoid “low-order” serial correlations Improves statistical performance significantly Improves statistical performance significantly Ran2 – Longer period generator (10 18 ) based upon L’Ecuyer’s method of combining sequences (with shuffle) Ran2 – Longer period generator (10 18 ) based upon L’Ecuyer’s method of combining sequences (with shuffle) Believed to produce “perfect” random numbers Believed to produce “perfect” random numbers $1000 waiting if you can prove it doesn’t $1000 waiting if you can prove it doesn’t Ran3 – Knuth’s subtractive method Ran3 – Knuth’s subtractive method Very robust & fast Very robust & fast Ran4 – Data encryption standard based generator Ran4 – Data encryption standard based generator

Mersenne Twister The Mersenne Twister by Matsumoto & Nishimura (1998) is a comparatively new algorithm with many useful properties The Mersenne Twister by Matsumoto & Nishimura (1998) is a comparatively new algorithm with many useful properties An enormous period, , (~ ) – significantly larger than other generators An enormous period, , (~ ) – significantly larger than other generators Provides equidistribution to at least 623-dimensions, and probably higher Provides equidistribution to at least 623-dimensions, and probably higher The algorithm can be implemented efficiently on RISC hardware The algorithm can be implemented efficiently on RISC hardware it avoids multiplication and division it avoids multiplication and division Requires little memory Requires little memory 624 words needed for the working area 624 words needed for the working area Its name derives from the fact that period length is chosen to be a Mersenne prime Its name derives from the fact that period length is chosen to be a Mersenne prime The GSL library includes an implementation of MT The GSL library includes an implementation of MT MT home page contains C, FORTRAN, as well as 32bit and 64bit versions MT home page contains C, FORTRAN, as well as 32bit and 64bit versions

Summary Monte Carlo methods can be applied to simulate physical systems that undergo essentially random interactions Monte Carlo methods can be applied to simulate physical systems that undergo essentially random interactions Can show with numerical experiments that random walk distance is proportional to √N where N is the number of collisions Can show with numerical experiments that random walk distance is proportional to √N where N is the number of collisions Hardware alternatives are available if you really need true random numbers Hardware alternatives are available if you really need true random numbers LCGs are useful for generating quick and dirty sequences, but are outclassed by more modern algorithms LCGs are useful for generating quick and dirty sequences, but are outclassed by more modern algorithms Use the Mersenne-Twister if you want really good random numbers and don’t have any time constraints Use the Mersenne-Twister if you want really good random numbers and don’t have any time constraints

Next Lecture Introduction to simple parallel programming Introduction to simple parallel programming

Schrage’s Method Suppose we have generated an value I j =z, and our next step is create the product az and then take az mod m Suppose we have generated an value I j =z, and our next step is create the product az and then take az mod m If m=aq, for some q, then we see immediately that az mod aq=a(z mod q) If m=aq, for some q, then we see immediately that az mod aq=a(z mod q) Thus we can do the modulo step before we take the product az and avoid overflow Thus we can do the modulo step before we take the product az and avoid overflow Schrage’s Method is exactly this idea extended to allow m=aq+r Schrage’s Method is exactly this idea extended to allow m=aq+r The resulting identity (z≥0) is The resulting identity (z≥0) is For the Park Miller example we take For the Park Miller example we take NON-EXAMINABLE