Sect. 1.5: Probability Distributions for Large N: (Continuous Distributions)

For the 1 Dimensional Random Walk Problem We’ve found: The Probability Distribution is Binomial: W N (n 1 ) = [N!/(n 1 !n 2 !)]p n1 q n2 Mean number of steps to the right: = Np Dispersion in n 1 : = Npq Relative Width: (Δ*n 1 )/ = (q ½ )  (pN) ½ for N increasing, mean value increases  N, & relative width decreases  (N) -½ N = 20 p = q = ½ q = 1 – p n 2 = N - n 1

Imagine N getting larger & larger. Based on what we just said, the relative width of W N (n 1 ) gets smaller & smaller & the mean value gets larger & larger. If N is VERY, VERY large, we can treat W(n 1 ) as a continuous function of a continuous variable n 1. For N large, it’s convenient to look at the natural log ln[W(n 1 )] of W(n 1 ), rather than the function itself. Do a Taylor’s series expansion of ln[W(n 1 )] about value of n 1 where W(n 1 ) has a maximum. Detailed math (in text) shows that this value of n 1 is it’s average value = Np. It also shows that the width is equal to the value of the width = Npq.

For N VERY, VERY large, treat W(n 1 ) as a continuous function of n 1. For N large, look at ln[W(n 1 )], rather than the function itself. Do a Taylor’s series expansion of ln[W(n 1 )] about the n 1 for W(n 1 ) = its maximum. Detailed math shows that this value of n 1 is it’s mean = Np. It also shows that the width is equal to = Npq. For ln[W(n 1 )], use Stirling’s Approximation (Appendix A-6) for logs of large factorials. Stirling’s Approximation If n is a large integer, the natural log of it’s factorial is approximately: ln[n!] ≈ n[ln(n) – 1]

In this large N, large n 1 limit, the Binomial Distribution W(n 1 ) becomes (shown in detail in the text): W(n 1 ) = Ŵexp[-(n 1 - ) 2 /(2 )] Here, Ŵ = [2π ] -½ This is called the Gaussian Distribution or the Normal Distribution. We’ve found that = Np, = Npq. The reasoning which led to this for large N & continuous n 1 limit started with the Binomial Distribution. But this is a very general result. Starting with ANY discrete probability distribution & taking the limit of LARGE N, will result in the Gaussian or Normal Distribution. This is called The Central Limit Theorem or The Law of Large Numbers.

One of the most important results of probability theory is The Central Limit Theorem: The distribution of any random phenomenon tends to be Gaussian or Normal if we average it over a large number of independent repetitions. This theorem allows us to analyze and predict the results of chance phenomena when we average over many observations.

Related to the Central Limit Theorem is The Law of Large Numbers: As a random phenomenon is repeated a large number of times, the proportion of trials on which each outcome occurs gets closer and closer to the probability of that outcome, and The mean of the observed values gets closer and closer to the mean  of a Gaussian Distribution which describes the data.

Sect. 1.6: The Gaussian Probability Distribution In the limit of a large number of steps in the random walk, N (>>1), the Binomial Distribution becomes a Gaussian Distribution: W(n 1 ) = [2π ] -½ exp[-(n 1 - ) 2 /(2 )] = Np, = Npq Recall that n 1 = ½(N + m), where the displacement x = mℓ & that = N(p – q). We can use this to convert to the probability distribution for displacement m, in the large N limit (after algebra): P(m) = [2π ] -½ exp[-(m - ) 2 /(2 )] = N(p – q), = 4Npq

P(m) = [2πNpq] -½ exp[-(m – N{p – q}) 2 /(8Npq)] We can express this in terms of x = mℓ. As N  >> 1, x can be treated as continuous. In this case, |P(m+2) – P(m)| << P(m) & discrete values of P(m) get closer & closer together. Now, ask: What is the probability that, after N steps, the particle is in the range x to x + dx? Let the probability distribution for this ≡ P (x). Then, we have: P (x)dx = (½)P(m)(dx/ℓ). The range dx contains (½)(dx/ℓ) possible values of m, since the smallest possible dx is dx = 2ℓ.

After some math, we obtain the standard form of the Gaussian (Normal) Distribution P (x)dx = (2π) -½ σ -1 exp[-(x – μ) 2 /2σ 2 ] μ ≡ N(p – q)ℓ ≡ mean value of x σ ≡ 2ℓ(Npq) -½ ≡ width of the distribution NOTE: The generality of the arguments we’ve used is such that a Gaussian Distribution occurs in the limit of large numbers for all discrete distributions!

P (x)dx = (2π) -½ σ -1 exp[-(x – μ) 2 /2σ 2 ] μ ≡ N(p – q)ℓ σ ≡ 2ℓ(Npq) -½ Note: To deal with Gaussian distributions, you need to get used to doing integrals with them! Many are tabulated!! Is P (x) properly normalized? That is, does  P (x)dx = 1? (limits -  < x <  )  P (x)dx =  (2π) -½ σ -1 exp[-(x – μ) 2 /2σ 2 ]dx = (2π) -½ σ -1  exp[-y 2 /2σ 2 ]dy (y = x – μ) = (2π) -½ σ -1 [(2π) ½ σ] (from a table)  P (x)dx = 1

P (x)dx = (2π) -½ σ -1 exp[-(x – μ) 2 /2σ 2 ] μ ≡ N(p – q)ℓ σ ≡ 2ℓ(Npq) -½ Compute the mean value of x ( ): =  x P (x)dx = (limits -  < x <  )  x P (x)dx =  (2π) -½ σ -1 xexp[-(x – μ) 2 /2σ 2 ]dx = (2π) -½ σ -  (y + μ)exp[-y 2 /2σ 2 ]dy (y = x – μ) = (2π) -½ σ -1  yexp[-y 2 /2σ 2 ]dy + μ  exp[-y 2 /2σ 2 ]dy  yexp[-y 2 /2σ 2 ]dy = 0 (odd function times even function)  exp[-y 2 /2σ 2 ]dy = [(2π) ½ σ] (from a table) = μ ≡ N(p – q)ℓ

P (x)dx = (2π) -½ σ -1 exp[-(x – μ) 2 /2σ 2 ] μ ≡ N(p – q)ℓ σ ≡ 2ℓ(Npq) -½ Compute the dispersion in x ( ) = =  (x – μ) 2 P (x)dx (limits -  < x <  ) =  x P (x)dx =  (2π) -½ σ -1 xexp[-(x – μ) 2 /2σ 2 ]dx = (2π) -½ σ -1  y 2 exp[-y 2 /2σ 2 ]dy (y = x – μ) = (2π) -½ σ -1 (½)(π) ½ σ(2σ 2 ) 1.5 (from a table) = σ 2 = 4Npqℓ 2

Comparison of Binomial & Gaussian Distributions Dots = Binomial Curve = Gaussian with the same mean μ & the same width σ

Some Well-known & Potentially Useful Properties of Gaussians Gaussian Width = 2σ 2σ2σ P(x) =

Areas Under Portions of a Gaussian Distribution Two Graphs with the Same Information in Different Forms

Again, Two Forms of the Same Information Areas Under Portions of a Gaussian Distribution

Sect. 1.7: Probability Distributions Involving Several Variables: Discrete or Continuous

Consider a statistical description of a situation with more than one random variable: Example, 2 variables, u, v The possible values of u are: u 1,u 2,u 3,…u M The possible values of v are: v 1,v 2,v 3,…v M P(u i,v j ) ≡ Probability that u = u i, & v = v j SIMULTANEOUSLY We must have: ∑ i = 1  M ∑ j = 1  N P(u i,v j ) = 1

P(u i,v j ) ≡ Probability that u = u i, & v = v j SIMULTANEOUSLY ∑ i = 1  M ∑ j = 1  N P(u i,v j ) = 1 Let P u (u i ) ≡ Probability that u = u i independent of the value v = v j So, P u (u i ) ≡ ∑ j = 1  N P(u i,v j ) Similarly, let P v (v j ) ≡ Probability that v = v j independent of value u = u i So, P v (v j ) ≡ ∑ i = 1  M P(u i,v j ) Of course, it must also be true that ∑ i = 1  M P u (u i ) = 1 & ∑ j = 1  N P v (v j ) = 1

In the special case that u & v are Statistically Independent or Uncorrelated: Then & only then can we write: P(u i,v j ) ≡ P u (u i )P v (v j )

A General Discussion of Mean Values If F(u,v) = any function of u,v, it’s mean value is: ≡ ∑ i = 1  M ∑ j = 1  N P(u i,v j )F(u i,v j ) If F(u,v) & G(u,v) are any 2 functions of u, v, we can easily show: = + If f(u) is any function of u & g(v) is any function of v, we can easily show: ≠ The only case when the inequality becomes an equality is if u & v are statistically independent.

Sect. 1.8: Comments on Continuous Probability Distributions Everything we’ve discussed for discrete distributions generalizes to continuous distributions in obvious ways. Let u ≡ a continuous random variable in the range: a 1 ≤ u ≤ a 2 The probability of finding u in the range u to u + du ≡ P(u) ≡ P (u)du P (u) ≡ Probability Density of the distribution function Normalization:  P (u)du = 1 (limits a 1 ≤ u ≤ a 2 ) Mean values: ≡  F(u) P (u)du.

Consider two continuous random variables: u ≡ continuous random variable in range: a 1 ≤ u ≤ a 2 v ≡ continuous random variable in range: b 1 ≤ v ≤ b 2 The probability of finding u in the range u to u + du AND v in the range v to v + dv is P(u,v) ≡ P (u,v)dudv P (u,v) ≡ Probability Density function Normalization:  P (u,v)dudv = 1 (limits a 1 ≤ u ≤ a 2, b 1 ≤ v ≤ b 2 ) Mean values: ≡  G(u,v) P (u,v)dudv

Functions of Random Variables An important, often occurring problem is: Consider a random variable u. Suppose φ(u) ≡ any continuous function of u. Question If P (u)du ≡ Probability of finding u in the range u to u + du, what is the probability W(φ)dφ of finding φ in the range φ to φ + dφ? Answer using essentially the “Chain Rule” of differentiation, but take the absolute value to make sure that probability W ≥ 0: W(φ)dφ ≡ P (u)|du/dφ|dφ Caution!! φ(u) may not be a single valued function of u!

Equally Likely  The probability of finding θ between θ & θ + dθ is: P(θ)dθ ≡ (dθ/2π) Question What is the probability W(B x )dB x that the x component of B lies between B x & B x + dB x ? Clearly, we must have –B ≤ B x ≤ B. Also, each value of dB x corresponds to 2 possible values of dθ. Also, dB x = |Bsinθ|dθ

So, we have: W(B x )dB x = 2P(θ)|dθ/dB x |dB x = (π) -1 dB x /|Bsinθ| Note also that: |sinθ| = [1 – cos 2 θ] ½ = [1 – (B x ) 2 /B 2 ] ½ so finally, W(B x )dB x = (π) -1 dB x [1 – (B x ) 2 /B 2 ] -½, –B ≤ B x ≤ B = 0, otherwise W not only has a maximum at B x = B, it diverges there! It has a minimum at B x = 0. So, it looks like    W diverges at B x = B, but it can be shown that it’s integral is finite. So that W(B x ) is a properly normalized probability:  W(B x )dB x = 1 (limits: –B ≤ B x ≤ B)