Graduate School of Information Sciences, Tohoku University Physical Fluctuomatics Applied Stochastic Process 3rd Random variable, probability distribution and probability density function Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) Probability Event and Probability Joint Probability and Conditional Probability Bayes Formula, Prior Probability and Posterior Probability Discrete Random Variable and Probability Distribution Continuous Random Variable and Probability Density Function Average, Variance and Covariance Uniform Distribution Gauss Distribution Last Talk Present Talk Physics Fluctuomatics / Applied Stochastic Process (Tohoku University)

Probability and Random Variable We introduce a one to one mapping X(A) from every events A to a mutual different real number. The mapping X(A) is referred to as Random Variable of A. The random variable X(A) is often denoted by just the notation X. Probability of the event X=x that the random variable X takes a real number x is denoted by Pr{X=x}. Here, x is referred to as the state of the random variable X．The set of all the possible states is referred to as State Space. If events X=x and X=x’ are exclusive of each other, the states x and x’ are excusive of each other. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Continuous Random Variable 　Random Variable in Discrete State Space 　　　　　　　　　　Example:{x1,x2,…,xM} Continuous Random Variable: 　Random Variable in Continuous State Space 　　　　　　　　　　Example：(−∞,+∞) Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Probability Distribution Let us suppose that the sample W is expressed by Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj are exclusive of each other. We introduce a one to one mapping X:Ai xi (i=1,2,…,M). If all the probabilities for the events X=x1, X=x2,…, X=xM are expressed in terms of a function P(x) as follows: Random Variable State Variable State the function P(x) and the variable x is referred to as Probability Distribution and State Variable, respectively. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Probability Distribution Probability distributions have the following properties: Normalization Condition Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Average and Variance Average of Random Variable X : μ Variance of Random Variable X: σ2 s：Standard Deviation Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Joint Probability Distribution If the joint probability Pr{(X=x)∩(Y=y)}= Pr{X=x,Y=y} is expressed in terms of a function P(x,y) as follows: P(x,y) is referred to as Joint Probability Distribution. Probability Vector State Vector Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Marginal Probability Distribution Let us suppose that the sample W is expressed by Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj are exclusive of each other. We introduce a one to one mapping X:Ai xi (i=1,2,…,M). Marginal Probability Distribution Simplified Notation Summation over all the possible events in which every pair of events are exclusive of each other. Normalization Condition Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Discrete Random Variable and Marginal Probability Marginal Probability of High Dimensional Probability Distribution Marginal Probability Distribution X Y Z U Marginalize Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Independency of Discrete Random Variable If random variables X and Y are independent of each others, Probability Distrubution of Random Variable Y Joint Probability Distribution of Random Variables X and Y Probability Distribution of Random Variable X Marginal Probability Distribution of Random Varuiable Y Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Covariance of Discrete Random Variables Covariance of Random Variables X and Y Covariance Matrix Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Example of Probability Distribution Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Example of Joint Probability Distributions Cov[X,Y] Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Example of Conditional Probability Distribution Conditional Probability of Binary Symmetric Channel Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Continuous Random Variable and Probability Density Function For a random variable X defined in the state space (−∞,+∞), the probability that the state x is in the interval (a,b) in expressed as Distribution Function Probability Density Function Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Continuous Random Variable and Probability Density Function Normalization Condition Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Average and Variance of Continuous Random Variable Average of Random Variable X Variance of Random Variable X Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Continuous Random variables and Joint Probability Density Function For random variables X and Y defined in the state space (−∞,+∞), the probability that the state vector (x,y) is in the region (a,b)(c,d) is expressed as 確率変数 X と Y の状態空間 (−∞,+∞) において状態 x と y が区間 (a,b)×(c,d) にある確率 Joint Probability Density Function Normalization Condition Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Continuous Random Variables and Marginal Probability Density Function Marginal Probability Density Function of Random Variable Y Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Independency of Continuous Random Variables Random variables X and Y are independent of each other. Probability Density Function of Y Joint Probability Density Function of X and Y Probability Density Function of X Marginal Probability Density Function Y Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Covariance of Continuous Random Variables Covariance of Random Variables X and Y Covariance Matrix Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Uniform Distribution U(a,b) Probability Density Function of Uniform Distribution p(x) x a b (b-a)-1 Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Gauss Distribution N(μ,σ2) Probability Density Function of Gauss Distribution with average μ and variance σ2 p(x) μ x The average and the variance are derived by means of Gauss Integral Formula Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Multi-Dimensional Gauss Distribution For a positive definite real symmetric matrix C, two-Dimensional Gaussian Distribution is defined by The covariance matrix is given in terms of the matrix C as follows: by using the following d -dimentional Gauss integral formula Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Law of Large Numbers Let us suppose that random variables X1,X2,...,Xn are identical and mutual independent random variables with average m. Then we have Central Limit Theorem We consider a sequence of independent, identical distributed random variables, {X1,X2,...,Xn}, with average m and variance s2. Then the distribution of the random variable tends to the Gauss distribution with average m and variance s2/n as n+. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) Summary Event and Probability Joint Probability and Conditional Probability Bayes Formula, Prior Probability and Posterior Probability Discrete Random Variable and Probability Distribution Continuous Random Variable and Probability Density Function Average, Variance and Covariance Uniform Distribution Gauss Distribution Last Talk Present Talk Physics Fluctuomatics / Applied Stochastic Process (Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-1 Let us suppose that a random variable X takes binary values 1 and the probability distribution is given by Derive the expression of average E[X] and variance V[X] and draw their graphs by using your personal computer. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-2 Let us suppose that random variables X and Y take binary values 1 and the joint probability distribution is given by Derive the expressions of Marginal Probability Destribution of X, P(X), and the covariance of X and Y, Cov[X,Y]. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-3 Let us suppose that random variables X and Y take binary values 1 and the conditional probability distribution is given by Show that it is rewritten as Hint cosh(c) is an even function for any real number c. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-4 Prove the Gauss integral formula: Hint Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-5 Let us suppose that a continuous random variable X takes any real number and its probability density function is given by Prove that the average E[X] and the variance V[X] are given by Draw the graphs of p(x) for μ=0, σ=10, 20, 40 by using your personal computer. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-6 Make a program for generating random numbers of uniform distribution U(0,1). Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000. In the C language, you can use the function rand() that generate one of values 0,1,2,…,randmax, randomly. Here, randmax is the maximum value of outputs of rand(). Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-7 Make a program that generates random numbers of Gauss distribution with average m and variance σ2. Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000. For n random numbers x1,x2,…,xn generated by any probability distribution, (x1+x2+…+xn )/n tends to the Gauss distribution with average m and variance σ2/n for sufficient large n. [Central Limit Theorem] Hint: First we have to generate twelve uniform random numbers x1,x2,…,x12 in the interval [0,1]. Gauss random number with average 0 and variaince 1 σξ+μ generate Gauss random numbers with average μ and variance σ2 Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-8 For any positive integer d and d d positive definite real symmetric matrix C, prove the following d-dimensional Gauss integral formulas: Hint: By using eigenvalues λi and their corresponding eigenvectors (i=1,2,…,d) of the matrix C, we have Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

Physics Fluctuomatics/Applied Stochastic Process (Tohoku University) Practice 3-9 We consider continuous random variables X1,X2,…,,Xd. The joint probability density function is given by Prove that the average vector is and the covariance matrix is C. Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)