CS6825: Probability Distributions An Introduction

Recall Probability Properties The probability of an event, say event A, is denoted P(A). The probability of an event, say event A, is denoted P(A). All probabilities are between 0 and 1. All probabilities are between 0 and 1. (i.e. 0 < P(A) < 1) The sum of the probabilities of all possible outcomes must be 1. The sum of the probabilities of all possible outcomes must be 1.

Remember our Shoe Examples and the Probabilities P(Nike) = 46/100 =.46 P(Adidas) = 24.5/100 =.245 P(Reebok) = 18.5/100 =.185 P(Asics) = 6.5/100 =.065 P(Other) = 4.5/100 =.045

Frequency distribution To better understand the main features of the data, we portray the frequency distribution which quickly reveals the shape of the data. To better understand the main features of the data, we portray the frequency distribution which quickly reveals the shape of the data. Frequency Distribution is a grouping of data into mutually exclusive classes (categories for nominal data, ranks for ranking, numerical ranges for interval and ratio data) showing the number of observations in each Frequency Distribution is a grouping of data into mutually exclusive classes (categories for nominal data, ranks for ranking, numerical ranges for interval and ratio data) showing the number of observations in each

Probability Distribution Probability Distribution is a Frequency Distribution where the frequency is represented by the probability (scale 0 to 1.0) Probability Distribution is a Frequency Distribution where the frequency is represented by the probability (scale 0 to 1.0) So, a probability distribution gives the chance of every event (if you have a discrete variable) or the probability across all ranges of events (if you have a continuous variable). So, a probability distribution gives the chance of every event (if you have a discrete variable) or the probability across all ranges of events (if you have a continuous variable).

Probability Distribution of our Shoe Examples

Steps to build frequency distribution (Suggestions but not the rules) 1. Decide on the number of classes  Sometimes you have natural knowledge of your problem and can define the classes you care about. Like in our shoe example “Nike”, “Reebok”, etc.  Sometimes you do not know the number of classes you need: Network Router Working, Network Router Failed. But, what about Router working at X% capacity? What about Y% and Z%?  When you cant decide on your number of classes: Good rule of thumb. To estimate the number of classes for “n” observations (test samples you have), find the smallest number k, such that 2 k >n

Steps to build frequency distribution 2. Define your class: Determine the class interval or width.  Not all classes are represented by distinct values like “Nike” in our shoe example. Instead they may have a continuous range of values. For example, think about a set of classes representing colors. You might have classes like “light blue”, “medium blue” and “dark blue”. How are these blue classes separated? This is referred to as the class interval or width in this case with respect to the color wheel (or color space).  Rule of Thumb: the class interval should be greater or equal than the difference between the highest and the lowest value divided by the number of classes. With no prior information can split a range of 0 to X values between k classes by spans of X/k. Hence:  Class 1 = range 0 to X/k  Class 2 = range X/k to 2X/k  Class 3 = range 2X/k to 3X/k  Class 4 = range 3X/k to 4X/k  Class k = range (k-1)X/k to X

Steps to build frequency distribution (Suggestions but not the rules) 3. Tally the observations (test data) into the appropriate classes. 4. Count the number of tallies (items) in each class. If doing Probability distribution (rather than just frequency), you simply divide the number of tallies of each class by the total number of observations. This gives us an estimated probability for each class

EXAMPLE Professor X wishes prepare to a report showing the number of hours per week students spend studying. She selects a random sample of 30 students and determines the number of hours each student studied last week. Professor X wishes prepare to a report showing the number of hours per week students spend studying. She selects a random sample of 30 students and determines the number of hours each student studied last week. Organize the data into a frequency distribution Organize the data into a frequency distribution 15.0, 23.7, 19.7, 15.4, 18.3, 23.0, 14.2, 20.8, 13.5, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 21.4, 18.3, 29.8, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 17.8, 33.8, 23.2, 12.9, 27.1, , 23.7, 19.7, 15.4, 18.3, 23.0, 14.2, 20.8, 13.5, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 21.4, 18.3, 29.8, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 17.8, 33.8, 23.2, 12.9, 27.1, 16.6.

Example Step 1 Determine the number of classes Step 1 Determine the number of classes There are 30 observations so n=30.There are 30 observations so n= =32>302 5 =32>30 Lets start with the assumption of at least 5 classes, i.e., k=5 (p.s. this may not be valid but, is a place to start….there are much better ways than guessing like this and are referred to a the estimation of k or the number of modes in a multi-modal distribution…..this is an area of continued research)Lets start with the assumption of at least 5 classes, i.e., k=5 (p.s. this may not be valid but, is a place to start….there are much better ways than guessing like this and are referred to a the estimation of k or the number of modes in a multi-modal distribution…..this is an area of continued research)

Example continued Step 2 Define Classes First determine the range of our possible class values.First determine the range of our possible class values. Min = 10.3Min = 10.3 Maximum = 33.8Maximum = 33.8 Interval range i:Interval range i: round up interval to 5 round up interval to 5 Set the lower limit of the first class at 10 hours (could make it 10.3)Set the lower limit of the first class at 10 hours (could make it 10.3) Class 1 = range 10 to 15Class 1 = range 10 to 15 Class 2 = range 15 to 20Class 2 = range 15 to 20 Class 3 = range 20 to 25Class 3 = range 20 to 25 Class 4 = range 25 to 30Class 4 = range 25 to 30 Class 5 = range 30 to 35Class 5 = range 30 to 35 AGAIN this like choosing k=5 Is a GUESS and may (will often) be wrong. There are much more informed techniques than this…but, this is simple for you as a beginner to understand.

Example continued Step 3 &4 place and count observations Class 1 = 8 / 30 Class 1 = 8 / 30 Class 2 = 11/ 30 Class 2 = 11/ 30 Class 3 = 7 / 30 Class 3 = 7 / 30 Class 4 = 3 / 30 Class 4 = 3 / 30 Class 5 = 1 / 30 P(Class 1) = P(Class 2) = P(Class 3) = P(Class 4) =.1 P(Class 5) = Class 5 = 1 / 30 P(Class 1) = P(Class 2) = P(Class 3) = P(Class 4) =.1 P(Class 5) = TEST DATA: 15.0, 23.7, 19.7, 15.4, 18.3, 23.0, 14.2, 20.8, 13.5, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 21.4, 18.3, 29.8, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 17.8, 33.8, 23.2, 12.9, 27.1, 16.6.

Normal Distribution If you have an outcome that is continuous and is caused by lots of independent factors it can probably be modeled with a Normal (aka Gaussian) distribution. The Normal distribution looks bell shaped and it is described by two values, a mean and variance. If you have an outcome that is continuous and is caused by lots of independent factors it can probably be modeled with a Normal (aka Gaussian) distribution. The Normal distribution looks bell shaped and it is described by two values, a mean and variance. Using the Normal function is commonly a technique used. Sometimes a Combination of normal functions (called a mixture of normals) is used. However, there are many techniques to automatically figure this out as well as to use other functions to model your distribution. Note –even a single point can be a normal function with 0 variance...even If this a possibly foolish model.

Normal approximation The actual distribution of baby weights at a hospital and approximated by Normal Distribution with mean=3400 grams variance= (standard deviation = 600 grams) The actual distribution of baby weights at a hospital and approximated by Normal Distribution with mean=3400 grams variance= (standard deviation = 600 grams)