Probability theory Petter Mostad

Sample space The set of possible outcomes you consider for the problem you look at You subdivide into different outcomes only as far as is relevant for your problem The sample space is the start of a simplified model for reality Events are subsets of the sample space

Set theory The complement of subset is denoted The intersection of and is denoted The union of and is denoted and are called mutually exclusive or disjoint if where is the empty set If are subsets of a sample space, then they are called collectively exhaustive if

Venn diagrams

Computations with sets Examples of rules you can prove: If are mutually exclusive and collectively exhaustive, then

Events as subsets The union of events corresponds to ”or”, the intersection to ”and”, and the complement to ”not”. Examples: –A : The patient is given drug X –B : The patient dies – : The patient is given drug X and dies – : The patient is given drug X or s/he dies or both – : The patient does not die – : The patient is not given drug X, and dies

Definitions of probability If A is a set of outcomes of an experiment, and if when repeating this experiment many times the frequency of A approaches a limit p, then the probability of A is said to be p. Or: The probability of A is your ”belief” that A is true or will happen: It is an attribute of your knowledge about A. Over time, events given probability p should happen with frequency p.

Properties of probabilities When probabilities are assigned to all outcomes in a sample space, such that –All probabilities are positive or zero. –The probabilities add up to one. then we say we have a probability model The probability of an event A is the sum of the probabilities of the outcomes in A, and is written P(A)

Computations with probabilities Some consequences of the set-theory results: When and are mutually exclusive, In general,

Conditional probability If some information limits the sample space to a subset A, the relative probabilities for outcomes in A are the same, but they are scaled up so that they sum to 1. We write P(B|A) for the probability of event B given event A. In symbols:

The law of total probability As and are disjoint, we get Together with the definition of conditional probability, this gives the law of total probability:

Statistical independence If P(B|A)=P(B), we say that B is statistically independent of A. We can easily see that this happens if and only if Thus B is statistically independent of A if and only if A is statistically independent of B.

Bayes theorem Bayes theorem says that: This can be deduced simply from definition of conditional probability: Together with the law of total probability:

Example A disease X has a prevalence of 1%. A test for X exists, and –If you are ill, the test is positive in 90% of cases –If you are not ill, the test is positive in 10% of cases. You have a positive test: What is the probability that you have X?

Joint and marginal probabilities Assume are mutually exclusive and collectively exhaustive. Assume the same for. Then: are called joint probabilities or are called marginal probabilities If every is statistically independent of every then the two subdivisions are called independent attributes

Odds The odds for an event is its probability divided by the probability of its complement. What is the odds of A if P(A) = 0.8? What can you say about the probability of A if its odds is larger than 1?

Overinvolvement ratios If you want to see how A depends differently on B or C, you can compute the overinvolvement ratio: Example: If the probability to get lung cancer is 0.5% for smokers and 0.1% for non-smokers, what is the overinvolvement ratio?

Random variables A random variable is a probability model where each outcome is a number. For discrete random variables, it is meaningful to talk about the probability of each specific number. For continuous random variables, we only talk about the probability for intervals.

PDF and CDF For discrete random variables, the probability density function (PDF) is simply the same as the probability function of each outcome. The cumulative density function (CDF) at a value x is the cumulative sum of the PDF for values up to and including x. Example: A die throw has outcomes 1,2,3,4,5,6. What is the CDF at 4?

Expected value The expected value of a discrete random variable is the weighted average of its possible outcomes, with the probabilities as weights. For a random variable X with outcomes x 1, x 2, …, x n with probabilities P(x 1 ), P(x 2 ), …, P(x n ), the expected value E(X) is Example: What is the expected value when throwing a die?

Properties of the expected value We can construct a new random variable Y=aX+b from a random variable X and numbers a and b. (When X has outcome x, Y has outcome ax+b, and the probabilities are the same). We can then see that E(Y) = aE(X)+b We can also construct for example the random variable X*X = X 2

Variance and standard deviation The variance of a stochastic variable X is where is the expected value. The standard deviation is the square root of the variance. We can show that We can also show

Example: Bernoulli random variable A Bernoulli random variable X takes the value 1 with probability p and the value 0 with probability 1-p. E(X) = p Var(X) = p(1-p) What is the variance for a single die throw?

Some combinatorics How many ways can you make ordered selections of s objects from n objects? Answer: n*(n-1)*(n-2)*…*(n-s+1)) How many ways can you order n objects? Answer: n*(n-1)*…*2*1 = n! (”n faculty”) How many ways can you make unordered selections of s objects from n objects? Answer: