Tim Marks, Dept. of Computer Science and Engineering Probability Tim Marks University of California San Diego

Tim Marks, Dept. of Computer Science and Engineering Probability Foundations of probability theory were developed around 1654 by Pascal and Fermat. –To answer a question from a gambling French nobleman, the Chevalier de Méré, concerning games of chance We will discuss probability in the context of a much more noble and important problem: Listening to music on my iPod

Tim Marks, Dept. of Computer Science and Engineering My iTunes library (100 songs) Number of songs Genre Decade Classical C Folk F Rock R 20 Seventies S Eighties E Nineties N 17310

Tim Marks, Dept. of Computer Science and Engineering Random experiment Experiment: –A song is randomly selected from my iTunes library –(There are 100 songs in my iTunes library) Outcomes: –There are 100 possible outcomes o 1 = Song 1 is selected o 2 = Song 2 is selected o 100 = Song 100 is selected Event –An event is a set of outcomes Example: C is the event “a classical song is selected” C = {o 2, o 5, o 17, …}  The set of all classical song outcomes

Tim Marks, Dept. of Computer Science and Engineering Events Every event A must have 0 ≤ P(A) ≤ 1 Examples of events: C = a Classical song is selected F = a Folk song is selected R = a Rock song is selected S = a song from the Seventies is selected E = a song from the Eighties is selected N = a song from the Nineties is selected Examples –P(F) = ? –P(N) = ?

Tim Marks, Dept. of Computer Science and Engineering Two special Events The certain event –The universal set (the entire sample space) of outcomes  = {o 1, o 2, o 3, …, o 100 } –P(  ) = 1  Probability that when an outcome occurs (when a song is selected from the iPod), it will be one of the outcomes in the sample space (one of the songs on the playlist). The impossible event –The empty set of outcomes (the complement of the set  ) ø = {} –P(ø) = 0  Probability that when an outcome occurs (when a song is selected from the iPod), it will not be one of the outcomes in the sample space (one of the songs on the playlist).

Tim Marks, Dept. of Computer Science and Engineering Events formed from other Events Intersection –The event “A and B” = A  B P(A and B) = P(A  B) = P(A, B) Examples: –P(R, E) = P(R  E) = ? Genre Decade Classical C Folk F Rock R 20 Seventies S Eighties E Nineties N – P(C, F) = ?

Tim Marks, Dept. of Computer Science and Engineering Events formed from other Events Union –The event “A or B” = A  B P(A or B) = P(A  B) P(A  B) = P(A) + P(B) – P(A, B) Examples: –P(F  S) = P(F) + P(S) – P(F, S) = ? Genre Decade Classical C Folk F Rock R 20 Seventies S Eighties E Nineties N – P(F  R) = ? – P(R  E) = ?

Tim Marks, Dept. of Computer Science and Engineering Conditional probability Examples: = ? Genre Decade Classical C Folk F Rock R 20 Seventies S Eighties E Nineties N – P(F | N) = ? P(A | B) = P(A, B) P(B) The probability of A given B P(R | S) = P(R, S) P(S)

Tim Marks, Dept. of Computer Science and Engineering Statistical Independence Examples: –S and C are independent –E and R are not independent Genre Decade Classical C Folk F Rock R 20 Seventies S Eighties E Nineties N – Are S and F independent? – Are C and R independent? We say that A and B are independent if P(A, B) = P(A)P(B)

Tim Marks, Dept. of Computer Science and Engineering Equivalent conditions for Independence P(A, B) = P(A)P(B)   P(A | B) = P(A) i.e.,knowing whether B happened gives you no information about whether A happened. »Review examples from last slide with this interpretation By similar reasoning,  P(B | A) = P(B)

Tim Marks, Dept. of Computer Science and Engineering Bayes’ Rule (a.k.a. Bayes’ Theorem) Reverend Thomas Bayes (1702–1761) Suppose we know P(A | B). –What if we want to know P(B | A) ? –Can infer P(B | A) from our knowledge of: P(A | B)  Likelihood P(B)  Prior Derivation of Bayes’ Rule (on board) Bayes’ rule: Example: