資訊理論 授課老師 : 陳建源 研究室 : 法 401 網站 Ch1: Elements of Probability

Outcome: the observed occurrence of the experiment Finite probability: the number of outcomes is finite Discrete probability: the number of outcomes is countable 1. 1 Probability 擲骰子二次，第一次出現 5 的 outcomes? 擲骰子若干次，直到出現 5 為止之 outcomes Outcome: (5,1), (5,2), (5,3), (5,4), (5,5), (5,6) Outcome: (5), (1,5), (2,5), (3,5), (4,5), (5,5), (6,5), (1,1,5)…

Ch1: Elements of Probability Def. 1: all possible outcomes : O 1, O 2, …, O n, associated probability P( O i ) lies in [0, 1] Then, P(O 1 ) + P(O 2 ) +…+ P(O n ) =1 Finite probability: the number of outcomes is finite 1. 1 Probability Discrete probability: the number of outcomes is countable

Ch1: Elements of Probability Coin: head H and tail T 1. 1 Probability Ideal coin: P(H) = P(T) = 1/2 Loaded or biased coin: P(H) ≠ P(T) perfect die: P(1) = P(2) = P(3) = P(4) = P(5) = P(6) =1/6

Ch1: Elements of Probability Def: The probability of an event E is the sum of the probability of the outcomes in which E occurs 1. 1 Probability If a coin is tossed once, let E 1 be the event that H appears. P(E 1 ) = P(H) = 1/2 If a coin is tossed twice, let E 2 be the event that H appears only once. P(E 2 ) = P(HT)+P(TH) = ¼ + ¼ = 1/2 0 ≦ P(E) ≦ 1

Ch1: Elements of Probability Def: E 1 and E 2 are said to be mutually exclusive If E 1 occurs, E 2 cannot and vice verse 1. 1 Probability E 1 ∩ E 2 = ψ P(E 1 ∩ E 2 )=0 Thm: P(E 1 ∪ E 2 ) =P(E 1 ) +P(E 2 )- P(E 1 ∩ E 2 ) E 1 and E 2 are said to be mutually exclusive iff P(E 1 ∪ E 2 ) =P(E 1 ) +P(E 2 )

Ch1: Elements of Probability 1. 1 Probability P(E 1 ∩ E 2 ) ≧ 0 Thm: P(E 1 ∪ E 2 ) =P(E 1 ) +P(E 2 )- P(E 1 ∩ E 2 ) P(E 1 ∪ E 2 … ∪ E n ) ≦ P(E 1 ) +P(E 2 )+…+P(E n ) P(E 1 ∪ E 2 ) ≦ P(E 1 ) +P(E 2 ) 多個 events

Ch1: Elements of Probability 1. 1 Probability P(E 1 ) = ½ P( E 2 ) = 1/2 Ex: If we toss a coin twice, E 1 ： a head appears on the first toss E 2 ： a head appears on the second toss P(E 1 ∩ E 2 ) =1/4 P(E 1 ∪ E 2 ) =P(E 1 ) +P(E 2 )- P(E 1 ∩ E 2 ) =1/2 +1/2 -1/4 = 3/4

Ch1: Elements of Probability 1. 2 Samples 已知 n 個物件，選出 r 個 Sampling with replacement: 可以重覆選 Sampling without replacement: 不可以重覆選 1/n r 1/n(n-1)…(n-r+1)

Ch1: Elements of Probability 1. 3 Conditional Probability Def: Conditional Probability Conditional Probability The probability of an event can alter as more is learned about it

Ch1: Elements of Probability 1. 3 Conditional Probability EX: A carton contains 80 light bulbs of which 20 are defective. What is probability that a second bulb chosen at random is defective if the first is not replaced? E 1 ： the event that the first bulb is defective E 2 ： the event that the second bulb is defective

Ch1: Elements of Probability 1. 3 Conditional Probability EX: A carton contains 80 light bulbs of which 20 are defective. What is probability that a second bulb chosen at random is defective if the first is not replaced? E 1 ： the event that the first bulb is defective E 2 ： the event that the second bulb is defective

Ch1: Elements of Probability 1. 4 Independence E 1 and E 2 are said to be statistically independent E 1 and E 2 are said to be statistically independent iff

Ch1: Elements of Probability 1. 4 Independence EX: The probability that a lawyer has a car accident in one year is p 1, and for a miner is p 2. If there are 5 times as many miners as lawyers, find the probability that one person selected at random from the combined group will have an accident in the second year if the person has one in the first year. E 1 ： the events of an accident in the first year E 2 ： the events of an accident in the second year

Ch1: Elements of Probability 1. 4 Independence

Ch1: Elements of Probability 1. 4 Independence E 1, E 2, E 3 are said to be statistically independent iff

Ch1: Elements of Probability 1. 5 The law of large numbers Given arbitrarily small ε>0 and δ>0, then For sufficiently large n. Ex: In an infinite decimal let the occurrence of 5 be regarded as a success and any other digit as a failure. If all digits are equally likely p = 1/10. Then the law of large numbers says that, of the first n figures, n/10 will be 5 with high probability as n→ ∞.