1. CSC-2259 Discrete Structures
Discrete Probability
CSC-2259 Discrete Structures
Konstantin Busch - LSU

2. Introduction to Discrete Probability
Unbiased die
Sample Space:
All possible outcomes

3. any subset of sample space
Event:
any subset of sample space
Experiment:
procedure that yields events
Throw die
Konstantin Busch - LSU

4. Probability of event :
Note that:
since
Konstantin Busch - LSU

5. What is the probability that
a die brings 3?
Event Space:
Sample Space:
Probability:

6. What is the probability that
a die brings 2 or 5?
Event Space:
Sample Space:
Probability:

7. Two unbiased dice
Sample Space:
36 possible outcomes
First die
Second die
Ordered pair

8. What is the probability that
two dice bring (1,1)?
Event Space:
Sample Space:
Probability:

9. What is the probability that
two dice bring same numbers?
Event Space:
Sample Space:
Probability:

10. Game with unordered numbers
Game authority selects
a set of 6 winning numbers out of 40
Number choices: 1,2,3,…,40
i.e. winning numbers: 4,7,16,25,33,39
Player picks a set of 6 numbers
(order is irrelevant)
i.e. player numbers: 8,13,16,23,33,40
What is the probability that a player wins?
Konstantin Busch - LSU

11. a single set with the 6 winning numbers
Winning event:
a single set with the 6 winning numbers
Sample space:
Konstantin Busch - LSU

12. Probability that player wins:
Konstantin Busch - LSU

13. each kind has 4 suits (h,d,c,s)
A card game
Deck has 52 cards
13 kinds of cards (2,3,4,5,6,7,8,9,10,a,k,q,j),
each kind has 4 suits (h,d,c,s)
Player is given hand with 4 cards
What is the probability that the cards
of the player are all of the same kind?
Konstantin Busch - LSU

14. Event: each set of 4 cards is of same kind Sample Space:
Konstantin Busch - LSU

15. Probability that hand has 4 same kind cards:
Konstantin Busch - LSU

16. Game with ordered numbers
Game authority selects from a bin 5 balls
in some order labeled with numbers 1…50
Number choices: 1,2,3,…,50
i.e. winning numbers: 37,4,16,33,9
Player picks a set of 5 numbers
(order is important)
i.e. player numbers: 40,16,13,25,33
What is the probability that a player wins?
Konstantin Busch - LSU

17. Sampling without replacement: After a ball is selected
it is not returned to bin
Sample space size:
5-permutations of 50 balls
Probability of success:
Konstantin Busch - LSU

18. Sampling with replacement: After a ball is selected
it is returned to bin
Sample space size:
5-permutations of 50 balls
with repetition
Probability of success:
Konstantin Busch - LSU

19. Probability of Inverse:
Proof:
End of Proof
Konstantin Busch - LSU

20. What is the probability that a binary string of 8 bits contains
Example:
What is the probability that
a binary string of 8 bits contains
at least one 0?
Konstantin Busch - LSU

21. Probability of Union:
Proof:
End of Proof
Konstantin Busch - LSU

22. What is the probability that a binary string of 8 bits
Example:
What is the probability that
a binary string of 8 bits
starts with 0 or ends with 11?
Strings that start with 0:
(all binary strings with 7 bits 0xxxxxxx)
Strings that end with 11:
(all binary strings with 6 bits xxxxxx11)
Konstantin Busch - LSU

23. Strings that start with 0 and end with 11:
(all binary strings with 5 bits 0xxxxx11)
Strings that start with 0 or end with 11:
Konstantin Busch - LSU

24. Probability Theory Sample space: Probability distribution function :
Konstantin Busch - LSU

25. Heads (H) with probability 2/3 Tails (T) with probability 1/3
Notice that it can be:
Example:
Biased Coin
Heads (H) with probability 2/3
Tails (T) with probability 1/3
Sample space:
Konstantin Busch - LSU

26. Uniform probability distribution:
Sample space:
Example:
Unbiased Coin
Heads (H) or Tails (T) with probability 1/2
Konstantin Busch - LSU

27. For uniform probability distribution:
Probability of event :
For uniform probability distribution:
Konstantin Busch - LSU

28. What is the probability that the die outcome is 2 or 6?
Example:
Biased die
What is the probability
that the die outcome is 2 or 6?
Konstantin Busch - LSU

29. Combinations of Events:
Complement:
Union:
Union of disjoint events:
Konstantin Busch - LSU

30. Conditional Probability
Three tosses of an unbiased coin
Tails
Heads
Tails
Condition:
first coin is Tails
Question:
What is the probability that
there is an odd number of Tails,
given that first coin is Tails?
Konstantin Busch - LSU

31. Restricted sample space given condition:
first coin is Tails
Konstantin Busch - LSU

32. Event without condition:
Odd number of Tails
Event with condition:
first coin is Tails
Konstantin Busch - LSU

33. the sample space changes to
Given condition,
the sample space changes to
(the coin is unbiased)
Konstantin Busch - LSU

34. Notation of event with condition:
event given
Konstantin Busch - LSU

35. Conditional probability definition:
(for arbitrary probability distribution)
Given sample space with
events and (where )
the conditional probability of given is:
Konstantin Busch - LSU

36. What is probability that a family of two children has two boys
Example:
What is probability that a family
of two children has two boys
given that one child is a boy
Assume equal probability to have boy or girl
Sample space:
Condition:
one child is a boy
Konstantin Busch - LSU

37. Conditional probability of event:
both children are boys
Conditional probability of event:
Konstantin Busch - LSU

38. Events and are independent iff:
Independent Events
Events and are independent iff:
Equivalent definition (if ):
Konstantin Busch - LSU

39. 4 bit uniformly random strings : a string begins with 1
Example:
4 bit uniformly random strings
: a string begins with 1
: a string contains even 1
Events and are independent
Konstantin Busch - LSU

40. Experiment with two outcomes: success or failure
Bernoulli trial:
Experiment with two outcomes:
success or failure
Success probability:
Failure probability:
Example:
Biased Coin
Success = Heads
Failure = Tails
Konstantin Busch - LSU

41. Independent Bernoulli trials:
the outcomes of successive Bernoulli trials
do not depend on each other
Example:
Successive coin tosses
Konstantin Busch - LSU

42. Throw the biased coin 5 times
What is the probability to have 3 heads?
Heads probability:
(success)
Tails probability:
(failure)
Konstantin Busch - LSU

43. Total numbers of ways to arrange in sequence 5 coins with 3 heads:
HHHTT
HTHHT
HTHTH
THHTH
Total numbers of ways to arrange in sequence
5 coins with 3 heads:
Konstantin Busch - LSU

44. Probability that any particular sequence has
3 heads and 2 tails is specified positions:
For example:
HHHTT
HTHHT
HTHTH
Konstantin Busch - LSU

45. Probability of having 3 heads:st
1st
sequence
success
(3 heads)
2nd
sequence
success
(3 heads)
sequence
success
(3 heads)
Konstantin Busch - LSU

46. Throw the biased coin 5 times
Probability to have exactly 3 heads:
Probability to have 3 heads and 2 tails
in specified sequence positions
All possible ways to arrange in sequence 5 coins with 3 heads
Konstantin Busch - LSU

47. Probability to have successes in independent Bernoulli trials:
Theorem:
Probability to have successes
in independent Bernoulli trials:
Also known as
binomial probability distribution:
Konstantin Busch - LSU

48. in specified positions
Proof:
Total number of
sequences with
successes and
failures
Probability that
a sequence has
successes and
failures
in specified positions
Example:
SFSFFS…SSF
End of Proof
Konstantin Busch - LSU

49. Random uniform binary strings probability for 0 bit is 0.9
Example:
Random uniform binary strings
probability for 0 bit is 0.9
probability for 1 bit is 0.1
What is probability of 8 bit 0s out of 10 bits?
i.e
Konstantin Busch - LSU

50. Birthday collision: two people have birthday in same day
Birthday Problem
Birthday collision: two people have birthday
in same day
Problem:
How many people should be in a room
so that the probability of birthday collision
is at least ½?
Assumption: equal probability to be born in any day
Konstantin Busch - LSU

51. If the number of people is 367 or more
366 days in a year
If the number of people is 367 or more
then birthday collision is guaranteed by
pigeonhole principle
Assume that we have people
Konstantin Busch - LSU

52. :probability that people have all different birthdays
We will compute
:probability that people have
all different birthdays
It will helps us to get
:probability that there is
a birthday collision among people
Konstantin Busch - LSU

53. Sample space: Cartesian product Sample space size: 1st person’s
Birthday
choices
2nd person’s
Birthday
choices
nth person’s
Birthday
choices
Sample space size:
Konstantin Busch - LSU

54. each person’s birthday is different
Event set:
each person’s birthday is different
1st person’s
birthday
2nd person’s
birthday
nth person’s
birthday
#choices
#choices
#choices
Sample size:
Konstantin Busch - LSU

55. Probability of no birthday collision
Probability of birthday collision:
Konstantin Busch - LSU

56. Probability of birthday collision:
Therefore:
people have probability
at least ½ of birthday collision
Konstantin Busch - LSU

57. The birthday problem analysis can be used
to determine appropriate hash table sizes
that minimize collisions
Hash function collision:
Konstantin Busch - LSU

58. Randomized algorithms: algorithms with randomized choices
(Example: quicksort)
Las Vegas algorithms:
randomized algorithms whose output
is always correct (i.e. quicksort)
Monte Carlo algorithms:
randomized algorithms whose output
is correct with some probability
(may produce wrong output)
Konstantin Busch - LSU

59. A Monte Carlo algorithm
Primality_Test( ) {
for( to ) {
if (Miller_Test( ) == failure)
return(false) // n is not prime }
return(true) // most likely n is prime
Konstantin Busch - LSU

60. Miller_Test( ) { for ( to ) { if ( or ) return(success) }
return(failure)
Konstantin Busch - LSU

61. A prime number passes the Miller test for every
A composite number passes the
Miller test in range
for fewer than numbers
false positive with probability:
Konstantin Busch - LSU

62. If the primality test algorithm returns false
then the number is not prime for sure
If the algorithm returns true then the
answer is correct (number is prime)
with high probability:
for
Konstantin Busch - LSU

63. Bayes’ Theorem Applications: Machine Learning Spam Filters
Konstantin Busch - LSU

64. Bayes’ Theorem Proof:
Konstantin Busch - LSU

65. Konstantin Busch - LSU

66. End of Proof
Konstantin Busch - LSU

67. Example: Select random box then select random ball in box
Question:
If a red ball is selected,
what is the probability it was taken from box 1?
Konstantin Busch - LSU

68. Question probability:
select red ball
select box 1
select green ball
select box 2
Question probability:
Question:
If a red ball is selected,
what is the probability it was taken from box 1?
Konstantin Busch - LSU

69. We only need to compute:
Bayes’ Theorem:
We only need to compute:
Konstantin Busch - LSU

70. select red ball select box 1 select green ball select box 2 Box 1
Probability
to select box 1
Probability
to select box 2
Konstantin Busch - LSU

71. select red ball select box 1 select green ball select box 2 Box 1
Probability to select
red ball from box 1
Probability to select
red ball from box 2
Konstantin Busch - LSU

72. Final
result
Konstantin Busch - LSU

73. What if we had more boxes?
Generalized Bayes’ Theorem:
Sample space
mutually exclusive events
Konstantin Busch - LSU

74. Spam Filters Training set: Spam (bad) emails Good emails
A user classifies each
in training set as good or bad
Konstantin Busch - LSU

75. Find words that occur in and
number of spam s
that contain word
number of good s
that contain word
Probability that
a good
contains
Probability that
a spam
contains
Konstantin Busch - LSU

76. Event that X contains word
A new X arrives
Event that X is spam
Event that X contains word
What is the probability that X is spam
given that it contains word ?
Reject if this probability is at least 0.9
Konstantin Busch - LSU

77. We only need to compute:
0.5
0.5
simplified
assumption
Computed from training set
Konstantin Busch - LSU

78. Training set for word “Rolex”:
Example:
Training set for word “Rolex”:
“Rolex” occurs in 250 of 2000 spam s
“Rolex” occurs in 5 of 1000 good s
If new contains word “Rolex”
what is the probability that it is a spam?
Konstantin Busch - LSU

79. “Rolex” occurs in 250 of 2000 spam emails
“Rolex” occurs in 5 of 1000 good s
Konstantin Busch - LSU

80. If new email X contains word “Rolex”
what is the probability that it is a spam?
Event that X is spam
Event that X contains word “Rolex”
Konstantin Busch - LSU

81. We only need to compute:
0.5
0.5
simplified
assumption
Computed from training set
Konstantin Busch - LSU

82. New email is considered to be spam because:
spam threshold
Konstantin Busch - LSU

83. Better spam filters use two words:
Assumption: and are independent
the two words appear
independent of each other
Konstantin Busch - LSU