1. Randomized Algorithms Fundamentals of Algorithmics.
Gilles Brassard and Paul
Bratley.
Chapter 10.
Randomiized
Algorithms
SC440/CSC401/CPE401 1

2. 24 → Introduction (‘Traditional’ algorithms)
Traditional algorithms have the following properties: •
They are always correct
They are deterministic --- there may be more than one
correct output, but the same instance of a problem
always produces the same output
They are always precise --- the answer is not given as a range • •
Each of them operates at the same
efficiency
for
the
same instance of
a problem
→ 2
24 →
Prime factor
Randomiized
Algorithms
SC440/CSC401/CPE401 2

3. 24 → Introduction (‘Randomized’ algorithms)
Randomized (probabilistic) algorithms can be •
Nondeterministic --- they can make random but
correct decisions : the same algorithm may behave differently when it is applied twice to the same
instance
Not very is given,
of a problem.
precise sometimes --- usually the more the better precision can be obtained •
time •
operating at different efficiencies
for the
runs.
same
instance
of a problem in different →
2 or 3
24 →
Prime factor
Randomiized
Algorithms
SC440/CSC401/CPE401 3

4. Incorrect sometimes: hope we have a high, known •
Incorrect sometimes: hope we have a high, known
probability of being correct.
Nonterminating sometimes : do not produce an answer at all (hope fore a low probability for that) •
because •
A randonmized algorithm is one
that
makes
random
choices
during
the
execution
Randomiized
Algorithms
SC440/CSC401/CPE401 4

5. Why do we need randomized (probabilistic algorithms)•
When an algorithm is confronted by a choice, it is
sometimes preferable to choose a course of action at random, rather than spending time to work out which alternative is the best.
Sometimes we do not have a better method than making random choices •
Randomiized
Algorithms
SC440/CSC401/CPE401 5

6. Classes of randomized (probabilistic) algorithms1.
Numerical probabilistic algorithms --- give an
approximation to the correct answer.
Monte Carlo Algorithms --- always give an answer, but there is a probability of being completely wrong. 2. 3.
Las Vegas Algorithms --- sometimes fail to give an
answer, but if an answer is given, it is
correct.
These
are
what we
cover in
this
chapter.
Randomiized
Algorithms
SC440/CSC401/CPE401 6

7. T123 T132 T213 T231 T312 T321 Expected versus average time + •
The average time of a deterministic algorithm refers to
the average time taken by the algorithm when each
possible instance of a given size is
considered
equally
likely.
E.g
sorting 3
integers
T123
T132
T213
T231
T312
T321 1, 2, 3, 2, 3, 1, 3 2 1 A
sorting
algorithm + 6
Randomiized
Algorithms
SC440/CSC401/CPE401 7

8. T1 • The expected time of a probabilistic algorithm is
defined on each individual instance.
It is the mean
time
over
that
and
it would take
over again.
to solve the
same
instance T1 T2 T3 T4 T5 T6 1, 2, 3 A
sorting
algorithm
Compute
the
mean
Randomiized
Algorithms
SC440/CSC401/CPE401 8

9. The worst case expected time of a probabilistic •
The worst case expected time of a probabilistic
algorithm refers to the expected time taken by the worst possible instance of a given size, not the time incurred if the worst possible probabilistic choices are unfortunately taken.
Randomiized
Algorithms
SC440/CSC401/CPE401 9

10. Numerical probabilistic algorithms•
For certain real life problems, computation of an
exact solution is not possible, maybe –
because of uncertainties in the experimental data
to be used
Because a digital computer cannot represent an irrational number exactly
Because a precise answer will take too long to compute – – •
A numerical probabilistic algorithm will give an
approximate answer (a confidence interval can be provided).
The precision of the answer increases when more time is given to the algorithm to work on •
Randomiized
Algorithms
SC440/CSC401/CPE401 10

11. 1/π. Buffon’s needle In the 18th
century, Georges de Buffon, proved that if
you throw a needle at random (in a random position
and at a random angle, with uniform distribution) on a floor made of planks of constant width, if the needle
is exactly half as long as the planks in the floor are wide and if the width of the cracks between the planks
is zero,
the probability
that
the
needle
will
fall
across a
crack is
1/π.
Randomiized
Algorithms
SC440/CSC401/CPE401 11

12. Monte Carlo Algorithms•
There are problems for which no efficient algorithm is
known, and approximate solutions do not make sense.
A Monte Carlo algorithm always gives an answer but occasionally makes a mistake.
But it finds a correct solution with high probability whatever the instance is processed, i.e. there is no instance on which the probability of error is high.
However, no warning is usually given when the algorithm gives a wrong solution. • • •
Example:
to find the mode (most common element) in a
bag of values.
Pick an element at random: the
probability that it is the mode is high, but there is a
Randomiized
Algorithms
SC440/CSC401/CPE401 17

13. non-zero probability that it
is completely wrong. Making several random choices and combining the results improves the
probability of correctness.
Randomiized
Algorithms
SC440/CSC401/CPE401 18

14. Primality testing
The problem is to decide whether a given odd integer is prime or composite.
The problem is important for public-key cryptography. • • A
Naive method
Given an input number N, we check whether it is
divisible by any integer greater than 1 and less than or equal to the square root of N. If the answer is NO, then N is a prime, otherwise not. This relies on the not so trivial fact that if there are any factors other than 1 or N, they
cannot all be greater than this square
root.
Composite
N →
Divisible by
prime
Randomiized
Algorithms
SC440/CSC401/CPE401 19

15. Probabilistic tests
Most popular primality tests are probabilistic tests. In a certain sense, those tests are not really primality •
tests -- they do not determine with certainty whether
number is prime or not.
The basic idea is as follows: a • 1. 2.
Randomly pick a number x called a witness.
Check some formula involving x and the given number N. If the formula is no fit, then N is a composite number and the test stops.
Repeat step 1 unless the required certainty is achieved. 3.
After several iterations, if N is not
composite number, then it can be
prime.
found to be a
declared probably
Randomiized
Algorithms
SC440/CSC401/CPE401 20

16. •
The simplest probabilistic primality test is the Fermat
primality test.
It is sometimes used if a rapid screening of numbers •
is needed, for instance in the key generation phase of
the RSA public key cryptographical
algorithm.
Fermat’s little Theorem
If n is a prime, then
an-1
mod n = 1
For any
integer
a such that
1 ≤ a ≤ n
- 1.
E.g. n
n n =
7, a =
8, a =
1 or 2 or 3 or … 6
2, we have 27
mod 8 = 0 ≠ 1
10, a = 2, we have 29
mod 10 = 2 ≠ 1
Randomiized
Algorithms
SC440/CSC401/CPE401 21

17. mod n ≠ 1) then return “n is composite”
The contrapositive:
If there is a value a such that 1 ≤ a ≤ n – 1 and an-1 mod n ≠ 1, then n is not a prime.
Fermat (n)
a = uniform(1, n-1); if
(an-1
mod n ≠ 1) then return “n is composite”
else return “n is prime” •
However, there are composite numbers such that
an-1
mod n = 1.
For example, n = 15,
414
mod 15 = 1. •
The Miller-Rabin test is a more sophisticated variant of
the Fermat test but with a lower error rate; it is often the method of choice.
Randomiized
Algorithms
SC440/CSC401/CPE401 22

18. Choose a random integer a with 2 ≤ a ≤ n - 2.•
Suppose n > 4 is an odd integer which we want to test
for primality.
Choose a random integer a with 2 ≤ a ≤ n - 2.
Find s and t such that t is odd and n - 1 = 2st . •
2rt
If (at mod n = 1) or (a
mod n = n - 1) for at
least
one r
= 0, ..., s-1, then n is declared "prime";
If not, then n is definitely composite. •
MillerRabin(n, k)
for (i = 1 to k do)
a = uniform(2, n – 2);
if (Btest(a, n) == false) then return return “prime”
“composite”
Randomiized
Algorithms
SC440/CSC401/CPE401 23

19. If (x == 1 or x == n – 1) then return For (i = 1 to s – 1) true
Btest(a, n)
s = 0;
t = n – 1;
Repeat
s = s + 1;
t = t/2;
Until t is odd;
x = at
mod n;
If (x == 1 or x == n – 1) then return
For (i = 1 to s – 1)
true
x = x2
mod n;
if (x == n – 1) then return true
Return false
The Miller-Rabin algorithm always answer when n > 4 is a prime. •
returns
the correct •
When n > 4 is an odd composite, each call
on Btest
has probability at most ¼ of making a mistake.
Randomiized
Algorithms
SC440/CSC401/CPE401 24

20. Las Vegas Algorithms •
Las Vegas algorithms make probabilistic choices to help guide them more quickly to a correct solution. •
Unlike Monte Carlo algorithms, they never return wrong answer.
Two main categories of Las Vegas algorithms. a •
First category •
The first kind is often used when a known
deterministic algorithm to solve a problem runs much
faster on the average than in the worst case, e.g.
quicksort. [See previous lecture]
Randomiized
Algorithms
SC440/CSC401/CPE401 26

21. queens, or we run into a dead end --- failure.
Second category •
The second kind of Las Vegas algorithms now and then make choices that bring the problem solving process to a dead end. For example,
Eight-queen problem •
When we use backtracking, we got the
1st
solution
after building 114 nodes of the 2057 nodes in the state-space tree. •
A greedy Las Vegas algorithm places the queens randomly on successive rows, taking care not to put a queen in an illegal position.
Being a greedy algorithm, we do not backtrack. We either succeed if we manage to place all the
queens, or we run into a dead end --- failure. •
Randomiized
Algorithms
SC440/CSC401/CPE401 33

22. Using a computer, it is calculated that the probability of Q Q Q Q Q Q Q
Using a computer, it is calculated that the probability of
success, p, is approximately
Randomiized
Algorithms
SC440/CSC401/CPE401 34

23. ∑i * P[ X = i] (1− ∑i * p)i -1 p p ∑i * (1− p)i p * 1 − p 1 − p 1 − p
After failing, try, try again •
If we fail to place all the queens in one attempt, we
restart from the beginning. •
The expected number of attempts ∞
until success is
∑i * P[ X = i]
i =1
E[X] = = ∞
∑i *
(1−
p)i -1 p
i =1 1
p ∑i * (1− ∞
p)i
p * 1 − p = = =
1 − p
1 − p p 2 p
i =1
So for the randomized 8-queen,
attempts to reach success.
we expect 8
Randomiized
Algorithms
SC440/CSC401/CPE401 35

24. Finding Hamiltonian paths•
Given a graph, is there a path, any path, that passes
through all the vertices of the graph exactly once. Such a path is called Hamiltonian path.
There are n vertices in graph G.
Try to build the path vertex by vertex. •
Start from an arbitrary vertex, v0 in G and
select a neighbour of v0 to get a path v0v1
randomly . •
Continue to randomly choose the next vertex
stop when all vertices are chosen
and
Randomiized
Algorithms
SC440/CSC401/CPE401 36

25. choose the next vertex randomly from the neighbours •
choose the next vertex randomly from the neighbours
of the current vertex which are not already on the path
Randomiized
Algorithms
SC440/CSC401/CPE401 37

26. If at a certain vi, all its neighbours are already on the •
If at a certain vi, all its neighbours are already on the
path, we pick one neighbour, vj,
change the path from
v0 v1 ,…,vjvj+1,…,
vi-1vi
to v0 v1 ,…,vjvivi-1,…,vj+1 and
continue to
choose
the
next
vertex for the path v5 v0
The path built:
v0 v1 v2v3v4 v5 v3 v
=> v v
v v v v (v
= v ) 4 j 2 v1 v2
What is the new path if
vj = v0?
Randomiized
Algorithms
SC440/CSC401/CPE401 38

27. if (there is vj is in N(vi) for 0 (v0, …, vi) = (v0, …,
Randomized_hamiltonian_path(G)
v0 = Random vertex in G;
i = 0; { do
{ N =
if N
N(vi) – {v0, …,
is not empty {
i = i + 1;
vi-1}
vi = Random vertex
in N; }
≤ j < i - 1)
else
if (there is vj is in N(vi) for 0 { }
(v0, …, vi) = (v0, …,
return no_solution;
vjvivi-1, …,
vj+1);
else
} while (i < |V|
return (v0, …,
Randomiized
Algorithms
- 1);
vi); }
SC440/CSC401/CPE401 39

28. If randomized_hamiltonian_path returns true, a •
If randomized_hamiltonian_path returns true, a
Hamiltonian path for graph G has indeed been found.
It is possible that the algorithm runs into a dead end and fails to find a path, although such a path exists.
The algorithm is good at finding a Hamiltonian path in large graphs.
The algorithm is almost always successful for • • •
graphs of
sufficiently large minimum
degree
Randomiized
Algorithms
SC440/CSC401/CPE401 40