1. Complexity Theory in Practice
The Power of Randomization
and Monte Carlo Methods
Presented by
Jonathan Pittman

2. Introductory Outline Background & Motivation
Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

3. Outline Background & Motivation Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

4. Background & Motivation
Deterministic Algorithms May:
Not terminate
Be difficult to implement
Require unrealistic resources
Randomized Algorithms May:
Be elegant
Be easier to implement
Have small resource requirements

5. Simple Motivation Example: Quicksort [3]
Best Case: O(n log n)
Worst Case: O(n2)
The input decides the complexity
Random pivot selection makes the worst case extremely infeasible

6. Simple Motivation Example: Numeric Integration [1]
Deterministically taking evenly spaced samples of a function
Fine for one variable
Very bad for larger degrees of freedom
100 dimensional vector with a grid of 10 points per dimension requires points
Randomly selecting points in N-dimensional space and averaging
Converges much faster
Sacrifices some accuracy

7. Outline Background & Motivation Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

8. PRNGs (Some Examples)‏
Linear Congruential Generators (LCGs)
Lagged Fibonacci Generators (LFGs)‏
Additive
Multiplicative

9. Outline Background & Motivation Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

10. Simple MC Methods Example: Approximate π [5]
Circle's Area =
Square's Area =
Circle to Square Ratio = d

11. Outline Background & Motivation Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

12. Parallel MC Method Simulations
MC Methods can and do benefit from parallelism [1]
Parallelizing PRN generators can introduce correlations [2]
Take care & test the output of your generator for the required statistical properties! ... (or use well tested ones, like SPRNG [4])‏

13. Parallel MC Method Example: CSE8163 - Parallel and Distributed Scientific Computing
Parallel Direct Simulation Monte Carlo
A team project in CSE8163, Parallel and Distributed Scientific Computing
Simulate Particle Collisions
Goals:
Exploit the natural parallelism found in Monte Carlo methods
Do so without damaging the results

14. Example Result

15. Outline Background & Motivation Pseudorandom Numbers (PRNs)
Monte Carlo (MC) Methods
Parallel MC Method Simulations
Conclusion

16. Thank you for your time References & Questions
[1] J. Amar, “The Monte Carlo Method in Science and Engineering,” Computing in Science
and Engineering, vol. 8, no. 2, 2006, pp
[2] M. Mascagni and A. Srinivasan, “Algorithm 806: SPRNG: a scalable library for
pseudorandom number generation,” ACM Transactions on Mathematical Software, vol.
26, no. 3, Sep. 2000, pp
[3] B. M. Moret, The Theory of Computation, Chapter 8.4: The Power of Randomization,
Addison-Wesley, Reading, Massachusetts, 1998, pp
[4] M. Mascagni, Sourcebook of Parallel Computing, Chapter 8.9: Deterministic Monte Carlo
Methods and Parallelism, Morgan Kaufmann, San Fransisco, California, 2003, pp.
[5] Wikipedia contributors, "Monte Carlo method," Wikipedia, The Free Encyclopedia,
(current Sep. 16, 2007).

17. Algorithm: Las Vegas Algorithm
The hierarchy of randomized complexity classes
Theorem: NP is a subset of PP 17

18. Randomized algorithms provide solutions for decision problems,
Las Vegas Algorithm and Monte Carlo algorithm are Randomized algorithms
Randomized algorithms provide solutions for decision problems,
Infact they are also used to provide approximate solutions for optimization problems
Both the Monte Carlo and Las Vegas algorithms are one sided decision algorithms
One sided means either ‘yes’ or ‘no’ instance
A Monte Carlo algorithm runs a polynomial time but may provides error with probability less than some constant say ½
Las Vegas Algorithm and Monte Carlo algorithm are Randomized algorithms Randomized algorithms provide solutions for decision problems, Infact they are also used to provide approximate solutions for optimization problems Both the Monte Carlo and Las Vegas algorithms are one sided decision algorithms One sided means either ‘yes’ or ‘no’ instance A Monte Carlo algorithm runs a polynomial time but may provides error with probability less than some constant say ½ 18

19. On the other hand, A Las Vegas algorithm never errs but may not run in polynomial time on all instances
In computing, a Las Vegas algorithm is a randomized algorithm that never gives incorrect results;
That is, it always produces the correct result or it informs about the failure.
In other words, a Las Vegas algorithm does not gamble with the truth of the result --- it only gambles with the resources used for the computation. A simple example is randomized quick sort. The usual definition of a Las Vegas algorithm includes the restriction that the expected run time always be finite, 19

20. running time on instance x is f(x)
Instead, it runs in polynomial time on average- i.e assuming all the instances of size n are equally likely and that the
running time on instance x is f(x)
The expression, ∑x 2-n f(x) is the sum of all the instances x of size n bounded by a polynomial in n.
Given a “no” instance, the computation tree will be having only “no” leaves, but if the instance is “yes” then the computation tree contains at least one “yes” leaf. 20

21. The problem can be solved in NP by using a randomized method i
The problem can be solved in NP by using a randomized method i.e by producing and verifying the random certificate.
If the result is “yes” then the probability of error will be zero, on the other side, if the result is “no” instance then the probability of error will be large.
Generally, the number of possible certificates will be 2|x| and since only one of them will lead to acceptance, the probability of error is bounded by (1- 2-|x| ) times the probability that instance x is a “yes” instance.
Since the bound depends on the input size, we cannot achieve a fixed probability of error by using a fixed number of trials. 21

22. In general we can say, nondeterministic algorithm is a generalization of a Monte Carlo algorithm and the latter itself is a generalization of Las Vegas algorithm.
Random Turing machine is model of computation, which is similar to nondeterministic machine, in that it has a choice of moves at each step and thus make decisions, where as the random Turing machine takes decision by tossing a fair coin.
A RTM operates in polynomial time if the height of its computation tree is bounded by a polynomial function. 22

23. If the computation is terminated after a polynomial number of moves, then the machine may not reach a conclusion.
Leaves of polynomially bounded computation tree will be one of “yes” , “no”, or “don’t Know”.
Definition of PP
PP is the class of all decision problems for which there exists a polynomial time random Turing machine (PTRTM) such that, for any instance x of TT;
--- if x is a “yes” instance, then the machine accepts x with probability larger than ½;
---if x is a “no” instance, then the machine rejects x with probability larger than ½; 23

24. “B” indicates the probability is bounded away from ½.
BPP is the class of all decision problems for which there exists a polynomial time random Turing machine (PTRTM) and a positive constant ε≤ ½ such that, for any instance x of TT;
--- if x is a “yes” instance, then the machine accepts x with probability no less than ½ + ε;
---if x is a “no” instance, then the machine rejects x with probability larger than ½+ ε;
“B” indicates the probability is bounded away from ½.
RP is the class of all decision problems for which there exists a polynomial time random Turing machine (PTRTM) and a positive constant ε≤ 1 such that, for any instance x of TT;
--- if x is a “yes” instance, then the machine accepts x with probability no less than ε;
---if x is a “no” instance, then the machine rejects x 24

25. RP is a one-sided class and so its complementary class will be coRP
RP U coRP corresponds to problems for which Monte Carlo algorithm exist where as RP n coRP corresponds to problems for which Las Vegas algorithms exist
Lemma 8.1
A problem TT belongs to RP n coRP iff there exits a PTRTM and +ve constant ε ≤ 1 such that,
--- the machine accepts or rejects an arbitrary instance with probability no less than ε
--- the machine accepts only “yes” instances and rejects only “no” instances. 25

26. Similarly we can define RP, and coRP
This definition is almost similar to the definition of NP coNP, with one change, i.e here ε depends on the instance, where as in the NP coNP it depends on the problem.
Similarly we can define RP, and coRP
We can immediately conclude that RP coRP is a subset of
NP coNP, RP is a subset of NP, coRP is a subset of coNP.
Furthermore, because all computation trees are limited to polynomial height, it is obvious that all of these classes are contained within Pspace. 26

27. Theorem 8.27 NP is a subset of PP
Finally, because no computation tree is required to have all of its leaves labeled “yes” for a “yes” instance and labeled “no” for a “no” instance, we can conclude that P is contained within all of these classes.
Theorem NP is a subset of PP
Random Turing machine is used to approximate the nondeterministic machine for a problem in NP.
Say, M be the nondeterministic machine and M’ be the random machine,
M accepts a “yes” instance with probability larger than zero but not larger than any fixed constant 27

28. Here we need to make the probability larger than 1/2 , and this can be done by tossing a coin before the start of any computation and at the same time we have to accept the instance a priori (pa ) if the toss produces say heads.
Hence the probability of acceptance of “yes” instance x is now at least ½+2-p(|x|) and the probability of rejection of a “no” instance, which was exactly 1 without the coin toss, is now 1- pa =1/2.
The solution is quite simple: it is enough to make Pa less than 1/2 , while still large enough so that Pa +2-p|x| >1/2. 28

29. Tossing an additional p(|x|) coins will do:
M’ accepts a priori exactly when the first toss returns heads and the next p(|x|) tosses do not return tails, so that pa = ½-2-p(|x|)-1.
Thus a ‘yes’ instance is accepted with probability pa + 2-p(|x|) = ½ +2-p(|x|)-1 and a “no” instance is rejected with probability 1-pa =1/2 +2-p(|x|)-1 . Because, M’ runs in polynomial time iff M does, hence our conclusion. 29

30. 1). Theory of computation: BERNARD M. MORET
References:
1). Theory of computation: BERNARD M. MORET
2) Wikipedia contributors, Wikipedia, The Free Encyclopedia,
Sep. 16, 2007).
Thank you for your time
Questions? if you have any I will try to answer 30

31. introduction theorem 8.28 theorem 8.29 theorem 8.30 conclusion
outline
introduction
theorem 8.28
theorem 8.29
theorem 8.30
conclusion

32. introduction Monto-corlo algorithms Las vegas algorithms
Hierarchy of randomized complexity classes

33. We prove this theorem by containment in each direction
Statement : ZPP equals RP ᴒ coRP
proof :
We prove this theorem by containment in each direction
1). ZPP C RP ᴒ coRP
construct a machine M' to answer the condition RPᴒcoRP and cuts the execution
at polynomial times

34. Theorem 8.28
This machine does not return any answer for some probability to prove the condition that probability is bounded by condition(ԑ > 0).
Let q() be the polynomial bound on the expected running time of M.
Let r() be the polynomial bound of M' where we are going to stop M on all paths of execution and results a desired result ԑ.
r(n) + r'(n) = q(n)‏

35. Theorem 8.28
All the computation paths that lead to a leaf with in bound r() do so in r(n).
pₓ be the probability that the machine does not answer.
For a instance size n, the expected running time of M is given by (1-pₓ).r(n)+pₓ.tmax(n)‏
By solving for pₓ we can make pₓ<1 always such that pₓ<ԑ.

36. Theorem 8.28
2). RP ᴒ coRP C ZPP
Let M be the machine for problem RP ᴒ coRP and M' answers the condition for ZPP.
1/k be the probability that results no answer
r() be the polynomial bound on the running time of M.
Kq(n) is the bound on time required to solve an instance of size n deterministically.

37. Theorem 8.28
M' runs M up to q(n), when M returns answer M' returns the same answer and stops.
If none of the q(n) trails for M returns the answer then M' runs the instance deterministically
The expected time of running M' is
(1-k-q(n)).r(n)+k-q(n).kq(n)
Hence the M' is bounded by polynomial in n.
Therefore ZPP equals RP ᴒ coRP

38. Theorem 8.29
Statement : let Π be a problem in BPP. Then, for any ᵟ>0, there exist a polynomial-time randomized algorithm that accepts “yes” instances and rejects “no” instances of Π with the probability at least 1- ᵟ.
Proof :
since Π belongs to BPP, it has an randomized algorithm A that accepts “yes” instances and rejects “no” instances of Π with the probability at least ½+ԑ for some ԑ>0.

39. Theorem 8.29
yes_count :=0;
for I := 1 to k do
if A(x) accepts
then yes_count := yes_count+1;
if yes_count > k div 2
then accept
else reject
if x is a “yes” instance of Π, then A(x) accepts with probability atleast (½ + ԑ).
Thus probability of observing exactly j acceptances in k runs of A(x) is at least
(kj)(½+ԑ)j.(½-ԑ)(k-j)‏

40. Theorem 8.29
When j does not exceed k/2, by equalizing the two powers in the above equation,
(kj)( ½ +ԑ)j.( ½ +ԑ)(k-j) < (kj)( ¼ -ԑ2 )k/2.
Summing the probabilities for values of j not exceeding k/2, our new algorithm rejects “yes”instance with the probability
By choosing k we can ensure that (1-4ԑ2)k/2< ᵟ.
K is a constant depending on the input constant ᵟ.

41. Theorem 8.30
Statement : BPP ia a subset of Ʃ2p ᴒ∏2p (Which are the nondeterministic and conondeterministic classes complexity at second level of polynomial hierarchy.
Proof:
if NP is not equal to co-NP then, either of these is closed.
But BPP is closed,so BPP cannot be equal to NP or co-NP.

42. Theorem 8.30
It results that adding an oracle to a machine for class BPP does not increase its power, represented by the notation BPPBPP equals BPP.
The above result holds good for class P but NPNP is the super set of NP.
Therefore we can say that BPP does not hold to contain NP, the scope of randomized algorithms is fairly restricted.

43. Theorem 8.30 Consider the Largest class, PP
Membership in PP is not likely to be of much help as the probabilistic guarantee on the error bound is very poor.
The amount of probability exceeds the bound depends on the instance size, this quantity is only
2-p(n) for instance of size n.
Reducing the error makes PP and #P closer.
#P is the class of enumeration problems corresponding to the decision problems in NP

44. Theorem 8.30
We known that the complete problem for #p is a complete problem for PP.
PP contains the decision version of the problem in #P.
An oracle for PP is also a good oracle for #P, hence Ppp is equal to p#P.
Therefore BPP which is the subset of P is further subset of #P which contains the decision problems of NP, hence BPP is subset of NPᴒcoNP.

45. conclusion
Randomized algorithms have the potential for providing the efficient and elegant solutions for many problems.
The concern about randomized algorithm is their dependence on the random bits they use.
The bits are not really random ,the randomized algorithms are fully deterministic, the entired computation is completely fixed.

46. References:
1). Theory of computation: BERNARD M. MORET