1. Randomized Algorithms (Probabilistic algorithm) Flip a coin, when you do not know how to make a decision!

2. 2 Summary Generate Random Number Randomly permuting arrays Randomized algorithm Performance of randomized algorithm Applications

3. Generate Random Number? Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin( 痴心妄想 ). John von Neumann

4. 4 Generate Random Numbers For many probabilistic algorithms, we need to generate a random number. -Las Vegas algorithms -Monte Carlo algorithms -Simulated Annealing algorithm -Evolutionary Algorithms -Generation of Cryptographic Keys -Many Combinatorial Optimisation Algorithms -DNA computation -Quantum computation

5. 5 Two Types of Random Numbers Pseudorandom numbers are numbers that appear random. they are obtained in a deterministic. They are repeatable. They are predictable. Truly random numbers are generated in non- deterministic ways. They are not predictable. They are not repeatable.

6. 6 The computer is not capable of generating truly random numbers Only generate pseudorandom numbers-- numbers that are generated by a formula Pseudorandom numbers are good enough for most purposes, but not all--for example, not for serious security applications Devices for generating truly random numbers do exist They are based on decay times of radioactive material, radio channel or audible noise…

7. 7 Psuedo-random number generators We really don’t have truly random random number generators. To generate psuedo-random sequence, let s ∈ X be a seed. This seed defines a sequence Where, f : X→X and g: X→Y, X is a sufficiently large set and Y is the domain of pseudorandom values to be generated. y i is the pseudorandom sequence

8. 8 LCG 1. Linear Congruential Generator (LCG) X n =( bX n-1 + c ) mod m where, b, c are constants and are not less than 0, m is 2 32 or 2 64, X 0 is the seed and X 0 is not less than m. Other generators have longer maximum periods. Bad choices of m result in very bad sequences -primes work best, powers of 2 are fast, but not nearly as good. Initial seed affects period.

9. 9 Lagged Fibonacci Generators Similar to Fibonacci Sequence Increasingly popular X n+1 = (X n-l + X n-k ) mod m (l>k>0) l seeds are needed m usually a power of 2

10. 10 Random functions Let a and b be real numbers such that a<b. Randomf(a, b): return a real number x chosen randomly in the interval a≤ x ≤ b Let i and j be real numbers such that i<j. Randomi(i, j): return an integer k chosen randomly in the interval i≤ k ≤ j If Randomf(0, 1) is directly available, then: Randomf(a, b) return a+(b-a)Randomf(0, 1) Randomi(i, j) return

11. 11 Randomly permuting arrays Many randomized algorithms randomize the input by permuting the given input array. We assume that we are given an array B which, without loss of generality, contains the elements 1 through n. Our goal is to produce a random permutation of the array.

12. 12 One common method is to assign each element B[i] of the array a random priority P[i], and then sort the elements of B according to these priorities. PermuteBySorting(B) 1 for i ← 1 to n do 2 P[i] ← Randomi(1, n 3 ) 3 sort B, using P as sort keys 4 return B

13. 13 A better method for generating a random permutation is to permute the given array in place. The procedure RandomizeInPlace does so in O(n) time. In iteration i, the element B[i] is chosen randomly from among elements B[i] through B[n]. Subsequent to iteration i, B[i] is never altered. RandomizeInPlace(B) 1 for i ←1 to n do 2 swap B[i] ↔B[randomi(i,n)]

14. 14 An algorithm is randomized algorithm if its behavior is determined not only by its input but also by values produced by a random-number generator. Randomization and probabilistic analysis are themes that cut across many areas of computer science, especially algorithm design. Randomized algorithm

15. 15 Two views When one thinks about random processes in the context of computation, it is usually in one of two distinct ways. One view is to consider the world as behaving randomly, one can consider traditional (determinated) algorithms that confront randomly generated input. This approach is often termed average-case analysis, since we are studying the behavior of an algorithm on an “average” input, rather than a worst-case input.

16. 16 A second view is to consider algorithms that behave randomly: The world provides the same worst-case input as always, but we allow our algorithm to make random decisions as it processes the input. Thus the role of randomization in this approach is purely internal to the algorithm and does not require new assumptions about the nature of the input.

17. 17 Deterministic vs. Probabilistic Given an instance of a problem: Deterministic – the algorithm always behaves the same. Must return correct answer. Probabilistic – the algorithm might behave differently. Might return correct answer. A randomization of a deterministic algorithm results by replacing certain steps that made canonical choices by steps that make these choices in some random fashion.

18. 18 Deterministic Algorithm: Identical behavior for different runs for a given input. Randomized Algorithm : Behavior is generally different for different runs for a given input. Algorithms DeterministicRandomized Worst-case Analysis Worst-case Running Time Probabilistic Analysis Average Running Time Probabilistic Analysis Average Running Time

19. 19 Thoughts For some problems, it is better to randomly choose, instead of taking the time to take the best choice. Factors: How much difference are between random choices and the best choice How long does it take to calculate the best choice Probabilistic algorithms might return different answers on the same problem instance

20. 20 Randomized algorithms Main characteristic: The same algorithm may behave differently when it is applied twice to the same instance Execution time and even the result obtained may vary considerably from one use to the next Major open question in field: Does every efficient randomized algorithm have an efficient deterministic counterpart? How to decrease the probability of making errors?

21. 21 Benefits: (i) a small probability to go astray (infinite loop, division by zero, etc) is allowed; (ii) the correct answers may be obtained by running many times; (iii) we allow the algorithm to sometimes yield erroneous results (iv) Randomization to improve the worst-case behavior of deterministic algorithms

22. 22 Randomized algorithms fall into four main design categories: 1. Numerical probabilistic algorithms 2. Randomizations of deterministic algorithms(Sherwood algorithms) 3. Las Vegas algorithms 4. Monte Carlo algorithms

23. 23 Given instance, same answer? Guarantee a correct answer? Deterministic Algorithm Start Randomized Algorithm Monte Carlo Algorithms Las Vegas Algorithms Exact Answer? Numerical Algorithms Y Y Y N N N Sherwood Algorithms No Answer? N Y Correct answer or erroneous answer

24. 24 Facetious Illustration How a randomized algorithm would answer the question “When did the WWWII begin?” Numerical algorithm (5 calls): {1940±2; 1939±2; 1937±2;1915±2; 1942±2} in a confidence interval with probability p% Monte Carlo algorithm (10 calls): {1939; 1939; 1939; 1941;1939; 1939; 1939; 1939; 56BC; 1939} ← is correct with probability p% Las Vegas algorithm (10 calls): {1939; 1939; Fail!;1939;1939; 1939; 1939; 1939; Fail !; 1939} ← may fail with probability ε% Sherwood algorithm (10 calls): {1939; 1939; 1939;1939;1939; 1939; 1939; 1939; 1939; 1939} ← success 100%

25. 25 Depending on the problem… We might need to have an exact answer: Sorting We might need an approximate answer: Shortest route to visit every capital city We might need multiple solutions: Minimum Spanning Trees Could get them via randomized, and not deterministic

26. 26 Performance of randomized algorithm The performance of a randomized algorithm depends not only on the input to the algorithm, but also on the results of randomized choices. If we run the algorithm many times with fixed input I, then we can expect that the algorithm performs t exp (I) basic operations on average. If the algorithm performs many random choices, then even for a single run we can expect that the number of basic operations performed is very close to t exp (I).

27. 27 Expected vs. Average Time Let B(n), W(n), A(n) be the best-case, worst-case, and average complexities of a deterministic algorithm. Average Time A(n) – the mean time taken when each possible instance of a given size is equally likely Expected Time t exp (n) – mean time to compute the same instance over and over What does this mean when applied to Deterministic algorithms? Probabilistic algorithms? What does probabilistic average expected time and probabilistic worst case expected time mean?

28. 28 Let B exp (n), W exp (n), A exp (n) be the expected best-case, worst-case, and average complexities of a probabilistic algorithm. These are defined by B exp (n) = min{t exp (I) | I ∈ I n }, W exp (n) = max{t exp (I) | I ∈ I n }, A exp (n) = E(t exp (I)), where I n denotes the set of all inputs of size n to the algorithm.

29. 29 For example The hiring problem Suppose that you need to hire a new office assistant. Your previous attempts at hiring have been unsuccessful, you are committed to having, at all times, the best possible person for the job. Therefore, you decide that, after interviewing each applicant, if that applicant is better qualified than the current office assistant, you will fire the current office assistant and hire the new applicant. You are willing to pay the resulting price of this strategy, but you wish to estimate what that price will be.

30. 30 HireAssistant(n) 1 best ← 0 // candidate 0 is a least-qualified dummy candidate 2 for i ← 1 to n do 3 interview candidate i 4 if candidate i is better than candidate best then 5 best ← i 6 hire candidate i.

31. 31 Interviewing has a low cost, say c i, whereas hiring is expensive, costing c h. Let m be the number of people hired. Then the total cost associated with this algorithm is O(nc i + mc h ). Worst-case /Best-case analysis Probabilistic analysis The use of probability in the analysis of problems. Make assumptions about, the distribution of the inputs.

32. 32 Analysis of the hiring problem let X i be the indicator random variable associated with the event in which the ith candidate is hired. Thus E[X i ] = Pr {candidate i is hired}=1/i.

33. 33 Now we can compute E[X]:

34. 34 Lemma Assuming that the candidates are presented in a random order, algorithm HireAssistant(n) has a total hiring cost of O(c h ln n).

35. 35 Randomized hire algorithms RandomizedHireAssistant(n) 1 randomly permute the list of candidates 2 best ← 0 //candidate 0 is a least-qualified dummy candidate 3 for i ← 1 to n do 4 interview candidate i 5 if candidate i is better than candidate best then 6 best ← i 7 hire candidate i

36. 36 Lemma The expected hiring cost of the procedure RandomizedHireAssistant(n) is O(c h ln n).

37. 37 Applications Random algorithms are useful when dealing with the following problem spaces: 1) The adversary conditions that cause deterministic algorithms to perform poorly can be thwarted using random methods to reduce or eliminate their negative affect. 2) If a search space contains a large number of acceptable solutions, a random sample from the population can be used to efficiently find one of them. 3) Random sampling also helps to obtain a solution from a subset of a population to approximately model the entire population. 4) Deadlock and symmetry problems show that randomness is helpful to load balance resources and avoid or break deadlock conditions.

38. 38 5) Environments where variety and uncertainty are necessary to provide training use randomness to approximate and simulate real-world effects. 6) Simulation must also be able to test scenarios and obtain statistical data, and random algorithms provide the necessary mechanisms to do so. 7) Where creativity is needed, especially in the field of artificial intelligence, random algorithms attempt to make decisions outside of the bounds of determinism and provide controlled noise in the form of unpredictable variety.

39. 39 Advanced randomized algorithm and application Application Optimization, pattern matching Optimization problems Decision making Classification Interconnected networks Pattern matching Computer security Algorithm Genetic algorithms Simulated annealing Bayesian networks k Nearest neighbors Randomized routing String matching Cryptography

40. 40 Hybrid Advanced Algorithms Parallel algorithms Randomized Approximation algorithms On-line algorithms Quantum Computation DNA Computation

41. 41 Homework Exercises: page 26 2,3 Experiments: 1 实现微信红包程序或者随机点名程序