Computational Complexity Jang, HaYoung BioIntelligence Lab.

Algorithm Analysis Why Analysis? to predict the resources that the algorithm requires, such as computational time, memory, communication bandwidth, or logic gates. The running time of an algorithm is the number of primitives operations or “step”(machine independent) executed.

Complextity Space / Memory Time Count a particular operation Count number of steps Asymptotic complexity

Time Complexity Worst-case an upper bound on the running time for any input Average-case We shall assume that all inputs of a given size are equally likely. Best-case to get the lower bound

Time Complexity Sequential search in a list of size n worst-case : n times best-case : 1 times average-case :

Asymptotic Notation Asymptotic upper bound, we use - notation. For a given function g(n), we denote by  (g(n)) the set of functions; (g(n)) = {f(n): there exist positive constants c and n 0 such that f(n) = n 0 }.

Asymptotic Notation -notation provides an asymptotic lower bound. For a given function g(n), we denote by (g(n)) the set of functions (g(n)) = {f(n): there exist positive constants c and n 0 such that f(n) >= cg(n) for all n >= n 0 }.

Asymptotic Notation (g(n)) = {f(n) : there exist positive constants c 1, c 2, and n 0 such that c 1 g(n) = n 0 }.

Asymptotic Notation

The sets (n 2 ), (n 2 ), and (n 2 )

Practical Complexities 10 9 instructions/second computer

Impractical Complexities 10 9 instructions/second computer

Faster Computer Vs Better Algorithm Algorithmic improvement more useful than hardware improvement. E.g. 2 n to n 3

Intractability A polynomial-time algorithm is one whose worst- case time complexity is bounded above by a polynomial function of its input size. example - worst-case time complexity - Polynomial-time : 2n, 3n 3 + 4n, 5n + n 10, n log n - Non polynomial-time : Intractable problem - No polynomial-time algorithm can solve it. W(n)   (p(n))

Three Categories of Problems (1) 1. Problems for which polynomial-time algorithms have been found 2. Problems that have been proven to be intractable - The first type is problems that require a non- polynomial amount of output. e.g. Determining all Hamiltonian Circuits. - The second type of intractability occurs when our requests are reasonable and we can prove that the problem cannot be solved in polynomial time. e.g. Halting Problem, Presburger Arithmetic Problem

Three Categories of Problems (2) Presburger Arithmetic is the theory of integers with addition (Z,+,=,<,0,1) and is known to require doubly exponential nondeterministic time. 3. Problems that have not been proven to be intractable but for which polynomial-time algorithms have never been found e.g. 3-SAT, 0-1 Knapsack, TSP, Sum-of-Subset, Partition, Graph-Coloring, Independent Set, Vertex-Cover, Clique,3D-Matching, Set Cover, etc.

The Sets P and NP (1) Definition P is the set of all decision problems that can be solved in polynomial-time. A NP algorithm have two stages: 1. Guessing (in nondeterministic polynomial time) Stage 2. Verification (in deterministic polynomial time) Stage Definition NP is the set of all decision problems that can be solved in nondeterministic polynomial-time.

The Sets P and NP (2)

Genetic Algorithms

An Abstract View of GA generate initial population G(0); evaluate G(0); t := 0 ; repeat t := t + 1; generate G(t) using G(t-1); evaluate G(t); until termination condition has reached;

Search Techniques SEARCH TECHNIQUES Calculus-based techniques. Guided Random Search Techniques Enumerative Techniques Directed Methods Indirected Methods Fibonacci Newton Simulated Evolutionary Annealing Algorithms Evolutionary Genetic Algorithms strategies Dynamic PGMing Parallel Sequential GAs Classes of Search techniques

Simple Genetic Algo's components 1. A mechanism to encode the solutions as binary strings 2. A population of binary strings 3. A fitness function 4. Genetic operators 5. Selection mechanism 6. Control parameters

Population (chromosomes) Selection (mating pool ) Evaluation (fitness) Genetic Operators Mates Manipulation New Generation Offspring* Decoded strings Parents* Reproduction = Evaluation + Selection Crossover & Mutation The GA Cycle

The mechanism for evaluating each string To maintain uniformity over various problem domains, normalize the obj.function to 0 to 1. The normalized value of the obj. function = the fitness of the string Fitness function (object function)

Models nature's "survival-of-the-fittest " mechanism A fitter string receives higher number of offspring. Proportionate selection scheme The roulette wheel selection scheme Selection

After Selection, pairs of string are picked at random If string length = n, randomly choose a number from 1 to n - 1, then use it as a crossover point. GA invokes crossover ONLY IF a randomly generated n o > p c. (p c = the crossover rate) Crossover

After crossover, string are subjected to mutation Flipping bits : 0 1, 1 0 Mutation rate : P m = probability that a bit will be flipped The bits in a string are independently mutated. = Role : restoring lost genetic material Mutation

Function Definition Find x from the range [-1, 2] which maximizes the f

Analysis of function f

Representation (1) Representation of string six places after decimal point The range [-1,2] should be divided into at least ranges. Binary representation : 22 bits Mapping from a binary string into a real number x Convert the binary string from the base 2 to base 10: Find a corresponding real number x:

Representation (2) String Example : string ( ) String x 2.0

Initial Population Create a population of strings, where each chromosome is a binary vector of 22 bits. All 22 bits for each string are initialized randomly.

Evaluation Function eval(v) = f(x) For example, Strings v 1=( ) v 2=( ) v 3=( ) xf(x) x 1= x 2= x 3= f(x 1 ) = f(x 2 ) = f(x 3 ) = The string v 3 is the best of the three strings, since its evaluation returns the highest value. The string v 3 is the best of the three strings, since its evaluation returns the highest value.

Genetic Operators : Mutation Mutation Mutation alters one or more genes with a probability equal to mutation rate. Mutation Example : v 3=( ) v 3’=( ) v 3’’=( ) Fifth gene10th gene

Genetic Operators : Crossover Crossover Example : The crossover on v2 and v3 Assume that the crossover point was randomly selected after 5th gene: v 2=(00000 | ) v 3=(11100 | ) v 2=(00000 | ) v 3=(11100 | ) v 2’=(00000 | ) v 3”=(11100 | ) v 2’=(00000 | ) v 3”=(11100 | ) Before Crossover After Crossover x 2= x 3= f(x 1 ) = f(x 2 ) = x 2’= x 3”= f(x 1’ ) = f(x 2” ) =

Parameters Population size = 50 Probability of crossover = 0.25 Probability of mutation = 0.01

Experimental results The best chromosome after 150 generations Vmax = ( ) Xmax = As expected,, and f(x max ) is slightly larger than 2.85.