Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan1 Heuristic Optimization Methods Prologue Chin-Shiuh Shieh

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan2 Course Information 課程名稱 : 最佳化方法 –Heuristic Optimization Methods 授課教師 : 謝欽旭 上課時間 : ( 一 )1-2,5 上課教室 : 育 504 –Could be switched to 資 501B 課程網站 –

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan3 Course Information (cont) Objective –The study of heuristic optimization methods Course Outline –Introduction to Optimization –Calculus, Optimization, and Search –Linear Programming, Combinatorial Optimization –Heuristic Approaches to Optimization –Genetic Algorithm –Ant Colony System –Simulated Annealing –Particle Swarm Optimization –Tabu Search –Memetic Algorithm

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan4 Course Information (cont) Readings –Kwang Y. Lee and Mohamed A. El-Sharkawi, Eds., Modern Heuristic Optimization Techniques, John Wiley & Sons, –Johann Dr´eo, Patrick Siarry, Alain P´etrowski, and Eric Taillard, Metaheuristics for Hard Optimization, Springer, 2006.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan5 What Is Optimization? Optimization defined in Wikipedia –In mathematics, statistics, empirical sciences, computer science, or management science, mathematical optimization (alternatively, optimization or mathematical programming) is the selection of a best element (with regard to some criteria) from some set of available alternatives.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan6 What Is Optimization? (cont) A mathematical formulation –Given: a function f : A  R from some set A to the real numbers –Sought: an element x 0 in A such that f(x 0 ) ≤ f(x) for all x in A ("minimization") or such that f(x0) ≥ f(x) for all x in A ("maximization").

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan7 Challenges Object function High dimensionalities Local optimum Huge search space

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan8 Example Optimization Problems Minimize the costs of shipping from production facilities to warehouses Maximize the probability of detecting an incoming warhead (vs. decoy) in a missile defense system Place sensors in manner to maximize useful information Determine the times to administer a sequence of drugs for maximum therapeutic effect Find the best red-yellow-green signal timings in an urban traffic network Determine the best schedule for use of laboratory facilities to serve an organization’s overall interests

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan9 Major Subfields Linear Programming, a type of convex programming, studies the case in which the objective function f is linear and the set of constraints is specified using only linear equalities and inequalities. Nonlinear Programming studies the general case in which the objective function or the constraints or both contain nonlinear parts.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan10 Major Subfields (cont) Integer Programming studies linear programs in which some or all variables are constrained to take on integer values. This is not convex, and in general much more difficult than regular linear programming. Stochastic Programming studies the case in which some of the constraints or parameters depend on random variables.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan11 Major Subfields (cont) Calculus of Variations seeks to optimize an objective defined over many points in time, by considering how the objective function changes if there is a small change in the choice path. Combinatorial Optimization is concerned with problems where the set of feasible solutions is discrete or can be reduced to a discrete one.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan12 Major Subfields (cont) Heuristics and Meta-heuristics make few or no assumptions about the problem being optimized. Usually, heuristics do not guarantee that any optimal solution need be found. On the other hand, heuristics are used to find approximate solutions for many complicated optimization problems.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan13 Optimization Methods (Exact) Algorithms –An algorithm is sometimes described as a set of instructions that will result in the solution to a problem when followed correctly. –Unless otherwise stated, an algorithm is assumed to give the optimal solution to an optimization problem. –That is, not just a good solution, but the best solution.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan14 Optimization Methods (cont) Approximation Algorithms –Approximation algorithms (as opposed to exact algorithms) do not guarantee to find the optimal solution. –However, there is a bound on the quality, e.g., for a maximization problem, the algorithm can guarantee to find a solution whose value is at least half that of the optimal value. –We will not see many approximation algorithms here, but mention them as a contrast to heuristic algorithms.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan15 Optimization Methods (cont) Heuristic Algorithms –Heuristic algorithms do not guarantee to find the optimal solution. –Heuristic algorithms do not even necessarily have a bound on how bad they can perform. –However, in practice, heuristic algorithms (heuristics for short) have proven successful. –Near optimal solutions in reasonable time. –Most of this course will focus on this type of heuristic solution method.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan16 What Is Heuristic? Heuristic defined in Wikipedia –Heuristic (Greek: “Ε ὑ ρίσκω", "find" or "discover") refers to experience-based techniques for problem solving, learning, and discovery. Where the exhaustive search is impractical, heuristic methods are used to speed up the process of finding a satisfactory solution; mental short cuts to ease the cognitive load of making a decision.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan17 Why Not Always Exact Methods? The running time of the algorithm –For reasons explained soon, the running time of an algorithm may render it useless on the problem you want to solve. The link between the real-world problem and the formal problem is weak –Sometimes you cannot properly formulate a COP/IP that captures all aspects of the real-world problem. If the problem you solve is not the right problem, it might be just as useful to have one (or more) heuristic solutions, rather than the optimal solution of the formal problem.

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan18 Heuristic Optimization Methods Random Walk Hill-climbing Genetic Algorithms Particle Swarm Optimization Ant Colony Optimization Simulated Annealing Tabu Search Memetic Algorithm

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan19 A Challenge Maximize F 1 (x,y)

Spring, 2013C.-S. Shieh, EC, KUAS, Taiwan20