Some issues of design philosophy…

Slides:



Advertisements
Similar presentations
© The McGraw-Hill Companies, Inc., Chapter 8 The Theory of NP-Completeness.
Advertisements

NP-Complete Problems Reading Material: Chapter 10 Sections 1, 2, 3, and 4 only.
NP-Complete Problems Problems in Computer Science are classified into
EAs for Combinatorial Optimization Problems BLG 602E.
Integer Programming Difference from linear programming –Variables x i must take on integral values, not real values Lots of interesting problems can be.
© P. Pongcharoen ISA/1 Applying Designed Experiments to Optimise the Performance of Genetic Algorithms for Scheduling Capital Products P. Pongcharoen,
D Nagesh Kumar, IIScOptimization Methods: M1L4 1 Introduction and Basic Concepts Classical and Advanced Techniques for Optimization.
Ant Colony Optimization: an introduction
Metaheuristics The idea: search the solution space directly. No math models, only a set of algorithmic steps, iterative method. Find a feasible solution.
Informed Search Methods
CHAPTER 12 ADVANCED INTELLIGENT SYSTEMS © 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang.
The Theory of NP-Completeness 1. Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking.
Genetic Algorithms and Ant Colony Optimisation
1 Local search and optimization Local search= use single current state and move to neighboring states. Advantages: –Use very little memory –Find often.
NEURAL NETWORKS FOR DATA MINING
Search Methods An Annotated Overview Edward Tsang.
Local Search: walksat, ant colonies, and genetic algorithms.
Exact and heuristics algorithms
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
Neural Networks and Machine Learning Applications CSC 563 Prof. Mohamed Batouche Computer Science Department CCIS – King Saud University Riyadh, Saudi.
NP-Complete Problems. Running Time v.s. Input Size Concern with problems whose complexity may be described by exponential functions. Tractable problems.
Machine Learning A Quick look Sources: Artificial Intelligence – Russell & Norvig Artifical Intelligence - Luger By: Héctor Muñoz-Avila.
CSE 589 Part V One of the symptoms of an approaching nervous breakdown is the belief that one’s work is terribly important. Bertrand Russell.
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
© P. Pongcharoen CCSI/1 Scheduling Complex Products using Genetic Algorithms with Alternative Fitness Functions P. Pongcharoen, C. Hicks, P.M. Braiden.
Artificial Intelligence By Mr. Ejaz CIIT Sahiwal Evolutionary Computation.
Constraints in Evolutionary Algorithms. Constraints: the big questions, page 233  how to evaluate and compare feasible and infeasible solutions  avoid,
Genetic Algorithms. Solution Search in Problem Space.
1 Intro to AI Local Search. 2 Intro to AI Local search and optimization Local search: –use single current state & move to neighboring states Idea: –start.
ICS 353: Design and Analysis of Algorithms NP-Complete Problems King Fahd University of Petroleum & Minerals Information & Computer Science Department.
The Theory of NP-Completeness
Chapter 10 NP-Complete Problems.
Hard Problems Some problems are hard to solve.
Richard Anderson Lecture 26 NP-Completeness
Differential Evolution
School of Computer Science & Engineering
Introduction to Genetic Algorithm (GA)
C.-S. Shieh, EC, KUAS, Taiwan
The Propositional Calculus
Richard Anderson Lecture 26 NP-Completeness
Hard Problems Introduction to NP
Chapter 6: Genetic Algorithms
CSC 380: Design and Analysis of Algorithms
Subject Name: Operation Research Subject Code: 10CS661 Prepared By:Mrs
ICS 353: Design and Analysis of Algorithms
Richard Anderson Lecture 25 NP-Completeness
SAT-Based Area Recovery in Technology Mapping
Introduction to Pattern Recognition
Richard Anderson Lecture 28 Coping with NP-Completeness
Metaheuristic methods and their applications. Optimization Problems Strategies for Solving NP-hard Optimization Problems What is a Metaheuristic Method?
Multi-Objective Optimization
Chapter 11 Limitations of Algorithm Power
NP-Complete Problems.
Design & Analysis of Algorithms Combinatorial optimization
Artificial Intelligence Lecture No. 28
Boltzmann Machine (BM) (§6.4)
CSE 6408 Advanced Algorithms.
NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979.
The Theory of NP-Completeness
Artificial Intelligence CIS 342
A Tutorial (Complete) Yiming
학습목표 공진화의 개념을 이해하고, sorting network에의 응용가능성을 점검한다
Dr. Arslan Ornek DETERMINISTIC OPTIMIZATION MODELS
Traveling Salesman Problem by Genetic Algorithm
Instructor: Aaron Roth
CSC 380: Design and Analysis of Algorithms
Our old list of problems
Beyond Classical Search
CSC 380: Design and Analysis of Algorithms
Presentation transcript:

Some issues of design philosophy… Basic Concepts Some issues of design philosophy…

Memory-Processor Link and the von Neumann Bottleneck Processor = CPU Memory = hierarchy of caches, main memory, and bulk storage Connection = address, control, data busses Source of contention = bus traffic during the FETCH/EXECUTE cycle 8/23/2019 Copyright G.A. Tagliarini, PhD

Memory-Processor Link Biological systems Memory unit = processor Large numbers of independent but interacting processors (circa 1010 neurons) Small computational complexity Massive connectivity (circa 1012 synapses) Connections and state represent solutions 8/23/2019 Copyright G.A. Tagliarini, PhD

Common Characteristics of BIP Parallel distributed processing Rummelhart, McClelland and Hinton Stigmergy Feedback Negative Positive 8/23/2019 Copyright G.A. Tagliarini, PhD

Copyright G.A. Tagliarini, PhD Emergent Behavior Adaptation Learning Supervised – a teacher instructs using a set of examples and the student generalizes to produce a representation Unsupervised – examples are presented and a rule for forming group representations may be given Evolution Fitness/selective pressure Crossover/recombination Mutation 8/23/2019 Copyright G.A. Tagliarini, PhD

Emergent Behavior (continued) Self-organization Cooperation Coordination Order arises without external intervention Contrast with Templates Supervisors Instructions/blueprints Key question: How to formulate behaviors that give rise to self-organization? 8/23/2019 Copyright G.A. Tagliarini, PhD

Example: How to Get Ants to Cooperate when Foraging Suppose ants have been endowed with search behaviors and individual carrying capacity? Allow the reward structure to be changeable What behaviors or conditions would elicit cooperation? 8/23/2019 Copyright G.A. Tagliarini, PhD

A Search Problem: Satisfiability Given A set of m logic variables {p1, p2, …, pm} A collection of n disjunctive clauses, e.g., ci = (p1 + p2’ + p5) Place the clauses in conjunctive normal form (CNF) to create c = c1 c2 …ci …cn Question Is there an assignment of truth values that satisfies c? “…satisfies c…” means “…for which c is true…” See further: http://en.wikipedia.org/wiki/Boolean_satisfiability_problem 8/23/2019 Copyright G.A. Tagliarini, PhD

An Optimization Problem: Maximum Satisfiability Given A set of m logic variables {p1, p2, …, pm} A collection of n disjunctive clauses, e.g., ci = (p1 + p2’ + p5) Place the clauses in conjunctive normal form (CNF) to create c = c1 c2 …ci …cn Question What is the assignment of truth values that maximizes the number of clauses satisfied? 8/23/2019 Copyright G.A. Tagliarini, PhD

An Optimization Problem: Weighted Maximum Satisfiability Given A set of m logic variables {p1, p2, …, pm} A collection of n disjunctive clauses, e.g., ci = (p1 + p2’ + p5) Each clause ci has a corresponding nonnegative weight wi Question What assignment of truth values maximizes the sum of the weights of the satisfied of clauses? See further: http://web.udl.es/usuaris/m4372594/MAX-SATweb.html 8/23/2019 Copyright G.A. Tagliarini, PhD

General Problem Solving Concerns Goal Representation Constraints (How do I know I have a solution?) “No free lunch theorem” (Wolpert and Macready, 1997) 8/23/2019 Copyright G.A. Tagliarini, PhD

Copyright G.A. Tagliarini, PhD Goal Search Find any solution; time to verify a solution bounded by some polynomial E.g., queen placement, knight’s tour, map coloring, SAT Leads to a Boolean assessment function Optimize Find solutions of a given “quality” (Maximize/Minimize) E.g., TSP, weighted MAX-SAT, WTA, QAP, network flow 8/23/2019 Copyright G.A. Tagliarini, PhD

Copyright G.A. Tagliarini, PhD Representation Choosing a feature space Telling time: rectangular versus polar coordinates Speech recognition: time domain versus frequency domain Face recognition: templates, eigenfaces 8/23/2019 Copyright G.A. Tagliarini, PhD

Copyright G.A. Tagliarini, PhD Constraints How do I know I have a solution? Linear Nonlinear Inequality Integer Zero/One 8/23/2019 Copyright G.A. Tagliarini, PhD

Why Can’t I Just Learn One Problem Solving Method? “No free lunch theorem” (Wolpert and Macready, 1997) No single optimization method will perform best on all instances of all problems. 8/23/2019 Copyright G.A. Tagliarini, PhD

Two Conventional Benchmark Techniques Hill Climbing Simulated Annealing 8/23/2019 Copyright G.A. Tagliarini, PhD

Hill Climbing (Minimization) Assume: Candidate solutions → s0 and s1 in a space S Goal → g Fitness function f such that f:S →R+ Repetition limit → L Algorithm Initialize s0 = s1, f0 = f(s0), f1 = f(s1), Counter While (counter<L, f0>g) s1 = mutate(s0), f1 = f(s1) If f1 < f0 (Note: Improvements are always kept) s0 = s1 f0 = f(s0) Increment counter 8/23/2019 Copyright G.A. Tagliarini, PhD

Stochastic Hill Climbing (Minimization) Assume: Candidate solutions, goal, fitness function, solution space, and iteration limit as in basic hill climbing A scaling parameter T Algorithm Initialize While (counter<L, f0>g) s1 = mutate(s0), f1 = f(s1) Df = (f0 – f1) If rand >1/ (1+exp(Df/T) (Note: Improvements are often kept) s0 = s1 f0 = f(s0) Increment counter 8/23/2019 Copyright G.A. Tagliarini, PhD

Copyright G.A. Tagliarini, PhD Simulated Annealing http://people.uncw.edu/tagliarinig/Courses/340/Lectures/Simulated%20Annealing.ppt#1 8/23/2019 Copyright G.A. Tagliarini, PhD

Evolutionary Algorithms Based on analogy to evolution in biological systems Key process elements Fitness function “Artificial selection” Crossover and recombination Mutation 8/23/2019 Copyright G.A. Tagliarini, PhD