CS-424 Gregory Dudek Lecture 10 Annealing (final comments) Adversary Search Genetic Algorithms (genetic search)

CS-424 Gregory Dudek Comments on Assignment 2 Who’s gotten started? JAVA has a class for calling C functions. –See SUN’s java 1.1 web page. It’s open ended. Give yourself time to ponder. What are good heuristics. –Things in the URL? –Things in the page content of the “current node”? –Things in the content of the destination URL? Tricks are important. –(This is typical of AI in practice). Measure your performance. –This is always important! Do it scientifically. Try easy stuff first. –A good general rule.

CS-424 Gregory Dudek A comment On minimal length paths…. The Dynamic-Programming Principle [Winston]: “The best way through a particular, intermediate place is the best way to it from the starting place, followed by the best way from it to the goal. There is no need to look at any other paths to or from the intermediate place.” A variation on the A* theme called dynamic programming. Use the h instead of h’ and efficiently compute true path costs.

CS-424 Gregory Dudek Simulated Annealing Initially act more randomly. As time passes, we assume we are doing better and act less randomly. Analogy: associate each state with an energy or “goodness”. Specifics: –Pick a random successor s 2 to the current state s 1. –Compute ∂E = energy(s 2 ) - energy(s 1 ). –If ∂E good (positive) go to the new state. –If ∂E bad (negative), still take it sometime¨, with probability e ( ∂E/T ) Annealing schedule specifies how to change T over time.

CS-424 Gregory Dudek Adversary Search Basic formalism 2 players. Each has to beat the other (competing objective functions). Complete information. Alternating moves for the 2 players.

CS-424 Gregory Dudek The players Players want to achieve “opposite” goals: –What’s bad for A is good for B, and vice versa. With respect to our static evaluation functions, one wants to maximize the function, the other wants to minimize it. Use a game tree to describe the state space. Approach: MINIMAX procedure.

CS-424 Gregory Dudek Game Tree Nodes represent board configurations. Edges represent allowed moves from one configuration to another. For most 2-player games, alternating levels in the tree refer to alternating moves by the 2 players. Static evaluation function measures the (apparent) goodness of a given configuration. –Returns a signed value, to indicate goodness for the minimizing or maximizing player.

CS-424 Gregory Dudek See overhead (based on RN Figure )

CS-424 Gregory Dudek MINIMAX If the limit of search has been reached, compute the static value of the current position relative to the appropriate player. Return the result Otherwise, use MINIMAX on the children of the current position. –If the level is a minimizing level, return the minimum of the results –If the level is a maximizing layer, return the maximum of the results.

CS-424 Gregory Dudek MINIMAX observations The static evaluation function is… –Crucial: all decisions are eventually based of the value it returns. –Irrelevant: if we search to the end of the game, we always know the outcome. In practice, if same has a large search space then the static evaluation is especially important.

CS-424 Gregory Dudek Minimax Divide tree into plies Propagate information up from terminal nodes. Storage needs grow exponentially with depth.

CS-424 Gregory Dudek Alpha-Beta Search Can we avoid all the search-tree expansion implicit in MINIMAX? Yes, with a simple observation closely related to A * : Once you have found a good path, you only need to consider alternative paths that are better than that one. See example….

CS-424 Gregory Dudek The  “    ” ” 

CS-424 Gregory Dudek The  principle. “If you have an idea that is surely bad, do not take time to see how truly awful it is.” [Winston]

CS-424 Gregory Dudek  cutoff: How good is it? What is the best-case improvement for alpha-beta? What is the worst-case improvement for alpha-beta? Best case: only examine one leftmost “pre-terminal” nodes fully. Worst-case: –For some trees, it won’t help at all. –For some trees, even though it could help, it might not help if the nodes are encountered in the wrong order. Not in text

CS-424 Gregory Dudek Progressive deepening How do you deal with (potential) time pressure? Search progressively deeper: first depth 1, then depth 2, etc. As before, ratio of interior to leaf nodes is b d (b-1)/(b d -1) or roughly (b-1) [so if b=30, it only takes 1/30th extra time to do the progressive part of progressive deepening] Not in text

CS-424 Gregory Dudek Tricks Several techniques have proven important in practice. Many critical techniques lead to improvements that have no importance in terms of worst-case asymptotic complexity. Transposition tables Cache positions and their eventual outcome. Is a position re-occurs, don’t need to reevaluate it, since the information is in cache.

CS-424 Gregory Dudek Issues Horizon effect Search-until-quiescent heuristic Singular-extension heuristic Not in text

CS-424 Gregory Dudek Next major topic: LEARNING First, a transitional problem. How can we do hill climbing in a space of –representations ? –programs ? –entire systems ? e.g. Find the best program for (small) problem X. Find the best settings for the knowb on a refinery. Find the best LOGO program for a picture. Find the best organism encoded as DNA.

CS-424 Gregory Dudek GA’s Genetic Algorithms Cool, novel, controversial, costly (?), powerful (?). Inspired by real evolution. Idea: Encode the problem as a string. For example, a character string on a restricted alphabet. Define a fitness function.

CS-424 Gregory Dudek GA’s Define a population of strings to encode a set of alternative solutions to a problem. Now, we want to keep good solutions and improve them. Add operators for: –mixing strings –randomly changing strings Add an algorithm to repeatedly select the best, add variations, and evaluate.

CS-424 Gregory Dudek GA’s Genetic Algorithms Plastic transparencies & blackboard.