1. Another CPS example: mathematical reasoning The problem: given a mathematical equation, find its solution. Problem space: nodes represent mathematical equations, arcs represent algebraic laws transforming one equation into another. Example: log e (x+1) + log e (x-1) = c log w U + log w V log w U * V log e (x+1) * (x-1) = c (U + V) * (U - V) U 2 - V 2 log e (x 2 - 1) = c log U V = W V = U W x 2 - 1 = e c U - V = W U = W + V x 2 = e c + 1 U2 = V U = +/- (sqr V) x = +/- (sqr (e c + 1))

2. How to reduce the complexity of the search? We can apply "expert knowledge" to choose the best operator at each step.To make this possible, we must: 1.Explicate implicit expert knowledge. 2.Formally represent it. Alan Bundy suggested that mathematicians use three categories of methods controlling the use of algebraic laws: 1.Attraction methods. 2.Collection methods 3.Isolation methods These methods are used to analyze how a given law transforms an equation, and whether a transformation leads towards the goal. They can be represented as production rules, and using pattern-matching and forward chaining the problem space can be efficiently searched.

3. Attraction methods Attraction methods move occurrences of the unknown closer together. Example: Consider the following algebraic law: W * U + W * V --> W * (U + V). Assume that only U and V contain the unknown, x. The initial expression is represented The transformed expression is represented by the following tree: by the following tree: + * * * W + W U W V U V The distance between occurrences of x is the number of steps required to move through the tree from one occurrence to another. Other algebraic laws belonging to this category are:  log w U + log w V --> log w U * V  (W U ) V --> W UV  U VW --> (U V ) W

4. Attraction methods (cont.) Attraction methods can be applied only to the least dominant term in the equation, i.e. to a term containing at least two subterms containing x. Example: [log (x) * x p ] + [(x - 1) * (x + 1)] + q each of the two subterms contain two occurrences of x. Expression, + also contains more than two occurrences of x.Therefore, this expression has three least dominant terms to which attraction methods can be applied:  log(x) * x p  (x - 1) * (x + 1)  [log (x) * x p ] + [(x - 1) * (x + 1)]

5. Collection methods Collection methods reduce the number of occurrences of the unknown in the equation. Example: Assume that U and V contain occurrences of the unknown, x, in the following law (U + V) * (U - V) --> U 2 - V 2 Trees describing the left hand side (lhs) and the right hand side (rhs) of this transformation are: * - + - ^2 ^2 U V U V U V Another algebraic law belonging to this category is U * W + U * Y --> U * (W + Y) ; assume that only U and V contain x. Collection methods are also applied to least dominant terms in the expression.

6. Isolation methods Isolation methods reduce the depth of the occurrences of the unknown, x. They are applied to the whole equation. Example: Assume that only U contains occurrences of the unknown, x, in the following law U - W = Y --> U = Y + W Trees describing the left hand side (lhs) and the right hand side (rhs) of this transformation are: = = - Y U + U W Y W Two more algebraic laws belong to this category (only U contains x): log W U = Y --> U = W Y U 2 = W --> U = +/- (sqrt W)

7. Implementation of a CPS for algebra problems 1. Representation of states: assuming that an equation is described in a prefix form, we can use a list to represent it. For example, the equation log e (x+1) + log e (x-1) = c can be represented as the following list: (= ) (= (+ (log (+ x 1) e) (log (- x 1) e)) c ) 2. Representation of operators: ( ) symbol returns a list of pairs To implement operator procedures, we need: i.A way to match components of the equation to those in a specific law being explored (i.e. we need a pattern-matching procedure) ii.A substitution procedure to use the results of the pattern-matching process.

8. Example Consider the following isolation law represented as a pattern, with W, U, Y being pattern variables: log W U = Y --> U = W Y The lhs expression can be represented as follows: (= (log (? arg) (? base)) (? rhs)) To match this pattern to a state means to define specific values for pattern variables, (? arg), (? base), (? rhs). Assume that these values are stored in a dictionary of bindings. Then, the rhs expression, U = W Y, represented as (= (? arg) (expt (? base) (? rhs))) can be computed by means of the following substitution procedure: Matching phase (= (log (? arg) (? base)) (? rhs)) Operator log W U = Y --> U = W Y is applied to generate a new state Substitution phase (= (? arg) (expt (? base) (? rhs)))

9. Additional constraints on the pattern-matching process 1. Some of pattern variables contain instances of x, while others do not. To account for this property, we can expand the representation of pattern variables as follows: (? patt-var contains-x?) (? patt-var no-x?) Example: log W U = Y --> U = W Y will be represented as follows (= log (? arg contains-x?) (? base no-x?) (? rhs no-x?) 2. There are laws that are always worth applying, because they reduce the complexity of the expression. For example:  U + 0 --> U  U * 0 --> 0  U * 1 --> U  If possible, replace a sub-term by its values. Example: sub-term (+ 2 2) can be replaced by its value, 4, thus reducing the complexity of the expression.

10. To use the same pattern-matching mechanism in applying these simplifying rules, we must account for the following cases: … + 0 + … --> … * 1 * … --> … * 0 * … --> 0 segment variables Let us call whatever comes before and after 0 and 1 terms segment variables. Segment variables may match more than one element on a list. To represent them, we use (?? segm-var). For example, (+ (?? pre) 0 (?? post)) --> (+ (?? pre) (?? post)) Segment variables allow a pattern to match an expression in more than one way. For Example, expression (a b foo foo foo c d) can be represented as ((?? before) foo (?? after)), where the following are several possible variable bindings: (?? before) = (a b) | (a b foo) | (a b foo foo) (?? after) = (foo foo c d) | (foo c d) | (c d)

11. To summarize, the pattern matcher must recognize two types of variable: 1.Segment variable. These match zero or several elements on a list, and start with ?? 2.Element variables. These match exactly one element on a list, and are represented as (? ) To handle pattern variables we need the following functions (refer to match.lsp):  pattern-variable?  element-variable?  segment-variable?  var-name  var-restriction  var-value  lookup-var  bind-element-var  bind-segment-var

12. One more detail that must be addressed is substituting expressions with their values. This can be implemented as follows: (+ (? num1 numberp) (? num2 numberp) (?? terms)) these two are to be added (+ (:eval (+ (? num1) (? num2))) (?? tems)) :eval is a special form which computes the result of the expression and places it in the dictionary. The simplification rule now becomes: (+ (? num1 numberp) (? num2 numberp) (?? terms)) --> (+ (:eval (+ (? num1) (? num2))) (?? tems))

13. Consider the following expression (+ A 2 3) Note that it will not match the simplification rule, but the equivalent expression (+ 2 3 A) will. To handle such cases we must be able to rearrange the expression bringing its numerical terms before non-numerical terms. This can be done by introducing a relationship "less than" on the order of term, where: 1.Numerical terms are "less than" non-numerical. 2.The expression is sorted with respect to this relationship. This is implemented by the form :splice. To run the CPS for solving algebra problem, you need the following files: search.lsp, variants.lsp, algebra.lsp, simplify.lsp, match.lsp A specific example is defined in algebra.lsp (defvar *bundy* '(= (+ (log (+ x 1) E) (log (- x 1) E)) C) "A single example problem from Alan Bundy's reasoner.") See book page 60 for example runs on the program.

14. Example (book, page 60) Function setup-algebra-problem in algebra.lsp defines a particular problem (same as in setup-subway-problem) (defun setup-algebra-problem () "Create an generic algebra 'problem space'." (make-problem :NAME 'Algebra :GOAL-RECOGNIZER 'got-algebra-goal? :OPERATOR-APPLIER 'find-algebra-operator :STATE-PRINTER #'(lambda (f) (format nil "~A" f)) :SOLUTION-ELEMENT-PRINTER #'print-derivation-step :STATES-IDENTICAL? 'equal :DISTANCE-REMAINING 'algebra-distance :OPERATORS '((Isolate-Log try-isolate-log) (Isolate-Sum try-isolate-sum) (Isolate-Difference try-isolate-difference) (Isolate-Square try-isolate-square) (Collect-Product-Difference try-collect-prod-diff) (Attract-Log-Sum try-attract-log-sum) (Canonicalize try-canonicalization))))

15. Implementation of the pattern-matcher Consider the following expression and the pattern it matches: (+ (log (+ x 1) e) (log (- x 1) e) (+ (log (? U has-unknown?) (? W no-unknown?)) (log (? V has-unknown? (? W)) The following substitutions are required for these matches: (V (- x 1)) Assume that these substitutions are stored in a (W e) dictionary of bindings as an association list. (U (+ x 1)) The dictionary has the following format: ((patt_var1 value) (patt_var2 value) …) Given the key, patt_varN, the corresponding value can be accessed via the assoc primitive: (assoc (var-name var) dictionary), where (var-name var) is a procedure to get to the name of pattern variable var in the dictionary.

16. The match function The match function takes the following parameters: pat, dat, and the current dictionary, and returns the new extended dictionary. It must consider the following cases:  A component is found for which the pattern and the expression do not match, therefore the match fails.  pat is a symbol  pat is an element variable  pat is an empty list  pat is a segment variable  pat is a list, but dat is an atom, therefore the match process fails.  If none of these is the case, match must return the dictionary accumulated so far.

17. The substitution process As a result of the substitution process, a new expression is created. This new expression represents the rhs of the simplification rule being applied. To carry out this process, we need:  the dictionary generated during the matching process,  the expression constituting the rhs of the simplification rule. Example: Assume that (?? A) has the value (1 2), and (?? B) has the value (3 4). Then:  expression (+ (?? A) (?? B)) is transform by the substitution process into (+ (1 2) (3 4)).  (:eval ((?? A) (?? B)) returns 10.  (* (:splice (42 (?? A) (?? B)))) returns (* 42 1 2 3 4)

18. Implementation of the simplifier Consider the following expression: (+ (* x -2) (* x 2)). It is always 0, which is why it must be eliminated as early as possible. A simplification of this type can be performed by a set of rewrite rules of the form ( ).Note that knowledge needed to recognize situations where simplification is possible must be acquired and embedded in the PS. There are 4 categories of simplification rules: 1.Rules for recognizing special arguments to operators. Example: Rule: 0 + U --> U. Encoded: ((+ (? Zero zero?) (?? E)) (+ (?? E))) Rule: e 0 --> 1. Encoded: ((expt (? E) (? Zero zero?)) 1) 2. Rules dealing with equivalences involving multiplication, squaring and exponent. Rule: (sqrt (U 2 )) --> |U|. Encoded: ((sqrt (sqr (? E))) (abs (? E))) 3. Rules for reducing expressions if numerical values are available. Examples: Rule: A / B --> value. Encoded: ((/ (? e1 numberp) (? e2 numberp)) (:eval (/ (? e1) (? e2)))) Rule: A + (B + C) --> A + B +C. Encoded: (((? op =/*?) (?? e1) ((? op) (?? e2) (?? e3)))((? op) (?? e1) (?? e2) (?? e3))) 4. Rules for producing canonical forms. Example: (+ a 2 3) --> (+ 2 3 a)

19. Problems with the CPS model In 1957, Herbert Simon wrote: "… It is not my aim to surprise or shock you … But the simplest way I can summarize is to say that there are now in the world machines that think, that learn and create. Moreover, their ability to do these things is going to increase rapidly - in a visible future - the range of problems they can handle will be coextensive with the range to which the human mind has been applied…“ Was Herbert Simon right? The short answer is NO. And the short explanation is that the CPS-type systems are based on the idea that there exists a general Problem solving method that can be applied to any problem requiring human intelligence. 10 years were needed to realize that the real power of human intelligence is not the problem solving per se, but knowledge behind the problem solving process. In CPS, knowledge is embedded implicitly (like in conventional computer programs), although it is separated from the search engine. To build a powerful AI program, we need a way to represent knowledge explicitly. This is true for both, domain knowledge and control knowledge.