1. Means-ends analysis Northwestern University CS 395 Behavior-Based Robotics Ian Horswill

2. Review Robot operates in an environment  State space S  Set of possible motor outputs A  Dynamics (physics) that determines how the environment changes state  Continuous dynamics (continuous-time actions) f = dS/dt: S  A  S, f = d 2 S/dt 2 : S  A  S, etc.  Discrete dynamics (atomic/ballistic actions, discrete time)  : S  A  S

3. Review Want to construct a policy to make the robot do the right thing  p: S  A  Complete environment-robot system evolves  Continuous case: curve through state-space ds/dt = f(s,p(s))  Discrete case: system evolves through series of states s 0 s 1 =  (s 0, p(s 0 )) s 2 =  (s 1, p(s 1 )) =  (  (s 0, p(s 0 )), p(  (s 0, p(s 0 )))) Etc.

4. Error feedback control  Goal state s g  Control action computed from error  ds/dt = f(s- s g )  d 2 s/dt 2 = f(s- s g )  Linear feedback control  f is a linear operator  ds/dt = k(s- s g )  P control (proportional control)  k is a gain you multiply by  k is a matrix when s is a vector  d 2 s/dt 2 = k p (s- s g )+ k d ds/dt + k i ∫s dt  PID control

5. Behavior-based control (“Bottom-up”) Combine policies by running them in parallel  Behavior = policy + trigger  Bottom-up integration of behaviors  Map several behaviors to a single composite behavior (or composite policy)  Several different composition operators  Behavior-or (prioritization/subsumption)  Behavior-+ (motor schemas/potential fields)  Behavior-max  Weighted voting  Etc.

6. Plan-based control (“Top-down”) Combine policies by running them serially  Behaviors → atomic actions  Still policy + activation level  Externally triggered  Self-terminating  Combine behaviors using serial controllers (plans)  Finite state machines  Individual states can Trigger actions Wait for them to terminate Wait for other external conditions Etc.

7. Planning-based control (“Top-down”) Combine policies “non-deterministically”  Idea: “guess the action that will ultimately work”  i.e. guess the one that leads to the goal  Problem: this doesn’t help much  Don’t know which action(s) will ultimately work  If you guess wrong, you’re screwed  Solution: simulation  Run actions in simulation  Search through possible sequences of actions (plans) to find one that works and remember it  Execute the successful plan in the real world

8. Logic-based representations of the state space  Represent states using propositions (true/false statements)  Find a set of propositions that let you distinguish all the states you care about  State = truth of each proposition  Advantage: partial state descriptions  P^Q is the set of states in which both P and Q are true

9. Means-ends analysis  Pair goals up with actions  For each proposition, keep track of the actions that can make it true  For each action, write the precondition (partial state descriptions) for being able to run it  To solve the goal P^Q  Look up the action A that achieves P  Recursively solve precondition(A)  Run A  Recursively solve Q without “clobbering” P

10. GPS (Newell and Simon)  “General Problem Solver”  Used means-ends analysis  Assumed priority ordering on propositions  Algorithm: GPS(goal) while goal not yet true p = highest priority unsatisfied subgoal (subgoal = proposition inside of goal) a = action to solve p GPS(precondition(a)) do a

11. The STRIPS representation Define actions in terms of  Add list: propositions the action makes true  Delete list: propositions the action may make false  Precondition list: propositions that must be true in order to run the action

12. Planning with STRIPS  Goal = set of propositions to make true  Algorithm: STRIPS(initial, goal) for each subgoal p in goal not in initial for each action a with p in its add list try the plan: STRIPS(initial, precondition(a)) a STRIPS(initial-delete_list(a) +add_list(a), goal) if both recursive calls worked, we win else, try another action, or another subgoal

13. Reactive planning in fully reactive systems  Collection of behaviors  Each behavior achieves some goal(s)  Each behavior has some precondition(s)  Higher level system drives some goal signal  Goal signals  Activate behaviors that achieve them  Drive goal signals of preconditions  Examples:GAPPS (Kaelbling 90), Behavior Networks (Maes 90)

14. GRL example  Extended STRIPS representation  Operators are just behaviors that activate themselves when they can achieve a goal. ( define-operator name motor-vector (achieves add-list-signals … ) (clobbers delete-list-signals …) (preconditions precondition-signals... ) (serial-preconditions precondition-signals... ) (required-resources names... ) (priority number ))

15. Computing activation-levels  A behavior is runnable if all its preconditions are satisfied  It is desirable if  It satisfies a maintenance goal  It satisfies some unsatisfied goal of achievement  It is unconflicted if  It doesn’t clobber a satisfied goal or a maintenance goal, and  None of its resources are required by desirable operators of higher priority

16. Compile-time property lists (define-signal-property (add-list x) '()) (define-signal-property (delete-list x) '()) (define-signal-property (preconditions x) '()) (define-signal-property (serial-preconditions x) '()) (define-signal-property (priority x) 0) (define-signal-property (required-resources x) '())

17. Making the behavior (letrec ((the-behavior (behavior (run? the-behavior) motor-vector-signal ))) the-behavior) (define-signal (run? x) (and (desirable? x) (runnable? x) (not (conflicted? x))))

18. Computing desirable? (define-signal-modality (desirable? x) (define adds (add-list x)) (signal-expression (or (apply or (unsatisfied-goal adds)) (apply or (maintain-goal adds))) (drives (goal (runnable? x))) (operator x)))

19. Computing runnable? (define-signal-modality (runnable? x) (signal-expression (parallel-and (apply parallel-and (preconditions x)) (apply serial-and (serial-preconditions x))

20. Gatherer functions  Accumulators can be declared with a gatherer  The gatherer is called by the compiler with a list of all signals being compiled  It returns the signals that should be used as inputs to the accumulator  Gatherers are called after signal expansion  They’re only passed the list of primitive signals into which calls to signal procedures have been expanded

21. Computing conflicted? (define-signal-procedure (conflicted? x) (define my-priority (priority x)) (let ((high-priority-clobbered-goals (filter (lambda (g) (>= (priority g) my-priority)) (delete-list x)))) (signal-expression (apply or (accumulate or (make-conflict-set-gatherer x)) (satisfied-goal,high-priority-c-goals)))))

22. Computing conflicted? ;;; A signal is conflicted if some other higher priority signal needs ;;; one of its resources or if it clobbers a goal we’ve already achieved. ;;; Since confliced? already checked for clobbering, we only need to ;;; worry about finding operators that need our resources. (define (make-conflict-set-gatherer me) (lambda (ignore signal-list) (define resources (required-resources me)) (define (desired-resource? r) (memq r resources)) (define (steals-my-resource? op) (any desired-resource? (required-resources op))) … ))

23. Computing conflicted? (define (make-conflict-set-gatherer me) (lambda (ignore signal-list) … (define my-priority (priority x)) (define (higher-priority? op) (define pri (priority op)) (cond ((= pri my-priority) (error "Conflict detected between actions of equal priority“ me op)) (else (> pri my-priority)))) …))

24. Computing conflicted? (define (make-conflict-set-gatherer me) (lambda (ignore signal-list) … (define (conflictor? desirable?-sig) (define op (operator desirable?-sig)) (and (not (eq? op me)) (steals-my-resource? op) (higher-priority? op))) ;; Now really compute the list of inputs (filter conflictor? signal-list))

25. A pathetic example (define-operator make-happy 'running-make-happy (achieves happy?) (preconditions (maintain content?)) (requires-resources a)) (define-operator make-happy2 'running-make-happy2 (achieves happy2?) (serial-preconditions not-depressed? content?) (requires-resources a b) (priority 5)) (define-operator make-content 'running-make-content (achieves content?) (clobbers happy?) (priority 0))

26. A pathetic example (define-operator make-not-depressed 'running-make-not-depressed (achieves not-depressed?)) (define-signal doit (behavior-or make-happy make-happy2 make-content make-not-depressed))

27. Compiled code > (compile doit) (begin (define (run) (update-grl-time!) (let* ((desirable?-make-happy #f) …) (while #t (update-grl-time!) (before-signal-update) (set! desirable?-make-happy (and want-happy? (not happy?))) (set! desirable?-make-happy2 (or (and want-happy2? (not happy2?)) stay-happy2?)) (set! make-happy-activation-level (and desirable?-make-happy content? (not desirable?-make-happy2))) (set! make-happy2-activation-level (and desirable?-make-happy2 not-depressed? content? (not #f))) (set! make-content-activation-level (and (or (and desirable?-make-happy2 not-depressed? (not content?)) desirable?-make-happy) (not (and want-happy? happy?)))) (set! doit-activation-level (or make-happy-activation-level make-happy2-activation-level make-content-activation-level (and desirable?-make-happy2 (not not-depressed?) (not #f)))) (set! doit-motor-vector (cond (make-happy-activation-level 'running-make-happy) (make-happy2-activation-level 'running-make-happy2) (make-content-activation-level 'running-make-content) (else 'running-make-not-depressed))) (after-signal-update)))))