1. Paradigms in Autonomous Robotics CENG585, Fall 2010-2011
Instructor:
Erol Şahin

2. Overview Control Architectures for Autonomous Robots Three paradigms
Hierarchical
Reactive/Behavior-based
Hybrid

3. Artificial Intelligence
Artificial Intelligence
Beginnings of AI: Dartmouth Summer Research Conference (1955)
Original topics studied:
Language
Neural nets
Complexity theory
Self-improvement
Abstractions
Creativity
Marvin Minsky:
an intelligent machine “would tend to build up within itself an abstract model of the environment in which it is placed. If it were given a problem it could first explore solutions within the internal abstract model of the environment and then attempt external experiments.”
Adapted from Lynne Parker’s slides

4. AI influence on Autonomous Robotics
AI influence on Autonomous Robotics
Through mid-80’s, AI research strongly dependent upon:
Representational knowledge
Deliberative reasoning methods
Hierarchical organization
Adapted from Lynne Parker’s slides

5. Classical AI (Good-old-fashioned AI)
Classical AI (Good-old-fashioned AI)
Key characteristics:
The ability to represent hierarchical structure by abstraction
The use of “strong” knowledge using explicit symbolic representation
Beliefs:
Knowledge and knowledge representation are central to intelligence
Robotics is no exception
Focus:
Human-level intelligence
Not of interest:
Animal-level intelligence
Adapted from Lynne Parker’s slides

6. Three+ Control Paradigms
Three+ Control Paradigms
Adapted from Lynne Parker’s slides

7. Hierarchical Control Paradigm
Hierarchical Control Paradigm
Heavily influenced by the classical AI thinking:
Sense: Sense the environment and create a model of it. (Considered to be easy)
Plan: Given a goal state, find a path from the current state using planning methods. (Considered to be the essence of intelligence and hence the most challenging part)
Act: Command the actuators to execute the discovered plan. (Considered to be easy)
Also known as:
[Sense-Model-Plan-Act]
paradigm.
Adapted from Lynne Parker’s slides

8.
An exemplary problem
Mission: To build a robot that searches light sources in an environment, and, once found, stays close it. ?

9. Hierarchical Control Solution
Hierarchical Control Solution
Sense (and Model): Use the two light sensors to locate the light source’s position with respect to the robot as (x,y).
Plan: Take (x,y) as the goal position and plan a trajectory from the robot’s current position (0,0) to the goal position.
Act: Take the planned trajectory as input and drive the motors of the robot such that it follows the desired trajectory. ?

10. Braitenberg’s solution
Braitenberg’s solution
By default, the motors turn!
Each sensor is connected to the motor on the same side inhibiting the motor.
The vehicle
Heads toward light sources
speeds up in dark areas and slows down in light areas
spends more time in light and less time in dark.
The vehicle “loves” light.

11. Famous Problem Solver Task: “Missionaries and Cannibals”
Famous Problem Solver Task: “Missionaries and Cannibals”
For some unknown reason, three cannibals and three missionaries are traveling together and come to river that they must cross.  There is a boat on their side of the river that can be used, but unfortunately, it only holds two people at a time.   And, while the cannibals will cooperatively share the task of river crossing, if the number of cannibals on either bank is greater than the number of missionaries at any time, old habits will kick in and the outnumbered missionaries will be eaten!
Can you plan the crossings necessary to successfully get all 6 travelers across the river in good condition?  
Was the subject of first paper (Amarel, 1968) that approached problem formulation from analytical viewpoint
Adapted from Lynne Parker’s slides

12. Formalizing Missionaries and Cannibals Problem
Formalizing Missionaries and Cannibals Problem
Ignore all irrelevant parts of the problem (e.g., weather conditions, crocodiles, etc.)
Define states, operators, goal test, path cost:
States: ordered sequence of 3 numbers:
(# missionaries, # cannibals, #boats on initial riverbank)
E.g.: Start state = (3, 3, 1)
Operators:
Take 1 missionary, 1 cannibal, 2 missionaries, 2 cannibals, or one of each across in boat.
Take care to avoid illegal states
Goal test:
We’ve reached state (0, 0, 0)
Path cost:
# of crossings
Adapted from Lynne Parker’s slides

13. Solving Missionaries and Cannibals Problem
Solving Missionaries and Cannibals Problem
Adapted from Lynne Parker’s slides

14. Search Strategies for Problem Solvers
Search Strategies for Problem Solvers
Key criteria:
Completeness: is strategy guaranteed to find a solution when one exists?
Time complexity: how long does it take to find solution?
Space complexity: how much memory is needed to find solution?
Optimality: does strategy find highest-quality solution?
Adapted from Lynne Parker’s slides

15. Alternative Search Strategies for Problem Solvers
Alternative Search Strategies for Problem Solvers
Variety of search approaches:
Uninformed search (no information about path cost from current state to goal):
Breadth-first
Depth-first
Depth-limited search
Etc.
Informed search A*
Hill-climbing/gradient descent
Simulated annealing
Adapted from Lynne Parker’s slides

16. Depth-first searching
Depth-first searching
A depth-first search (DFS) explores a path all the way to a leaf before backtracking and exploring another path
For example, after searching A, then B, then D, the search backtracks and tries another path from B
Node are explored in the order A B D E H L M N I O P C F G J K Q
N will be found before J L M N O P G Q H J I K F E D B C A
Adapted from David Matuszek’s slides

17. Breadth-first searching
Breadth-first searching
A breadth-first search (BFS) explores nodes nearest the root before exploring nodes further away
For example, after searching A, then B, then C, the search proceeds with D, E, F, G
Node are explored in the order A B C D E F G H I J K L M N O P Q
J will be found before N L M N O P G Q H J I K F E D B C A
Adapted from David Matuszek’s slides

18. Planning-Based Approach to Robot Control
Planning-Based Approach to Robot Control
Job of planner: generate a goal to achieve, and then construct a plan to achieve it from the current state.
Must define representations:
Representation of actions: programs that generate successor state descriptions, defining preconditions and effects
Representation of states: data structure describing current situation
Representation of goals: what is to be achieved
Representation of plans: solution is a sequence of actions
First-order logic and theorem proving enable planning of strategies from start state to goal
Adapted from Lynne Parker’s slides

19. Propositional logic
a formal system in which formulas of a formal language may be interpreted as representing propositions.
A system of inference rules and axioms allows certain formulas to be derived, called theorems; which may be interpreted as true propositions.
The series of formulas which is constructed within such a system is called a derivation and the last formula of the series is a theorem, whose derivation may be interpreted as a proof of the truth of the proposition represented by the theorem.
Example:
P: the weather is cloudy. (true/false)
Q: it’s raining (true/false)
Q => P is also true.
Assume that P is true. Is Q true?

20. First Order Predicate Calculus in AI
First Order Predicate Calculus in AI
First order predicate calculus: formal language useful for making inferences and deductions
First-order logic is distinguished from propositional logic by its use of quantifiers; each interpretation of first-order logic includes a domain of discourse over which the quantifiers range.
Elementary components:
Predicate symbols (e.g., WRITES(), LIVES(), OWNS(), MARRIED())
Variable symbols (e.g., x, y)
Function symbols (e.g., father(x) returns the father of x)
Constant symbols (e.g., HOUSE, BUSH)
Connectives
and, or, negation, implies ∧∨¬⇒
Quantification
Universal ∀x meaning “For all x”
Existential ∃ x meaning “For some x”
Adapted from Lynne Parker’s slides

21. Situation Calculus Situation calculus:
Situation Calculus
Situation calculus:
Refers to a particular way of describing change in first-order logic
Conceives of world as consisting of a sequence of situations, each of which is a snapshot of the state of the world
All relations/properties of world that can change over time:
Specify extra situation argument to the predicate
Instead of: At(agent, location)
Indicate: At(agent, location, Si), where Si is a particular situation, or point in time
Represent how world changes from one situation to next:
Result(Forward, S0) = S1
Result(Turn(Right), S1) = S2
Results(Forward, S2) = S3
Adapted from Lynne Parker’s slides

22. Example of World Represented by Situation Calculus
Example of World Represented by Situation Calculus
Adapted from Lynne Parker’s slides

23. Early robots - Shakey Shakey (SRI), 1960’s One of first mobile robots
Early robots - Shakey
Shakey (SRI), 1960’s
One of first mobile robots
Sensors:
VidiconTV camera
Optical range finder
Whisker bump sensors
Environment: Office environment with specially colored and shaped objects
STRIPS planner: developed for this system
Used world model to determine what actions robot should take to achieve goals
Adapted from Lynne Parker’s slides

24.
Shakey the robot
1970-Shakey the robot reasons about its blocks Built at Stanford Research Institute, Shakey was remote controlled by a large computer. It hosted a clever reasoning program fed very selective spatial data, derived from weak edge-based processing of camera and laser range measurements. On a very good day it could formulate and execute, over a period of hours, plans involving moving from place to place and pushing blocks to achieve a goal.
From Hans Moravec
Adapted from Jeremy Wyatt’s slides

25. Shakey outline Planex Strips ILAs LLAs World Hardware Model
Shakey outline
Planex
Strips
ILAs
LLAs
Hardware
World
Model
Movie, hardware; LLAs; ILAs; until “and they incorporate powerful subroutines for analysing pictures and for navigating”
central representation
logic based
error recovery at several levels
communication through model
Adapted from Jeremy Wyatt’s slides

26. STRIPS-Based Approach to Robot Control
STRIPS-Based Approach to Robot Control
Use first-order logic and theorem proving to plan strategies from start state to goal
STRIPS language:
“Classical” approach that most planners use
Lends itself to efficient planning algorithms
Retains expressiveness of situation calculus
Shakey of SRI 1960’s
Adapted from Lynne Parker’s slides

27. STRIPS Representation -Actions
STRIPS Representation -Actions
STRIPS operators consist of:
Action description: name for what an agent does
Precondition: conjunction of positive literals (atoms) saying what must be true before operator can be applied
Effect: conjunction of literals (positive or negative) that describes how the situation changes when operator is applied
Organize effect as:
ADD LIST
DELETE LIST
Adapted from Lynne Parker’s slides

28. STRIPS Representation – States and Goals
STRIPS Representation – States and Goals
States: Conjunctions of function-free ground literals (i.e., predicates applied to constant symbols, possibly negated)
Example: At(Home) Have(Milk) Have(Bananas)
Common assumption: If state description does not mention a given positive literal, then the literal is assumed to be false
Goals: Conjunctions of literals, which can include variables
Example: At(x) Sells(x,Milk)
Assumption: variables are existentially quantified
Adapted from Lynne Parker’s slides

29. Simple Example of STRIPS Blocks-World Problem
Simple Example of STRIPS Blocks-World Problem
Goal State: ON(A,B)
Start state: ON(A, Table); ON(B, Table); EMPTYTOP(A); EMPTYTOP(B)
Operator:
MOVE(x,y):
Preconditions: ON(x,Table); EMPTYTOP(y)
Add-List: ON(x,y)
Delete-List: EMPTYTOP(y); ON(x,Table)
Adapted from Lynne Parker’s slides

30.
Shakey’s STRIPS World
Types of actions Shakey can make (at least in simulation):
Move from place to place:
Go(y):
PRECOND: At(Shakey,x)
In(x,r) ∧In (y,r)
On(Shakey,Floor)
EFFECT: At(y)
Push movable objects:
Push(b, x, y):
PRECOND: Pushable(b)
At(b,x)
At(Shakey,x)
In(x,r) ∧ In (y,r)
EFFECT: At(b,y)
Adapted from Lynne Parker’s slides

31. Shakey’s STRIPS World (con’t.)
Shakey’s STRIPS World (con’t.)
Types of actions Shakey can make (at least in simulation):
Climb onto rigid objects:
Climb(b):
PRECOND: Climbable(b)
At(Shakey,x) ∧ At(b,x)
On(Shakey,Floor)
EFFECT: On(Shakey,b)
Climb down from rigid objects:
Turn light switches on and off:
Adapted from Lynne Parker’s slides

32. Search strategy – first option
Apply all applicable operators to the initial world to create successor worlds.
Continue to apply applicable operators to these successors and their descendants (say in breadth-first fashion) until a model is produced in which the goal formula is a theorem.
Cons:
The number of applicable operators to any given world model can be high.
Hence the search tree can be very large and search would be impractical.

33. Search strategy - differences
Extract “differences” between the present world model and the goal
Then, identify operators that are “relevant” to reducing these differences.
Once a relevant operator is determined, attempt to solve the subproblem of producing a world model to which is is applicable.
If such a model is found, then apply the relevant operator and reconsider the original goal in the resulting model.

34. Goal checking
STRIPS uses a theorem prover to prove that the goal wff G0 follows from the set M0 of wffs describing the initial world model.
If G0 follows from the set M0, the task is trivially solved in the initial model.
Otherwise the theorem will fail to find a proof.
In this case, the uncompleted proof is taken to be the “difference” between M0 and G0.
Next, operators that might be relavent to “reducing” this difference are sought.
The effects of these operators would enable the proof to be continued.
In determining relevance, the parameters of the operators may be partially of fully instantiated.
The corresponding instantiated precondition wff schemata of the relevant operators are then taken to be the new subgoals.

35. Frame problem
initially formulated[1] as the problem of expressing a dynamical domain in logic without explicitly specifying which conditions are not affected by an action.
The name derives from a technique used cartoon makers called framing where the currently moving parts of the cartoon are superimposed on the "frame," which depicts the background of the scene, which does not change.
In the logical context, actions are typically specified by what they change, with the implicit assumption that everything else (the frame) remains unchanged.
[1]J. McCarthy and P. J. Hayes (1969). Some philosophical problems from the standpoint of artificial intelligence. Machine Intelligence, 4:

36. STRIPS: solving the frame problem
Solution through using Add and DELETE lists

37. STRIPS Representation --Plans
STRIPS Representation --Plans
A Plan is a data structure consisting of:
A set of plan steps
Each step is one of operators for the problem.
A set of step ordering constraints. Si ≺ Sj
Ordering constraints specify that an action has to occur sometime before another action
A set of variable binding constraints.
Variable constraints are of form v=x, where v is variable at some step, and x is either a constant or a variable
A set of causal links.
Record the purpose of the steps in the plan.
E.g., purpose of Si is to achieve the precondition c of Sj .
Adapted from Lynne Parker’s slides

38. Generating Plans in STRIPS
Generating Plans in STRIPS
Use principle of least commitment:
Make choices only regarding things you care about; leave others for later
Leave ordering of steps unspecified if it doesn’t matter for now
Plan in which some steps are ordered and others aren’t: partial order plan (as compared to atotal order plan):
“Instantiate”: bind a variable to a constant
Example:
At(Shakey,x) -> At(Shakey, Home)
Variable x is bound to constant “Home”
Fully instantiated plans:
Plans in which every variable is bound to a constant
Adapted from Lynne Parker’s slides

39. Partial Order vs. Total Order Plans
Partial Order vs. Total Order Plans
Adapted from Lynne Parker’s slides

40.
Solutions in STRIPS
Solution: plan that an agent can execute, and that guarantees achievement of goal
Easy way to make guarantee:
Require only fully instantiated, totally ordered plans
However, not satisfactory, because:
Easier for planners to return partially ordered plan than to arbitrarily select from alternative total orderings
Some agents can perform actions in parallel
Want to maintain flexibility of plan execution, in case other constraints arise later
Adapted from Lynne Parker’s slides

41. Solutions in STRIPS (con’t.)
Solutions in STRIPS (con’t.)
Therefore, solution is:
A Complete, Consistent plan
Complete plan:
One in which every precondition of every step is achieved by some other step
One in which preconditions are not “un-done” by another step prior to its need in a given step
Consistent plan:
One in which there are no contradictions in the ordering or binding constraints
Adapted from Lynne Parker’s slides

42. Summary of the Planning Problem
Summary of the Planning Problem
Planning agents use look-ahead to find actions to contribute to goal achievement
Planning agents differ from problem solvers in their use of more flexible representationof states, actions, goals, and plans
The STRIPS language describes actions in terms of preconditions and effects, capturing much of the expressive power of situational calculus
It is infeasible to search through space of situations; therefore, search through space of plans
Adapted from Lynne Parker’s slides

43. After Shakey After twenty years the approach still didn’t extend
After Shakey
After twenty years the approach still didn’t extend
Visual modelling too hard and slow
Non-linear planning intractable (NP-complete)
Feedback through world model cumbersome
People began to wonder if the ideas were right
Adapted from Jeremy Wyatt’s slides

44. After Shakey Shakey looked promising But it worked in a very
After Shakey
Shakey looked promising
But it worked in a very
restricted environment
Could it be extended to
natural worlds?
Play BBC video
Stanford Cart, 1970s
Adapted from Jeremy Wyatt’s slides

45. HILARE HILARE (LAAS-CNRS), 1970’s
HILARE
HILARE (LAAS-CNRS), 1970’s
Sensors:Video camera, 14 sonar, laser range finder
Three wheels:two drive, one caster
Weight:400 kg
World: smooth floors, office environment
Planning:
Conducted in multi-level geometric representational space
Use: for experimentation for over a decade
Adapted from Lynne Parker’s slides

46. Stanford Cart Stanford Cart, 1970’s (Moravec)
Stanford Cart
Stanford Cart, 1970’s (Moravec)
Sensors: Stereo vision used for navigation
Speed:Very slow, moving at about 1 meter per minutes
Full run: 5 hours
Obstacles: added to internal map as enclosing spheres
Search: Used graph search algorithm to find shortest path
Accomplishments: Successfully navigated complex 20-meter courses, visually avoiding obstacles
Adapted from Lynne Parker’s slides

47. CMU Rover CMU Rover, 1980’s Follow-on to Stanford Cart Sensors:
CMU Rover
CMU Rover, 1980’s
Follow-on to Stanford Cart
Sensors:
Camera mounted on pan/tilt
Infrared and sonar sensors
Actuators: Three independently powered/steered wheels
Accomplishments: Set stage for upcoming behavior-based robotics
Adapted from Lynne Parker’s slides